Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Importing camt.053 file fails during conversion step #8574

Closed
4 tasks done
laeckerv opened this issue Feb 22, 2024 · 6 comments
Closed
4 tasks done

Importing camt.053 file fails during conversion step #8574

laeckerv opened this issue Feb 22, 2024 · 6 comments
Labels
bug Verified and replicated bugs and issues. fixed Bugs that are fixed (in a coming release).
Milestone
importer-v1.4.5

Comments

@laeckerv
Copy link

Support guidelines

I've found a bug and checked that ...

Description

When im trying to import a camt.053 file the importer stops during the conversion. If needed I can try to get a minimal anonymized example of the xml file...

Debug information

Debug information generated at 2024-02-22 22:02:46 Europe/Zurich for Firefly III Data Importer version 1.4.4.

System information
ItemValue
Version1.4.4
Build#, base #80
SystemPHP 8.3.3, Linux, apache2handler
App information
ItemValue
TimezoneEurope/Zurich, Europe/Zurich
Environmentlocal
Debug modetrue, cache 'file'
Log leveldebug, stack
Display errorsOff, ALL errors
BCscale12
Trusted proxies**
User information
ItemValue
User agentMozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:122.0) Gecko/20100101 Firefox/122.0

Expected behaviour

The conversion shouldn't fail.

Steps to reproduce

  1. import the camt.053 file

Additional info

Truncated from this point <----|) must be of type string, null given at /var/www/html/app/Services/Camt/Conversion/TransactionMapper.php:595)
[stacktrace]
#0 /var/www/html/app/Services/Camt/Conversion/TransactionMapper.php(595): bccomp()
#1 /var/www/html/app/Services/Camt/Conversion/TransactionMapper.php(349): App\\Services\\Camt\\Conversion\\TransactionMapper->sanityCheck()
#2 /var/www/html/app/Services/Camt/Conversion/TransactionMapper.php(45): App\\Services\\Camt\\Conversion\\TransactionMapper->mapTransactionGroup()
#3 /var/www/html/app/Services/Camt/Conversion/RoutineManager.php(117): App\\Services\\Camt\\Conversion\\TransactionMapper->map()
#4 /var/www/html/app/Http/Controllers/Import/ConversionController.php(183): App\\Services\\Camt\\Conversion\\RoutineManager->start()
#5 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Controller.php(54): App\\Http\\Controllers\\Import\\ConversionController->start()
#6 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php(43): Illuminate\\Routing\\Controller->callAction()
#7 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Route.php(259): Illuminate\\Routing\\ControllerDispatcher->dispatch()
#8 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Route.php(205): Illuminate\\Routing\\Route->runController()
#9 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(806): Illuminate\\Routing\\Route->run()
#10 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(144): Illuminate\\Routing\\Router->Illuminate\\Routing\\{closure}()
#11 /var/www/html/app/Http/Middleware/IsReadyForStep.php(44): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#12 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): App\\Http\\Middleware\\ConversionControllerMiddleware->handle()
#13 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Middleware/SubstituteBindings.php(50): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#14 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Routing\\Middleware\\SubstituteBindings->handle()
#15 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/VerifyCsrfToken.php(78): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#16 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\VerifyCsrfToken->handle()
#17 /var/www/html/vendor/laravel/framework/src/Illuminate/View/Middleware/ShareErrorsFromSession.php(49): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#18 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\View\\Middleware\\ShareErrorsFromSession->handle()
#19 /var/www/html/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php(121): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#20 /var/www/html/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php(64): Illuminate\\Session\\Middleware\\StartSession->handleStatefulRequest()
#21 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Session\\Middleware\\StartSession->handle()
#22 /var/www/html/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/AddQueuedCookiesToResponse.php(37): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#23 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Cookie\\Middleware\\AddQueuedCookiesToResponse->handle()
#24 /var/www/html/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/EncryptCookies.php(67): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#25 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Cookie\\Middleware\\EncryptCookies->handle()
#26 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(119): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#27 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(805): Illuminate\\Pipeline\\Pipeline->then()
#28 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(784): Illuminate\\Routing\\Router->runRouteWithinStack()
#29 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(748): Illuminate\\Routing\\Router->runRoute()
#30 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(737): Illuminate\\Routing\\Router->dispatchToRoute()
#31 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(200): Illuminate\\Routing\\Router->dispatch()
#32 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(144): Illuminate\\Foundation\\Http\\Kernel->Illuminate\\Foundation\\Http\\{closure}()
#33 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#34 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ConvertEmptyStringsToNull.php(31): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle()
#35 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\ConvertEmptyStringsToNull->handle()
#36 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#37 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TrimStrings.php(40): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle()
#38 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\TrimStrings->handle()
#39 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php(27): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#40 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\ValidatePostSize->handle()
#41 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/PreventRequestsDuringMaintenance.php(99): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#42 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\PreventRequestsDuringMaintenance->handle()
#43 /var/www/html/vendor/laravel/framework/src/Illuminate/Http/Middleware/HandleCors.php(49): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#44 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Http\\Middleware\\HandleCors->handle()
#45 /var/www/html/vendor/laravel/framework/src/Illuminate/Http/Middleware/TrustProxies.php(39): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#46 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Http\\Middleware\\TrustProxies->handle()
#47 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(119): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#48 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(175): Illuminate\\Pipeline\\Pipeline->then()
#49 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(144): Illuminate\\Foundation\\Http\\Kernel->sendRequestThroughRouter()
#50 /var/www/html/public/index.php(76): Illuminate\\Foundation\\Http\\Kernel->handle()
#51 {main}
"} 
[2024-02-22 22:02:37] local.DEBUG: isReadyForFileStep("conversion")  
[2024-02-22 22:02:37] local.DEBUG: isReadyForFileStep: Return true  
[2024-02-22 22:02:37] local.DEBUG: Now at App\Http\Controllers\Import\ConversionController::status(conversion-cgFL4DOAYywv)  
[2024-02-22 22:02:37] local.DEBUG: Now in startOrFindConversion(conversion-cgFL4DOAYywv)  
[2024-02-22 22:02:37] local.DEBUG: Conversion status is "conv_running"
@laeckerv
Copy link
Author

Here is a minimal example that fails for me on :develop.

<?xml version="1.0" encoding="UTF-8"?>
<Document
    xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.04"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <BkToCstmrStmt>
        <GrpHdr>
            <MsgId>M/3322163396/CAM</MsgId>
            <CreDtTm>2024-02-22T19:36:31+01:00</CreDtTm>
            <MsgPgntn>
                <PgNb>1</PgNb>
                <LastPgInd>true</LastPgInd>
            </MsgPgntn>
            <AddtlInf>Prod</AddtlInf>
        </GrpHdr>
        <Stmt>
            <Id>M/3322163396/CAM</Id>
            <ElctrncSeqNb>2</ElctrncSeqNb>
            <CreDtTm>2024-02-22T17:27:50+01:00</CreDtTm>
            <FrToDt>
                <FrDtTm>2001-01-01T00:00:00+01:00</FrDtTm>
                <ToDtTm>2024-01-31T00:00:00+01:00</ToDtTm>
            </FrToDt>
            <Acct>
                <Id>
                    <IBAN>CH5604835012345678009</IBAN>
                </Id>
                <Ccy>CHF</Ccy>
                <Ownr>
                    <Nm>Lastname Firstname</Nm>
                </Ownr>
            </Acct>
            <Bal>
                <Tp>
                    <CdOrPrtry>
                        <Cd>OPBD</Cd>
                    </CdOrPrtry>
                </Tp>
                <Amt Ccy="CHF">0.00</Amt>
                <CdtDbtInd>CRDT</CdtDbtInd>
                <Dt>
                    <Dt>2001-01-01</Dt>
                </Dt>
            </Bal>
            <Bal>
                <Tp>
                    <CdOrPrtry>
                        <Cd>CLBD</Cd>
                    </CdOrPrtry>
                </Tp>
                <Amt Ccy="CHF">6858.13</Amt>
                <CdtDbtInd>CRDT</CdtDbtInd>
                <Dt>
                    <Dt>2024-01-31</Dt>
                </Dt>
            </Bal>
            <Bal>
                <Tp>
                    <CdOrPrtry>
                        <Cd>CLAV</Cd>
                    </CdOrPrtry>
                </Tp>
                <Amt Ccy="CHF">6858.13</Amt>
                <CdtDbtInd>CRDT</CdtDbtInd>
                <Dt>
                    <Dt>2024-01-31</Dt>
                </Dt>
            </Bal>
            <TxsSummry>
                <TtlNtries>
                    <NbOfNtries>57</NbOfNtries>
                    <Sum>376395.25</Sum>
                    <TtlNetNtry>
                        <Amt>6858.13</Amt>
                        <CdtDbtInd>CRDT</CdtDbtInd>
                    </TtlNetNtry>
                </TtlNtries>
                <TtlCdtNtries>
                    <NbOfNtries>38</NbOfNtries>
                    <Sum>191626.69</Sum>
                </TtlCdtNtries>
                <TtlDbtNtries>
                    <NbOfNtries>19</NbOfNtries>
                    <Sum>184768.56</Sum>
                </TtlDbtNtries>
            </TxsSummry>
            <Ntry>
                <NtryRef>1883128456</NtryRef>
                <Amt Ccy="CHF">0.00</Amt>
                <CdtDbtInd>CRDT</CdtDbtInd>
                <Sts>BOOK</Sts>
                <BookgDt>
                    <Dt>2021-12-01</Dt>
                </BookgDt>
                <ValDt>
                    <Dt>2021-12-01</Dt>
                </ValDt>
                <AcctSvcrRef>O/1883128456</AcctSvcrRef>
                <BkTxCd>
                    <Domn>
                        <Cd>PMNT</Cd>
                        <Fmly>
                            <Cd>NTAV</Cd>
                            <SubFmlyCd>NTAV</SubFmlyCd>
                        </Fmly>
                    </Domn>
                </BkTxCd>
                <NtryDtls>
                    <TxDtls>
                        <Amt Ccy="CHF">0.00</Amt>
                        <CdtDbtInd>CRDT</CdtDbtInd>
                    </TxDtls>
                </NtryDtls>
                <AddtlNtryInf>Technische Buchung Auftragsnummer: 1883128456, 01.12.21
</AddtlNtryInf>
            </Ntry>
        </Stmt>
    </BkToCstmrStmt>
</Document>

@JC5
Copy link
Member

JC5 commented Feb 23, 2024

Nice find, thanks for letting me know. I can fix this, but it means this transaction will be skipped, since it has no amount information. Transactions with no amount will not be stored by Firefly III.

@JC5 JC5 added the bug Verified and replicated bugs and issues. label Feb 23, 2024
@JC5 JC5 added this to the importer-v1.4.5 milestone Feb 23, 2024
@JC5 JC5 added the fixed Bugs that are fixed (in a coming release). label Feb 23, 2024
@laeckerv
Copy link
Author

laeckerv commented Feb 23, 2024 •
edited

Wow. Thanks for that fast response. I'll test it once the devlop image is updated and reprt back.

@JC5
Copy link
Member

JC5 commented Feb 23, 2024

Not a problem! Thanks for the reminder new develop tag should be up and running.

@laeckerv
Copy link
Author

Works like a charm. Thank you!

@github-actions GitHub Actions
Copy link
Contributor

Hi there! This is an automatic reply. Share and enjoy

This issue is now 🔒 closed. Please be aware that closed issues are not monitored by the developer of Firefly III.

  • If the original bug is not actually fixed, please open a new issue. Refer to this issue for clarity.
  • Follow-up questions must be posted in a new discussion
  • Further replies to this issue may get no response.

If there is more to discuss, please open a new issue or discussion.

Thank you for your contributions.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 11, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
bug Verified and replicated bugs and issues. fixed Bugs that are fixed (in a coming release).
Projects
Data Importer
Status: Done
Milestone
importer-v1.4.5
Development

No branches or pull requests
2 participants
@laeckerv @JC5