[CSV-Importer] Crash when rerunning an import

[Alaksin]

Support guidelines

• I've read the support guidelines

I've found a bug and checked that ...

• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem

Description

I made a successful import, then made some rules for the new transactions, removed all the transactions and the import tag. When trying to run the import again I get this error for every transaction

Line #0: Submission HTTP error: Status code is 500: {"message":"Internal Firefly III Exception: Attempt to read property \"transaction_group_id\" on null","exception":"ErrorException"}

Expected behaviour

I have done this several times before and it has always worked, until today.

Debug information

Debug information generated at 2021-10-01 09:46:46 Asia/Kuala_Lumpur for Firefly III version 5.6.1.

Scope	Version
Firefly III	5.6.1
Firefly III API	1.5.4
PHP	8.0.10
Host	Linux

System info	Value
Installation ID	6ec83b7f-8f96-4c74-b42c-97b6c8e09ab6
Using docker?	true
Telemetry	false
Layout
System TZ	Asia/Kuala_Lumpur
Browser TZ	Asia/Kuala_Lumpur
App environment	local
App debug mode	false
App cache driver	file
App logging	info, stack
Display errors	Off
Error reporting	ALL errors
Interface	apache2handler
Default language	en_US
Default locale	equal
BCscale	24
DB drivers	mysql, pgsql, sqlite
Current driver	mysql
DB version	18 (exp. 18)
Login provider	eloquent
Trusted proxies (.env)

User info	Value
Session start	2021-10-01 00:00:00
Session end	2021-10-31 23:59:59
Session first	2017-08-25 00:00:00
User ID	1
User language	en_US
User locale	en_US
Attempt at "en_US.utf8"	'en_US.utf8'
Attempt at "en_US.UTF-8"	'en_US.UTF-8'
User agent	Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:92.0) Gecko/20100101 Firefox/92.0

Steps to reproduce

1. Run an import
2. Remove imported transactions and tags
3. Run the import again

Additional info

[2021-10-01 03:43:25] production.DEBUG: Now in storeJobStatus(8nMSe9QCIoCjdpIY): waiting_to_start
[2021-10-01 03:43:25] production.INFO: No info on group.
[2021-10-01 03:43:25] production.DEBUG: Duplicate detection method is "classic", so this method is skipped (return true).
[2021-10-01 03:43:25] production.DEBUG: Submitting to Firefly III: {"group_title":null,"error_if_duplicate_hash":true,"transactions":[{"type":"withdrawal","date":"2021-09-26 00:00:00","currency_id":51,"currency_code":null,"amount":"7500.000000000000","description":"TRANSFER DEBIT payment TRANSACTION DESCRIPTION","source_id":1,"source_name":null,"destination_id":null,"destination_name":"(no name)","original_source":"jc5-csv-import-v2.6.1","tags":[],"source_iban":null,"source_number":null,"source_bic":null,"destination_iban":null,"destination_number":null,"destination_bic":null}]}
[2021-10-01 03:43:25] production.ERROR: Status code is 500: {"message":"Internal Firefly III Exception: Attempt to read property "transaction_group_id" on null","exception":"ErrorException"}
[2021-10-01 03:43:25] production.ERROR: #0 /var/www/html/vendor/firefly-iii/api-support-classes/src/Request/Request.php(235): GrumpyDictator\FFIIIApiSupport\Request\Request->authenticatedSubmission()
#1 /var/www/html/vendor/firefly-iii/api-support-classes/src/Request/PostTransactionRequest.php(68): GrumpyDictator\FFIIIApiSupport\Request\Request->authenticatedPost()
#2 /var/www/html/app/Services/Import/Routine/APISubmitter.php(231): GrumpyDictator\FFIIIApiSupport\Request\PostTransactionRequest->post()
#3 /var/www/html/app/Services/Import/Routine/APISubmitter.php(80): App\Services\Import\Routine\APISubmitter->processTransaction()
#4 /var/www/html/app/Services/Import/ImportRoutineManager.php(190): App\Services\Import\Routine\APISubmitter->processTransactions()
#5 /var/www/html/app/Console/StartImport.php(67): App\Services\Import\ImportRoutineManager->start()
#6 /var/www/html/app/Console/AutoImports.php(141): App\Console\Commands\AutoImport->startImport()
#7 /var/www/html/app/Console/AutoImports.php(110): App\Console\Commands\AutoImport->importFile()
#8 /var/www/html/app/Console/Commands/AutoImport.php(84): App\Console\Commands\AutoImport->importFiles()
#9 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): App\Console\Commands\AutoImport->handle()
#10 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Util.php(40): Illuminate\Container\BoundMethod::Illuminate\Container{closure}()
#11 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\Container\Util::unwrapIfClosure()
#12 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(37): Illuminate\Container\BoundMethod::callBoundMethod()
#13 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(651): Illuminate\Container\BoundMethod::call()
#14 /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Command.php(136): Illuminate\Container\Container->call()
#15 /var/www/html/vendor/symfony/console/Command/Command.php(299): Illuminate\Console\Command->execute()
#16 /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Command.php(121): Symfony\Component\Console\Command\Command->run()
#17 /var/www/html/vendor/symfony/console/Application.php(978): Illuminate\Console\Command->run()
#18 /var/www/html/vendor/symfony/console/Application.php(295): Symfony\Component\Console\Application->doRunCommand()
#19 /var/www/html/vendor/symfony/console/Application.php(167): Symfony\Component\Console\Application->doRun()
#20 /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Application.php(92): Symfony\Component\Console\Application->run()
#21 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(129): Illuminate\Console\Application->run()
#22 /var/www/html/artisan(37): Illuminate\Foundation\Console\Kernel->handle()
#23 {main}
[2021-10-01 03:43:25] production.DEBUG: Add error on index #4 (line no. 5): Submission HTTP error: Status code is 500: {"message":"Internal Firefly III Exception: Attempt to read property "transaction_group_id" on null","exception":"ErrorException"}
[2021-10-01 03:43:25] production.DEBUG: Now in storeJobStatus(8nMSe9QCIoCjdpIY): waiting_to_start
[2021-10-01 03:43:25] production.INFO: No info on group.
[2021-10-01 03:43:25] production.INFO: Done submitting 5 transactions to your Firefly III instance.
ERROR in line #1: Submission HTTP error: Status code is 500: {"message":"Internal Firefly III Exception: Attempt to read property "transaction_group_id" on null","exception":"ErrorException"}
ERROR in line #2: Submission HTTP error: Status code is 500: {"message":"Internal Firefly III Exception: Attempt to read property "transaction_group_id" on null","exception":"ErrorException"}
ERROR in line #3: Submission HTTP error: Status code is 500: {"message":"Internal Firefly III Exception: Attempt to read property "transaction_group_id" on null","exception":"ErrorException"}
ERROR in line #4: Submission HTTP error: Status code is 500: {"message":"Internal Firefly III Exception: Attempt to read property "transaction_group_id" on null","exception":"ErrorException"}
ERROR in line #5: Submission HTTP error: Status code is 500: {"message":"Internal Firefly III Exception: Attempt to read property "transaction_group_id" on null","exception":"ErrorException"}
Import complete.
Done importing from file /import/hsbc-debit.csv using configuration /import/hsbc-debit.json.
[2021-10-01 03:43:25] production.DEBUG: Log log {"messages":[[],[],[],[],[]],"warnings":[[],[],[],[],[]],"errors":[["Submission HTTP error: Status code is 500: {"message":"Internal Firefly III Exception: Attempt to read property \"transaction_group_id\" on null","exception":"ErrorException"}"],["Submission HTTP error: Status code is 500: {"message":"Internal Firefly III Exception: Attempt to read property \"transaction_group_id\" on null","exception":"ErrorException"}"],["Submission HTTP error: Status code is 500: {"message":"Internal Firefly III Exception: Attempt to read property \"transaction_group_id\" on null","exception":"ErrorException"}"],["Submission HTTP error: Status code is 500: {"message":"Internal Firefly III Exception: Attempt to read property \"transaction_group_id\" on null","exception":"ErrorException"}"],["Submission HTTP error: Status code is 500: {"message":"Internal Firefly III Exception: Attempt to read property \"transaction_group_id\" on null","exception":"ErrorException"}"]]}

[JC5]

Could you find the logs for Firefly III itself please? That will tell me where the error occurred.

[Alaksin]

Looks like it's related to duplicate transaction. Makes sense that there are duplicates since I run the same import, but the duplicated transactions have all been marked as deleted, so they should be ignored

[2021-10-01 09:00:29] local.INFO: Will trigger duplication alert for this journal.  

[2021-10-01 09:00:29] local.WARNING: Found a duplicate in errorIfDuplicate because hash e5369415db1e4c47094e70092c4c847c5fd1e597ee4770a58b86dec6a726404b is not unique!  

[2021-10-01 09:00:29] local.ERROR: Exception is: {"class":"ErrorException","errorMessage":"Attempt to read property \"transaction_group_id\" on null","time":"Fri, 01 Oct 2021 09:00:29 +0800","file":"\/var\/www\/html\/app\/Factory\/TransactionJournalFactory.php","line":345,"code":0,"version":"5.6.1","url":"http:\/\/192.168.11.11:8088\/api\/v1\/transactions","userAgent":"GuzzleHttp\/7","json":true}  

[2021-10-01 09:00:29] local.ERROR: Exception when mailing: Address in mailbox given [] does not comply with RFC 2822, 3.6.2.  

[2021-10-01 09:00:29] local.ERROR: Attempt to read property "transaction_group_id" on null {"userId":1,"exception":"[object] (ErrorException(code: 0): Attempt to read property \"transaction_group_id\" on null at /var/www/html/app/Factory/TransactionJournalFactory.php:345)
�      
[stacktrace]

#0 /var/www/html/app/Factory/TransactionJournalFactory.php(345): Illuminate\\Foundation\\Bootstrap\\HandleExceptions->handleError()

#1 /var/www/html/app/Factory/TransactionJournalFactory.php(157): FireflyIII\\Factory\\TransactionJournalFactory->errorIfDuplicate()

#2 /var/www/html/app/Factory/TransactionJournalFactory.php(120): FireflyIII\\Factory\\TransactionJournalFactory->createJournal()

#3 /var/www/html/app/Factory/TransactionGroupFactory.php(67): FireflyIII\\Factory\\TransactionJournalFactory->create()

#4 /var/www/html/app/Repositories/TransactionGroup/TransactionGroupRepository.php(360): FireflyIII\\Factory\\TransactionGroupFactory->create()

#5 /var/www/html/app/Api/V1/Controllers/Models/Transaction/StoreController.php(94): FireflyIII\\Repositories\\TransactionGroup\\TransactionGroupRepository->store()

#6 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Controller.php(54): FireflyIII\\Api\\V1\\Controllers\\Models\\Transaction\\StoreController->store()

#7 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php(45): Illuminate\\Routing\\Controller->callAction()

#8 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Route.php(261): Illuminate\\Routing\\ControllerDispatcher->dispatch()

#9 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Route.php(204): Illuminate\\Routing\\Route->runController()

#10 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(695): Illuminate\\Routing\\Route->run()

#11 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(128): Illuminate\\Routing\\Router->Illuminate\\Routing\\{closure}()

#12 /var/www/html/app/Api/V1/Controllers/Models/Transaction/StoreController.php(68): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()

#13 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(149): FireflyIII\\Api\\V1\\Controllers\\Models\\Transaction\\StoreController->FireflyIII\\Api\\V1\\Controllers\\Models\\Transaction\\{closure}()

#14 /var/www/html/app/Api/V1/Controllers/Controller.php(67): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()

#15 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(149): FireflyIII\\Api\\V1\\Controllers\\Controller->FireflyIII\\Api\\V1\\Controllers\\{closure}()

#16 /var/www/html/app/Http/Middleware/Binder.php(78): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()

#17 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): FireflyIII\\Http\\Middleware\\Binder->handle()

#18 /var/www/html/app/Http/Middleware/Authenticate.php(75): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()

#19 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): FireflyIII\\Http\\Middleware\\Authenticate->handle()

#20 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(103): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()

#21 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(697): Illuminate\\Pipeline\\Pipeline->then()

#22 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(672): Illuminate\\Routing\\Router->runRouteWithinStack()

#23 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(636): Illuminate\\Routing\\Router->runRoute()

#24 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(625): Illuminate\\Routing\\Router->dispatchToRoute()

#25 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(166): Illuminate\\Routing\\Router->dispatch()

#26 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(128): Illuminate\\Foundation\\Http\\Kernel->Illuminate\\Foundation\\Http\\{closure}()

#27 /var/www/html/app/Http/Middleware/InstallationId.php(52): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()

#28 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): FireflyIII\\Http\\Middleware\\InstallationId->handle()

#29 /var/www/html/vendor/fideloper/proxy/src/TrustProxies.php(57): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()

#30 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Fideloper\\Proxy\\TrustProxies->handle()

#31 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()

#32 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ConvertEmptyStringsToNull.php(31): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle()

#33 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Foundation\\Http\\Middleware\\ConvertEmptyStringsToNull->handle()

#34 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()

#35 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TrimStrings.php(40): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle()

#36 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Foundation\\Http\\Middleware\\TrimStrings->handle()

#37 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php(27): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()

#38 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Foundation\\Http\\Middleware\\ValidatePostSize->handle()

#39 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/PreventRequestsDuringMaintenance.php(86): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()

#40 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Foundation\\Http\\Middleware\\PreventRequestsDuringMaintenance->handle()

#41 /var/www/html/app/Http/Middleware/SecureHeaders.php(51): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()

#42 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): FireflyIII\\Http\\Middleware\\SecureHeaders->handle()

#43 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(103): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()

#44 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(141): Illuminate\\Pipeline\\Pipeline->then()

#45 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(110): Illuminate\\Foundation\\Http\\Kernel->sendRequestThroughRouter()

#46 /var/www/html/public/index.php(76): Illuminate\\Foundation\\Http\\Kernel->handle()

#47 {main}

"} 

[JC5]

Thanks. Keep in mind deleted transactions will NOT be ignored.