Bunq support in v4.7.4

[JC5]

@OGKevin @HippoGrief @gldrenthe89 @paul999 @jonathanjuursema @Xeli @IcyPalm @Want100Cookies @FreekBloemhof @japsikado @SpaceK33z

Hi guys!

I've tagged all of you because you've worked with bunq + Firefly III, or have complained about its implementation.

Currently live in the develop-branch is new working code for the bunq-import routine that works on my machine, as developers tend to say.

I would like to ask you to install a development version of Firefly III and test if it works as expected.

Since I am not sure what your technical expertise is, here are some general instructions.

1. Clone the git repository and check out the develop branch.
2. Copy the .env.example file to .env and fill in all the details.
3. Check out the BUNQ_USE_SANDBOX=false at the bottom. You can switch to the sandbox if you want to.
4. Run the following commands to install all tables and stuff:

composer install --no-dev --no-scripts
composer install --no-dev
php artisan migrate:refresh --seed
php artisan passport:install

5. Browse to Firefly III. It should self-identify as 4.7.4
6. Try the import!

If you are a Docker fan, the latest develop build should contain the new Bunq code as well but you have to have a database already.

I am currently writing tests, so be sure to git pull regularly.

Thanks in advance for your help!

Cheers,
James

[OGKevin]

Will give it a look later today 👍

[OGKevin]

references #1387

[paul999]

Will try it tonight, thanks for your work!

[japsikado]

Got error when i click the submit button in bunq import section

(1/1) BunqExceptionGenerating a new private key failed. See "http://php.net/manual/en/function.openssl-pkey-new.php" for more info.
--

freshly installed 4.7.4 from develop branch

[OGKevin]

@japsikado can you paste the full traceback and on what system are you.

[JC5]

Thanks for letting me know! What OS are you using?

[japsikado]

I am using windows 10
have installed it on xampp with vhost to public folder
version 4.7.4

`Debug information generated at 2018-05-23 13:59:41 Europe/Amsterdam for Firefly III version 4.7.4.

Variable	Content
FF version	4.7.4
FF API version	0.3
App environment	local
App debug mode	true
App cache driver	file
App logging	notice, daily
PHP version	7.2.1
Display errors	Off
Session start	2018-05-01 00:00:00
Session end	2018-05-31 23:59:59
Session first	2018-01-01 00:00:00
Error reporting	ALL errors
Host	WINNT
Interface	apache2handler
UserID	1
Attempt at "en"	'en'
Attempt at "English"	'English_United States.1252'
Attempt at "en_US"	false
Attempt at "en_US.utf8"	false
Attempt at "en_US.UTF-8"	false
DB drivers	mysql, sqlite
Current driver	mysql
Using Sandstorm?	no
Is Sandstorm (.env)	false
Is Docker (.env)	false
Trusted proxies (.env)
User agent	Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36
Loaded extensions	Core, bcmath, calendar, ctype, date, filter, hash, iconv, json, SPL, pcre, readline, Reflection, session, standard, mysqlnd, tokenizer, zip, zlib, libxml, dom, PDO, bz2, SimpleXML, xml, wddx, xmlreader, xmlwriter, apache2handler, openssl, curl, fileinfo, gd, gettext, intl, mbstring, exif, mysqli, pdo_mysql, pdo_sqlite, Phar, ftp
Installed packages	bacon/bacon-qr-code@1.0.3, bunq/sdk_php@0.13.1, davejamesmiller/laravel-breadcrumbs@5.1.0, defuse/php-encryption@v2.2.0, doctrine/annotations@v1.6.0, doctrine/cache@v1.7.1, doctrine/collections@v1.5.0, doctrine/common@v2.8.1, doctrine/dbal@v2.7.1, doctrine/inflector@v1.3.0, doctrine/lexer@v1.0.1, dragonmantank/cron-expression@v2.1.0, egulias/email-validator@2.1.4, erusev/parsedown@1.7.1, fideloper/proxy@4.0.0, firebase/php-jwt@v5.0.0, guzzlehttp/guzzle@6.3.3, guzzlehttp/promises@v1.3.1, guzzlehttp/psr7@1.4.2, laravel/framework@v5.6.23, laravel/passport@v5.0.3, laravelcollective/html@v5.6.8, lcobucci/jwt@3.2.2, league/commonmark@0.17.5, league/csv@9.1.4, league/event@2.1.2, league/flysystem@1.0.45, league/fractal@0.17.0, league/oauth2-server@6.1.1, monolog/monolog@1.23.0, nesbot/carbon@1.25.0, paragonie/constant_time_encoding@v2.2.2, paragonie/random_compat@v2.0.12, phpseclib/phpseclib@2.0.11, pragmarx/google2fa@v3.0.1, pragmarx/google2fa-laravel@v0.2.0, psr/container@1.0.0, psr/http-message@1.0.1, psr/log@1.0.2, psr/simple-cache@1.0.1, ramsey/uuid@3.7.3, rcrowe/twigbridge@v0.9.6, rmccue/requests@v1.7.0, swiftmailer/swiftmailer@v6.0.2, symfony/console@v4.0.10, symfony/css-selector@v4.0.10, symfony/debug@v4.0.10, symfony/event-dispatcher@v4.0.10, symfony/finder@v4.0.10, symfony/http-foundation@v4.0.10, symfony/http-kernel@v4.0.10, symfony/polyfill-ctype@v1.8.0, symfony/polyfill-mbstring@v1.8.0, symfony/polyfill-php56@v1.8.0, symfony/polyfill-php72@v1.8.0, symfony/polyfill-util@v1.8.0, symfony/process@v4.0.10, symfony/psr-http-message-bridge@v1.0.2, symfony/routing@v4.0.10, symfony/translation@v4.0.10, symfony/var-dumper@v4.0.10, tijsverkoyen/css-to-inline-styles@2.2.1, twig/twig@v1.35.3, vlucas/phpdotenv@v2.4.0, zendframework/zend-diactoros@1.7.1,
`

Truncated from this point <----|dation\\Http\\{closure}(Object(Illuminate\\Http\\Request))
#58 C:\\xampp\\htdocs\\firefly-iii\\vendor\\fideloper\\proxy\\src\\TrustProxies.php(57): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#59 C:\\xampp\\htdocs\\firefly-iii\\vendor\\laravel\\framework\\src\\Illuminate\\Pipeline\\Pipeline.php(151): Fideloper\\Proxy\\TrustProxies->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#60 C:\\xampp\\htdocs\\firefly-iii\\vendor\\laravel\\framework\\src\\Illuminate\\Routing\\Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#61 C:\\xampp\\htdocs\\firefly-iii\\vendor\\laravel\\framework\\src\\Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest.php(31): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#62 C:\\xampp\\htdocs\\firefly-iii\\vendor\\laravel\\framework\\src\\Illuminate\\Pipeline\\Pipeline.php(151): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#63 C:\\xampp\\htdocs\\firefly-iii\\vendor\\laravel\\framework\\src\\Illuminate\\Routing\\Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#64 C:\\xampp\\htdocs\\firefly-iii\\vendor\\laravel\\framework\\src\\Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest.php(31): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#65 C:\\xampp\\htdocs\\firefly-iii\\vendor\\laravel\\framework\\src\\Illuminate\\Pipeline\\Pipeline.php(151): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#66 C:\\xampp\\htdocs\\firefly-iii\\vendor\\laravel\\framework\\src\\Illuminate\\Routing\\Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#67 C:\\xampp\\htdocs\\firefly-iii\\vendor\\laravel\\framework\\src\\Illuminate\\Foundation\\Http\\Middleware\\ValidatePostSize.php(27): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#68 C:\\xampp\\htdocs\\firefly-iii\\vendor\\laravel\\framework\\src\\Illuminate\\Pipeline\\Pipeline.php(151): Illuminate\\Foundation\\Http\\Middleware\\ValidatePostSize->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#69 C:\\xampp\\htdocs\\firefly-iii\\vendor\\laravel\\framework\\src\\Illuminate\\Routing\\Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#70 C:\\xampp\\htdocs\\firefly-iii\\vendor\\laravel\\framework\\src\\Illuminate\\Foundation\\Http\\Middleware\\CheckForMaintenanceMode.php(51): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#71 C:\\xampp\\htdocs\\firefly-iii\\vendor\\laravel\\framework\\src\\Illuminate\\Pipeline\\Pipeline.php(151): Illuminate\\Foundation\\Http\\Middleware\\CheckForMaintenanceMode->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#72 C:\\xampp\\htdocs\\firefly-iii\\vendor\\laravel\\framework\\src\\Illuminate\\Routing\\Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#73 C:\\xampp\\htdocs\\firefly-iii\\vendor\\laravel\\framework\\src\\Illuminate\\Pipeline\\Pipeline.php(104): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#74 C:\\xampp\\htdocs\\firefly-iii\\vendor\\laravel\\framework\\src\\Illuminate\\Foundation\\Http\\Kernel.php(151): Illuminate\\Pipeline\\Pipeline->then(Object(Closure))
#75 C:\\xampp\\htdocs\\firefly-iii\\vendor\\laravel\\framework\\src\\Illuminate\\Foundation\\Http\\Kernel.php(116): Illuminate\\Foundation\\Http\\Kernel->sendRequestThroughRouter(Object(Illuminate\\Http\\Request))
#76 C:\\xampp\\htdocs\\firefly-iii\\public\\index.php(75): Illuminate\\Foundation\\Http\\Kernel->handle(Object(Illuminate\\Http\\Request))
#77 {main}
"} 

```
`

full traceback 
`Whoops, looks like something went wrong.
(1/1) BunqException
Generating a new private key failed. See "http://php.net/manual/en/function.openssl-pkey-new.php" for more info.

in KeyPair.php line 66
at KeyPair::generate()
in ApiContext.php line 139
at ApiContext->initializeInstallationContext()
in ApiContext.php line 130
at ApiContext->initialize('Firefly III v4.7.4', array('MY IP'))
in ApiContext.php line 111
at ApiContext::create(object(BunqEnumApiEnvironmentType), 'MY API KEY', 'Firefly III v4.7.4', array('MY IP'))
in BunqPrerequisites.php line 123
at BunqPrerequisites->storePrerequisites(array('_token' => 'flCf51GmdkNnhKV3NgjJxxrG2efdo3VyJt8TxsTD', 'api_key' => 'MY API KEY', 'external_ip' => 'MY IP'))
in PrerequisitesController.php line 148
at PrerequisitesController->post(object(Request), 'bunq', null)
at call_user_func_array(array(object(PrerequisitesController), 'post'), array(object(Request), 'import_provider' => 'bunq', null))
in Controller.php line 54
at Controller->callAction('post', array(object(Request), 'import_provider' => 'bunq', null))
in ControllerDispatcher.php line 45
at ControllerDispatcher->dispatch(object(Route), object(PrerequisitesController), 'post')
in Route.php line 212
at Route->runController()
in Route.php line 169
at Route->run()
in Router.php line 659
at Router->Illuminate\Routing\{closure}(object(Request))
in Pipeline.php line 30
at Pipeline->Illuminate\Routing\{closure}(object(Request))
in PrerequisitesController.php line 57
at PrerequisitesController->FireflyIII\Http\Controllers\Import\{closure}(object(Request), object(Closure))
in Pipeline.php line 133
at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
in Pipeline.php line 53
at Pipeline->Illuminate\Routing\{closure}(object(Request))
in Controller.php line 109
at Controller->FireflyIII\Http\Controllers\{closure}(object(Request), object(Closure))
in Pipeline.php line 133
at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
in Pipeline.php line 53
at Pipeline->Illuminate\Routing\{closure}(object(Request))
in Binder.php line 77
at Binder->handle(object(Request), object(Closure))
in Pipeline.php line 151
at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
in Pipeline.php line 53
at Pipeline->Illuminate\Routing\{closure}(object(Request))
in Range.php line 63
at Range->handle(object(Request), object(Closure))
in Pipeline.php line 151
at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
in Pipeline.php line 53
at Pipeline->Illuminate\Routing\{closure}(object(Request))
in AuthenticateTwoFactor.php line 78
at AuthenticateTwoFactor->handle(object(Request), object(Closure))
in Pipeline.php line 151
at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
in Pipeline.php line 53
at Pipeline->Illuminate\Routing\{closure}(object(Request))
in Authenticate.php line 73
at Authenticate->handle(object(Request), object(Closure))
in Pipeline.php line 151
at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
in Pipeline.php line 53
at Pipeline->Illuminate\Routing\{closure}(object(Request))
in CreateFreshApiToken.php line 50
at CreateFreshApiToken->handle(object(Request), object(Closure))
in Pipeline.php line 151
at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
in Pipeline.php line 53
at Pipeline->Illuminate\Routing\{closure}(object(Request))
in VerifyCsrfToken.php line 67
at VerifyCsrfToken->handle(object(Request), object(Closure))
in Pipeline.php line 151
at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
in Pipeline.php line 53
at Pipeline->Illuminate\Routing\{closure}(object(Request))
in ShareErrorsFromSession.php line 49
at ShareErrorsFromSession->handle(object(Request), object(Closure))
in Pipeline.php line 151
at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
in Pipeline.php line 53
at Pipeline->Illuminate\Routing\{closure}(object(Request))
in StartSession.php line 63
at StartSession->handle(object(Request), object(Closure))
in Pipeline.php line 151
at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
in Pipeline.php line 53
at Pipeline->Illuminate\Routing\{closure}(object(Request))
in AddQueuedCookiesToResponse.php line 37
at AddQueuedCookiesToResponse->handle(object(Request), object(Closure))
in Pipeline.php line 151
at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
in Pipeline.php line 53
at Pipeline->Illuminate\Routing\{closure}(object(Request))
in EncryptCookies.php line 59
at EncryptCookies->handle(object(Request), object(Closure))
in Pipeline.php line 151
at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
in Pipeline.php line 53
at Pipeline->Illuminate\Routing\{closure}(object(Request))
in Sandstorm.php line 58
at Sandstorm->handle(object(Request), object(Closure))
in Pipeline.php line 151
at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
in Pipeline.php line 53
at Pipeline->Illuminate\Routing\{closure}(object(Request))
in Pipeline.php line 104
at Pipeline->then(object(Closure))
in Router.php line 661
at Router->runRouteWithinStack(object(Route), object(Request))
in Router.php line 636
at Router->runRoute(object(Request), object(Route))
in Router.php line 602
at Router->dispatchToRoute(object(Request))
in Router.php line 591
at Router->dispatch(object(Request))
in Kernel.php line 176
at Kernel->Illuminate\Foundation\Http\{closure}(object(Request))
in Pipeline.php line 30
at Pipeline->Illuminate\Routing\{closure}(object(Request))
in TrustProxies.php line 57
at TrustProxies->handle(object(Request), object(Closure))
in Pipeline.php line 151
at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
in Pipeline.php line 53
at Pipeline->Illuminate\Routing\{closure}(object(Request))
in TransformsRequest.php line 31
at TransformsRequest->handle(object(Request), object(Closure))
in Pipeline.php line 151
at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
in Pipeline.php line 53
at Pipeline->Illuminate\Routing\{closure}(object(Request))
in TransformsRequest.php line 31
at TransformsRequest->handle(object(Request), object(Closure))
in Pipeline.php line 151
at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
in Pipeline.php line 53
at Pipeline->Illuminate\Routing\{closure}(object(Request))
in ValidatePostSize.php line 27
at ValidatePostSize->handle(object(Request), object(Closure))
in Pipeline.php line 151
at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
in Pipeline.php line 53
at Pipeline->Illuminate\Routing\{closure}(object(Request))
in CheckForMaintenanceMode.php line 51
at CheckForMaintenanceMode->handle(object(Request), object(Closure))
in Pipeline.php line 151
at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
in Pipeline.php line 53
at Pipeline->Illuminate\Routing\{closure}(object(Request))
in Pipeline.php line 104
at Pipeline->then(object(Closure))
in Kernel.php line 151
at Kernel->sendRequestThroughRouter(object(Request))
in Kernel.php line 116
at Kernel->handle(object(Request))
in index.php line 75`

[JC5]

It means you need a working OpenSSL installation. Please check out this page for some extra info.

[OGKevin]

You can also have a look at this issue: bunq/sdk_php#102

[JC5]

However, if you do nothing, and try again, Firefly III will give you a nice error message.

I've just pushed a catch routine that should give you more background and a way to fix this.

[jonathanjuursema]

Hey! Thanks for the update! I just gave the 4.7.4 update a go and I seem to get stuck at running the import.

The import job is still in the database as running, with no actually imported data.

Bunq recognizes that the API key has been used, so that seems to be working.

I get this in the application log. Anything I can/should try?

~ Edit: Debian 9.4, PHP 7.1 (FPM), nginx 1.10.3

[2018-05-23 21:35:18] local.ERROR: Exception when mailing: Connection could not be established with host smtp.transip.email [No route to host #113]
[2018-05-23 21:35:18] local.ERROR: Type error: Argument 1 passed to bunq\Model\Core\BunqModel::createFromResponseArray() must be of the type array, string given, called in /var/www/firefly/firefly-23052018/vendor/bunq/sdk_php/src/Model/Core/BunqModel.php on line 236 {"userId":1,"email":"firefly@jonathanj.nl","exception":"[object] (Symfony\\Component\\Debug\\Exception\\FatalThrowableError(code: 0): Type error: Argument 1 passed to bunq\\Model\\Core\\BunqModel::createFromResponseArray() must be of the type array, string given, called in /var/www/firefly/firefly-23052018/vendor/bunq/sdk_php/src/Model/Core/BunqModel.php on line 236 at /var/www/firefly/firefly-23052018/vendor/bunq/sdk_php/src/Model/Core/BunqModel.php:97)
[stacktrace]
#0 /var/www/firefly/firefly-23052018/vendor/bunq/sdk_php/src/Model/Core/BunqModel.php(236): bunq\\Model\\Core\\BunqModel::createFromResponseArray('8507571d-bb07-4...', NULL)
#1 /var/www/firefly/firefly-23052018/vendor/bunq/sdk_php/src/Model/Core/BunqModel.php(201): bunq\\Model\\Core\\BunqModel::createListFromResponseArray(Array)
#2 /var/www/firefly/firefly-23052018/vendor/bunq/sdk_php/src/Model/Core/BunqModel.php(159): bunq\\Model\\Core\\BunqModel::determineFieldContents(Object(ReflectionProperty), Array)
#3 /var/www/firefly/firefly-23052018/vendor/bunq/sdk_php/src/Model/Core/BunqModel.php(113): bunq\\Model\\Core\\BunqModel::createInstanceFromResponseArray(Array)
#4 /var/www/firefly/firefly-23052018/vendor/bunq/sdk_php/src/Model/Core/BunqModel.php(236): bunq\\Model\\Core\\BunqModel::createFromResponseArray(Array, NULL)
#5 /var/www/firefly/firefly-23052018/vendor/bunq/sdk_php/src/Model/Core/BunqModel.php(201): bunq\\Model\\Core\\BunqModel::createListFromResponseArray(Array)
#6 /var/www/firefly/firefly-23052018/vendor/bunq/sdk_php/src/Model/Core/BunqModel.php(159): bunq\\Model\\Core\\BunqModel::determineFieldContents(Object(ReflectionProperty), Array)
#7 /var/www/firefly/firefly-23052018/vendor/bunq/sdk_php/src/Model/Core/BunqModel.php(113): bunq\\Model\\Core\\BunqModel::createInstanceFromResponseArray(Array)
#8 /var/www/firefly/firefly-23052018/vendor/bunq/sdk_php/src/Model/Core/BunqModel.php(206): bunq\\Model\\Core\\BunqModel::createFromResponseArray(Array)
#9 /var/www/firefly/firefly-23052018/vendor/bunq/sdk_php/src/Model/Core/BunqModel.php(159): bunq\\Model\\Core\\BunqModel::determineFieldContents(Object(ReflectionProperty), Array)
#10 /var/www/firefly/firefly-23052018/vendor/bunq/sdk_php/src/Model/Core/BunqModel.php(134): bunq\\Model\\Core\\BunqModel::createInstanceFromResponseArray(Array)
#11 /var/www/firefly/firefly-23052018/vendor/bunq/sdk_php/src/Model/Core/BunqModel.php(102): bunq\\Model\\Core\\BunqModel::createFromResponseArrayAnchorObject(Array)
#12 /var/www/firefly/firefly-23052018/vendor/bunq/sdk_php/src/Model/Core/BunqModel.php(236): bunq\\Model\\Core\\BunqModel::createFromResponseArray(Array, NULL)
#13 /var/www/firefly/firefly-23052018/vendor/bunq/sdk_php/src/Model/Core/BunqModel.php(296): bunq\\Model\\Core\\BunqModel::createListFromResponseArray(Array, NULL)
#14 /var/www/firefly/firefly-23052018/vendor/bunq/sdk_php/src/Model/Generated/Endpoint/MonetaryAccount.php(100): bunq\\Model\\Core\\BunqModel::fromJsonList(Object(bunq\\Http\\BunqResponseRaw))
#15 /var/www/firefly/firefly-23052018/app/Support/Import/Routine/Bunq/StageNewHandler.php(104): bunq\\Model\\Generated\\Endpoint\\MonetaryAccount::listing()
#16 /var/www/firefly/firefly-23052018/app/Support/Import/Routine/Bunq/StageNewHandler.php(74): FireflyIII\\Support\\Import\\Routine\\Bunq\\StageNewHandler->listAccounts()
#17 /var/www/firefly/firefly-23052018/app/Import/Routine/BunqRoutine.php(66): FireflyIII\\Support\\Import\\Routine\\Bunq\\StageNewHandler->run()
#18 /var/www/firefly/firefly-23052018/app/Http/Controllers/Import/JobStatusController.php(140): FireflyIII\\Import\\Routine\\BunqRoutine->run()
#19 [internal function]: FireflyIII\\Http\\Controllers\\Import\\JobStatusController->start(Object(FireflyIII\\Models\\ImportJob))
#20 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Controller.php(54): call_user_func_array(Array, Array)
#21 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php(45): Illuminate\\Routing\\Controller->callAction('start', Array)
#22 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Route.php(212): Illuminate\\Routing\\ControllerDispatcher->dispatch(Object(Illuminate\\Routing\\Route), Object(FireflyIII\\Http\\Controllers\\Import\\JobStatusController), 'start')
#23 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Route.php(169): Illuminate\\Routing\\Route->runController()
#24 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Router.php(659): Illuminate\\Routing\\Route->run()
#25 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(30): Illuminate\\Routing\\Router->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#26 /var/www/firefly/firefly-23052018/app/Http/Controllers/Import/JobStatusController.php(56): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#27 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(133): FireflyIII\\Http\\Controllers\\Import\\JobStatusController->FireflyIII\\Http\\Controllers\\Import\\{closure}(Object(Illuminate\\Http\\Request), Object(Closure))
#28 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#29 /var/www/firefly/firefly-23052018/app/Http/Controllers/Controller.php(109): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#30 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(133): FireflyIII\\Http\\Controllers\\Controller->FireflyIII\\Http\\Controllers\\{closure}(Object(Illuminate\\Http\\Request), Object(Closure))
#31 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#32 /var/www/firefly/firefly-23052018/app/Http/Middleware/Binder.php(77): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#33 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(151): FireflyIII\\Http\\Middleware\\Binder->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#34 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#35 /var/www/firefly/firefly-23052018/app/Http/Middleware/Range.php(63): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#36 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(151): FireflyIII\\Http\\Middleware\\Range->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#37 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#38 /var/www/firefly/firefly-23052018/app/Http/Middleware/AuthenticateTwoFactor.php(78): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#39 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(151): FireflyIII\\Http\\Middleware\\AuthenticateTwoFactor->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#40 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#41 /var/www/firefly/firefly-23052018/app/Http/Middleware/Authenticate.php(73): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#42 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(151): FireflyIII\\Http\\Middleware\\Authenticate->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#43 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#44 /var/www/firefly/firefly-23052018/vendor/laravel/passport/src/Http/Middleware/CreateFreshApiToken.php(50): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#45 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(151): Laravel\\Passport\\Http\\Middleware\\CreateFreshApiToken->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#46 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#47 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/VerifyCsrfToken.php(67): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#48 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(151): Illuminate\\Foundation\\Http\\Middleware\\VerifyCsrfToken->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#49 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#50 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/View/Middleware/ShareErrorsFromSession.php(49): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#51 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(151): Illuminate\\View\\Middleware\\ShareErrorsFromSession->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#52 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#53 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php(63): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#54 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(151): Illuminate\\Session\\Middleware\\StartSession->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#55 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#56 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/AddQueuedCookiesToResponse.php(37): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#57 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(151): Illuminate\\Cookie\\Middleware\\AddQueuedCookiesToResponse->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#58 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#59 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/EncryptCookies.php(59): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#60 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(151): Illuminate\\Cookie\\Middleware\\EncryptCookies->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#61 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#62 /var/www/firefly/firefly-23052018/app/Http/Middleware/Sandstorm.php(58): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#63 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(151): FireflyIII\\Http\\Middleware\\Sandstorm->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#64 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#65 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(104): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#66 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Router.php(661): Illuminate\\Pipeline\\Pipeline->then(Object(Closure))
#67 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Router.php(636): Illuminate\\Routing\\Router->runRouteWithinStack(Object(Illuminate\\Routing\\Route), Object(Illuminate\\Http\\Request))
#68 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Router.php(602): Illuminate\\Routing\\Router->runRoute(Object(Illuminate\\Http\\Request), Object(Illuminate\\Routing\\Route))
#69 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Router.php(591): Illuminate\\Routing\\Router->dispatchToRoute(Object(Illuminate\\Http\\Request))
#70 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(176): Illuminate\\Routing\\Router->dispatch(Object(Illuminate\\Http\\Request))
#71 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(30): Illuminate\\Foundation\\Http\\Kernel->Illuminate\\Foundation\\Http\\{closure}(Object(Illuminate\\Http\\Request))
#72 /var/www/firefly/firefly-23052018/vendor/fideloper/proxy/src/TrustProxies.php(57): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#73 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(151): Fideloper\\Proxy\\TrustProxies->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#74 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#75 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(31): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#76 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(151): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#77 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#78 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(31): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#79 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(151): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#80 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#81 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php(27): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#82 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(151): Illuminate\\Foundation\\Http\\Middleware\\ValidatePostSize->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#83 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#84 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/CheckForMaintenanceMode.php(51): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#85 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(151): Illuminate\\Foundation\\Http\\Middleware\\CheckForMaintenanceMode->handle(Object(Illuminate\\Http\\Request), Object(Closure))
#86 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))
#87 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(104): Illuminate\\Routing\\Pipeline->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))
#88 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(151): Illuminate\\Pipeline\\Pipeline->then(Object(Closure))
#89 /var/www/firefly/firefly-23052018/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(116): Illuminate\\Foundation\\Http\\Kernel->sendRequestThroughRouter(Object(Illuminate\\Http\\Request))
#90 /var/www/firefly/firefly-23052018/public/index.php(75): Illuminate\\Foundation\\Http\\Kernel->handle(Object(Illuminate\\Http\\Request))
#91 {main}

[JC5]

That seems to be an error within the bunq SDK unfortunately.

[OGKevin]

@JC5 This has to do with #1387 (comment)

[OGKevin]

Ive given you a heads up 🙈

[JC5]

Haha ok I didn’t realize these were connected. Anything I can do?

[OGKevin]

lock the dependency (bunq/sdk_php) in composer to the latest commit on develop until the next release. See https://stackoverflow.com/questions/21314381/how-to-correctly-require-a-specific-commit-in-composer-so-that-it-would-be-avail

[IcyPalm]

Will test it later this afternoon! Great work on it man! @JC5

[OGKevin]

Hmm, it seems that the docker image has no connection to the internet after pulling develop. Not sure if this has something to do with my system or not. Will continue investigating. Might be that the network bridge is broken.

The Curl in the docker container is to old as well.

curl 7.38.0 (x86_64-pc-linux-gnu) libcurl/7.38.0 OpenSSL/1.0.1t zlib/1.2.8 libidn/1.29 libssh2/1.4.3 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API SPNEGO NTLM NTLM_WB SSL libz TLS-SRP

bunq/sdk_php requires curl version 7.54 and up for pinned key support. See bunq/sdk_php#7

[OGKevin]

I will create a PR to update the DockerFile to install install curl accordingly.

[JC5]

👍

I've pushed a bunch of new code. Should be no operational changes, but testability has improved (coverage as well) and composer.lock is pinned to the commit you indicated.

[JC5]

@jonathanjuursema Could you re-test when you have the time?

[paul999]

I have upgraded my instance, and the import seem to be working.

The only thing I saw was that the change done in #1398 has been gone, which is unfortunate as it makes importing a bit hard with multiple accounts.

[OGKevin]

@paul999 can you re-add it ? Or at least give an option to auto create the asset account instead of canceling the import, create the asset account and then re-doing the import.

[paul999]

I already have all of the accounts in my install, but they are set as default option to do not import. IIRC, if I look closely at the code this should already work, but it doesn't actually select the account.

Another things I noticed is that if you do another import, you now get a list of errors when transactions are already imported:

De import is klaar. Unknown import result

There were some errors during the import. Please review them carefully.

Row #0 ("Brood by Alex UTRECHT, NL") could not be imported. It already exists.
Row #1 ("CCV*KAFE BELGIE UTRECHT, NL") could not be imported. It already exists.
Row #2 ("Manneke Pis B.V. UTRECHT, NL") could not be imported. It already exists.
Row #3 ("Sodexo-NL081540 UTRECHT, NL") could not be imported. It already exists.
Row #4 ("") could not be imported. It already exists.
Row #5 ("Brood by Alex UTRECHT, NL") could not be imported. It already exists.
Row #6 ("CCV*Taplokaal Gist B.V UTRECHT, NL") could not be imported. It already exists.
Row #7 ("Sodexo-NL081540 UTRECHT, NL") could not be imported. It already exists.
Row #8 ("BBC Oliver. UTRECHT, NL") could not be imported. It already exists.
Row #9 ("Albert Heijn 1467 UTRECHT, NL") could not be imported. It already exists.
Row #10 ("") could not be imported. It already exists.
Row #11 ("") could not be imported. It already exists.

I also have the idea it is going back a very short timeframe? With 4.7.3 it would go back from the start (I think?) now, it seems to just go back for the last 12? transactions.
While I can see that it might not be a great idea to go back a long time if you already have those accounts from the previous upgrade, as all transactions would be added again (I needed to delete a few transactions as they were imported previously, and now imported again), but in general I just do the import once a week, and I have more then the 12 (Or 15, don't know exactly sure how many transactions it did) so I would miss a bunch of transactions.

[JC5]

@paul999 I have to check but I thought the code is there:

{% if currentIban != '' and currentIban == localAccount.iban %} selected{% endif %}

I will verify if it works as expected.

As for the import: I haven't changed anything in particular but I am not sure what time frame the bunq SDK offers by default. Perhaps I can extend the range or send parameters. I'll do some research.

For future versions (beyond 4.7.4) I can build a time / range select thing.

[paul999]

Yeah, I noticed that code, however when I open the import window it is shown like this:

Just checked to make sure, the filled ibans are correct.

Another thing I noticed, which might be related, when imported a ING sheet, it didn't detect any duplicates. Old transactions that were already imported got imported again ( I am kinda lazy, so always select the first of the month for the exported file from ING, which worked fine in 4.7.3).

[JC5]

Because there are users running this baby on RP2's so 0.3 seconds per transaction is already pushing it. Given the default of 30 seconds. The code should be able to simply keep POST-ing the job until it is finished, so I'd rather do that.

And I'll do that, but don't hit me when I forget 😉

[jonathanjuursema]

Running a bunq import on the most recent develop branch seems to work without any errors on my machine! @JC5, thanks a bunch for all you've done and probably will do. :)

I don't have any duplicates anymore when importing internal transfers, so that's that.

The only thing is that it doesn't import all transactions. It has transactions all the way back to when I started using bunq (Jan 8, 2018), but it seems to be missing random transactions in between. I'm not sure if this is something in Firefly or with the bunq API. Is this something known? A second import doesn't seem to import the missing transactions, but the duplicate detection works flawlessly. :)

[IcyPalm]

I have just had time to test and the import ran without errors!

I am noticing the issue that @jonathanjuursema has as well. I have a transaction almost every day but notice the gaps in here:

I am importing multiple accounts at once.

[OGKevin]

@IcyPalm @jonathanjuursema This is definitely not on bunq's end. If you can see the transactions in the bunq app, then you should be able to see them via the API as well.

[JC5]

@jonathanjuursema Please enable debug mode and either share the log with me (accepting that I may see private info) or try to find the error yourself. The importer will first download everything, and then import transactions. If you need more debug info, let me know. If the transaction is there, downloaded from bunq, it might not be imported later on. Either way, that's what I need to know.

@IcyPalm Same thing really. It could be that the import fails but it should give you feedback on errors, even system errors.

[jonathanjuursema]

@JC5 with APP_ENV=local and APP_DEBUG=true set in .env, and tailing storage/logs/* all messages I get during a bunq import are:

[2018-05-29 19:07:22] local.WARNING: Cannot find anything on empty currency code and empty currency ID!  
[2018-05-29 19:07:22] local.WARNING: Cannot find anything on empty currency code and empty currency ID!  
[2018-05-29 19:07:55] local.WARNING: Cannot find anything on empty currency code and empty currency ID!  

Anywhere else I can check for errors?

[JC5]

Make sure you set the log level to DEBUG.

[jonathanjuursema]

That did the trick. The only thing I can make out that seems strange is a whole bunch of [2018-05-29 21:47:33] local.DEBUG: Array does not have valid opening balance data.. I've also noticed that when I set the opening balance the settings is never applied (the fields are empty again when I return to the asset page). I'm not sure if this is the intended behaviour.

If it helps, I can send you the anonymised log of the import. :)

[IcyPalm]

I quickly scanned through the debug logs but found nothing weird, except the same issue as @jonathanjuursema.

Tomorrow I am going to dive deeper in the logs and anonymise if needed

[JC5]

@jonathanjuursema That error is related to the creation of new expense and revenue accounts. The system is also able to process opening balance information (for asset accounts) and will tell you when that info isn't there. It's not a problem when that information is missing. I can't replicate your opening balance problem on the demo site.

[JC5]

I've pushed some small updates, most notably Firefly III will now download the last 100 transactions.

[jonathanjuursema]

@JC5 I suppose the opening balance issue is because I was trying to set it to €0 on 1/1/2018, which is when I opened my account with bunq. I suppose it doesn't save the info if the opening balance is, €0, since other amounts indeed work fine! I suppose there's also little use for setting an opening balance of €0, coming to think of it.

Also, looking through the bunq API docs, could the issue I and other people have regarding the idea that we were 'missing' some payments be because Firefly pulls 100 transactions per account? So that would mean that I would pull all transactions from an old account I used till February, and only the 100 transactions I did in May, for example, missing all transactions from February, March and April.

[JC5]

Yes. That will be the situation with the next release.

[JC5]

OK, I've pushed the last code. This will enable the rules engine automatically on new bunq imports. As for the "gaps": Firefly III will now download the last 100 transactions per account. There should be no gaps in this download but depending on your history this could mean the import only goes back so far.

Code coverage is complete and everything seems to work just fine. As far as I can see, it's all ready for the release. Probably somewhere during this weekend I will make sure it gets released.

New issues will be created afterwards to tackle the following feature requests.

• Configurable do / do not apply rules. Will be a global config item for bunq, not per-job probably.
• Auto import everything, and keep track of where we were.
• Command line importing (for bunq and other import providers)
• API importing (for bunq and other import providers)

[OGKevin]

@JC5 is it possible to store the payment id that bunq uses to identify this payment ? This will make it convenient for self made tools.

Use Case:
I'm looking at my overview in firefly and i see i payed 10EUR for someone but i never saw this money coming back in, i have a simple script to send requests via the bunq API. I copy the payment id (external references) from firefly and dump it in my script.

Or i dont recognize it and i want to ask bunq support about it etc etc.

So TL;DR it would be nice to store the external reference.

[JC5]

The external reference is stored, but it might not be visible yet. Check the journal_meta table for bunq_id or something. I'm not on my dev-machine so I'm not sure.

[OGKevin]

Yes you're right its indeed there.

So then yes, if this could be shown in the transaction details that would be nice. You can also use this info when importing. Filter ASC to get the latest bunq_payment_id and use that as newer_id. 2 birds one stone 😁

[JC5]

It should be visible under both "external ID", "internal reference" and "bunq payment ID". 😁.

The first two are user-editable.

[JC5]

4.7.4 is live. 🎉

[OGKevin]

@JC5 will you still add support for only importing new transactions ? instead of always the last 100. The red "warnings" after the import is really not needed when not importing via CSV or something along those lines :)

[JC5]

Yes, in 4.7.5 :)