Import via CLI did not match bills

[AndreiGavriliu]

I am running Firefly III version 4.6.9

Description of my issue:

I created a cronjob that downloads all transactions and automatically imports them into FF every 6 hours. I noticed the bills didn't match, until I manually re-scanned the old transactions. Is it possible the CLI import doesn't trigger the matching functions?

[JC5]

Transactions are not applied to bills. This would make the import process too resource heavy.

How about a command line to rescan active bills, with the possibility set a date range?

[AndreiGavriliu]

That would be awesome!

[zjean]

Sligtly different from the CLI part, but matching the import part of this issue:
Would it also be possible to add an option to rescan all bills after the import has finished in the web UI?
Maybe even with a prefilled datum range that spans the imported transactions.
Or maybe an option to match bills for all transactions with a specific tag, since all an imported batch of transactions is always linked to a new tag..

[AndreiGavriliu]

@JC5 has a valid point regarding the resources, but @zjean's idea is also good. Maybe leave it to the user if he wants to wait some more until the transactions have been scanned for bill matches. It would be a bit weird to provide the user a CLI tool to scan all bills but to manually go through all bills to re-scan them via the web interface

[JC5]

I have some ideas. In the future, the import will not scan rules, will not scan bills, and won't do any other (future) extra's. Those must be separate routines running only when the user wants them to.

The import routine would get extra checkboxes for both options.

[AndreiGavriliu]

👍

[Aquariu]

Hi!

Very nice additions indeed. Is there also a timeout inside Firefly different from the PHP timer ? I beleive I've set my php.ini timeout to 3600s but I still get an error after 30s after trying to apply all rules to new transactions.

Also, yes, a single command to apply rules/bills/whatever to the new just-impoted transactions would save some import time. Something like: sudo -u www-data php artisan firefly:apply-rules -tstart "21:10:2017" -tend "26:10:2017" --token=xxxxx would apply the rules to all the transactions in that period.
For instance, with my weekly script, I would just compute the date "today minus 7 days" for the start date, and gavu would have today's date for both start and end times as he/she runs his import script faster than daily.
Or else, there's also probably a way to retrieve the Tags of the last imports (filtered by date ?) and use that to define what transactions need to go through the rules set. It would look like sudo -u www-data php artisan firefly:apply-rules --tag=xxxx --token=yyyy. It looks more complicated this way, though.

I guess we can also have firefly:match-bills and many other things with the same command line parameters.

[JC5]

Regarding this issue: the option to apply rules and/or match bills is now something that must be set in the import configuration file, or during the import routine through the app: either way, the configuration file will end up with two new options:

"column-mapping-complete": false,
"apply_rules": true,
"match_bills": true

The column-mapping-complete should be familiar, the other two are new.

I shall expand the configuration files here to add these options and I suggest you do the same with your configurations.

The current version of Firefly III will simply ignore them. The new version will read and use these config values.

Please note that this is a change from standard behavior: any current imports will no longer have the rules applied, since both options are presumed false when not present.

[AndreiGavriliu]

You are the most “awesomest” person, like ever! It’s like a pre-Christmas present for lazy people!

[AndreiGavriliu]

What is the timespan for the rules, beginning-of-time until now?

[JC5]

They are applied to whatever you're importing. For the rules, it's all the rules and rule-groups you have (as it is already). For bills, all your bills that are active and have auto-match enabled.

Other transactions are not affected.

[AndreiGavriliu]

awesome, thanks! this is amazing!