hledger-flow refuses to process statements in the year 2011

[lestephane]

I've issued many hledger-flow import commands yesterday, and it could well be that one of them created include files where it shouldn't have. Now i'm unable to import.

~/Finance/import/personal/somebank$ hledger-flow import --sequential
Collecting input files...
Found 12 input files in 0.054998203s. Proceeding with import...
I couldn't find the right number of directories between "import" and the input file:
.../Finance/import/personal/somebank/depot/2011-include.journal

hledger-flow expects to find input files in this structure:
import/owner/bank/account/filestate/year/trxfile

Have a look at the documentation for a detailed explanation:
https://github.com/apauley/hledger-flow#input-files

I don't think it's worth it to figure out (just yet) how I got there.

But my question is: how do I safely remove all the include files generated by previous hledger-flow import runs, so that I may start fresh, from a known good state? Is the command below a good first approximation?

find . -type f | grep -E "[0-9]{4}-include.journal|all-years.journal" | xargs rm

[apauley]

I usually remove all the generated include files first:
$ grep -rl 'Generated by hledger-.*DO NOT EDIT' . | xargs rm

Then I may also want to remove the preprocessed files and the generated journals:
$ find . -type f -path '*2-preprocessed*' -or -type f -path '*3-journal*'|xargs rm

This should be safe if all your files are committed to version control

[apauley]

I've issued many hledger-flow import commands yesterday, and it could well be that one of them created include files where it shouldn't have. Now i'm unable to import.

~/Finance/import/personal/somebank$ hledger-flow import --sequential

I'm looking at your current directory in the prompt, and it seems that you've changed to a subdirectory.

hledger-flow assumes the basedir is your current dir, unless you've specified the basedir on the command-line.

So you probably don't need to remove any files, just cd back to the basedir.

I can see this is probably surprising behaviour, we should consider being smarter at detecting the basedir.

It will also help to have an option that can print out with which options hledger-flow is running (issue #11).

[apauley]

So just after I wrote the above I tried it out on the example data, and it seems that I'm wrong, it does work when running it in a deeper dir, I have just never tried it before.

I think the issue is related to the year 2011.
Yes, hledger-flow doesn't support 2011, sorry. Not a fan of that year :-)

Specifically, it looks for input files with this find function: find (has (suffix "1-in")).
And this unfortunately also matches an include file instead of only input files: 2011-include.journal

We also won't support 2021, 2001, 1991 and any year ending with a 1.

hledger-flow/src/Hledger/Flow/CSVImport.hs

Line 31 in f211c27

(inputFiles, diff) <- time $ single . shellToList . onlyFiles $ find (has (suffix "1-in")) $ baseDir opts

[lestephane]

interesting. I did manage to run the command once (after cleaning), but never twice. This explains it.

[apauley]

@lestephane Can you test if the latest release solves the 2001 issue?

https://github.com/apauley/hledger-flow/releases/tag/v0.11.1.1-beta

[lestephane]

Looks fine now, I'm able to run the import command more than once without errors.