created cash.csv is shown as invalid in Homebank's import menu #3

Ablesius · 2019-10-12T15:31:19Z

Hi,

unfortunately, the script doesn't seem to produce valid output anymore :(

Homebank Version: 5.2.8
Python Version: 3.7.4
Env: Arch Linux (Kernel 5.3.4-arch1-1-ARCH)

Ablesius · 2019-10-12T15:56:01Z

@hamvocke I tested a little more since I saw the last commit is about the encoding:

alex@T460:~/Informatik/Development/dkb2homebank$ ./dkb2homebank.py --cash /tmp/test_20190909-20191014.csv 
Traceback (most recent call last):
  File "./dkb2homebank.py", line 122, in <module>
    main()
  File "./dkb2homebank.py", line 115, in main
    convertDkbCash(args.filename)
  File "./dkb2homebank.py", line 51, in convertDkbCash
    reader = csv.DictReader(transactionLines(csvfile), dialect=dialect, fieldnames=dkbFieldNames)
  File "/usr/lib/python3.7/csv.py", line 87, in __init__
    self.reader = reader(f, dialect, *args, **kwds)
TypeError: argument 1 must be an iterator
alex@T460:~/Informatik/Development/dkb2homebank$ ./dkb2homebank.py --cash /tmp/test.csv 
Traceback (most recent call last):
  File "./dkb2homebank.py", line 122, in <module>
    main()
  File "./dkb2homebank.py", line 115, in main
    convertDkbCash(args.filename)
  File "./dkb2homebank.py", line 51, in convertDkbCash
    reader = csv.DictReader(transactionLines(csvfile), dialect=dialect, fieldnames=dkbFieldNames)
  File "/usr/lib/python3.7/csv.py", line 87, in __init__
    self.reader = reader(f, dialect, *args, **kwds)
TypeError: argument 1 must be an iterator
alex@T460:~/Informatik/Development/dkb2homebank$ ./dkb2homebank.py --cash /tmp/test2.csv 
Traceback (most recent call last):
  File "./dkb2homebank.py", line 122, in <module>
    main()
  File "./dkb2homebank.py", line 115, in main
    convertDkbCash(args.filename)
  File "./dkb2homebank.py", line 51, in convertDkbCash
    reader = csv.DictReader(transactionLines(csvfile), dialect=dialect, fieldnames=dkbFieldNames)
  File "/usr/lib/python3.7/csv.py", line 87, in __init__
    self.reader = reader(f, dialect, *args, **kwds)
TypeError: argument 1 must be an iterator

So I'm not exactly sure what the problem with my test csvs might be but there seems to be a bug in the code. I'll try to dig further when I find the time.

hamvocke · 2019-10-12T16:23:48Z

Thanks for reporting.

The most recent change around encoding should not be a problem as long as you're using Python 3 (and apparently you are).

Did you verify that your input CSV file (the one you downloaded from DKB) actually contains data? Does it look sane in some way to you?

Edit: Can reproduce the issue when the input CSV file doesn't contain the required header line. Can you open your downloaded .csv file and check that it contains a line similar to this one?

"Buchungstag";"Wertstellung";"Buchungstext";"Auftraggeber / Begünstigter";"Verwendungszweck";"Kontonummer";"BLZ";"Betrag (EUR)";"Gläubiger-ID";"Mandatsreferenz";"Kundenreferenz";

I just downloaded a fresh csv file for my account and could convert it as expected.

Ablesius · 2019-10-12T19:01:43Z

Hi,

my input file didn't contain the header. I'll try again in a few moments :)

Ablesius · 2019-10-12T19:09:55Z

@hamvocke unfortunately it doesn't work with the header line, either. I'll mask my input csv and share it, maybe you can gain some more insights from that

Buchungstag,Wertstellung,Buchungstext,Auftraggeber / Begünstigter,Verwendungszweck,Kontonummer,BLZ,Betrag (EUR),Gläubiger-ID,Mandatsreferenz,Kundenreferenz
01.10.2011,01.10.2011,GUTSCHR. UEBERWEISUNG,MR FOO BAR,Foo-Verwendung,DE12345678901234567890,BYLADEM1001,10,,,
01.10.2012,01.10.2012,Kartenzahlung/-abrechnung,FOOMARKT,Foo-Verwendung,DE12345678901234567890,BYLADEM1002,20,,,"0,45678912345678A-1231"
01.10.2013,01.10.2013,Kartenzahlung/-abrechnung,CAFE FOO,Foo-Verwendung,DE12345678901234567890,BYLADEM1003,-10,,,"0,45678912345678A-1232"
01.10.2014,01.10.2014,Kartenzahlung/-abrechnung,CAFE FOO,Foo-Verwendung,DE12345678901234567890,BYLADEM1004,-20,,,"0,45678912345678A-1233"
01.10.2015,01.10.2015,Kartenzahlung/-abrechnung,CAFE FOO,Foo-Verwendung,DE12345678901234567890,BYLADEM1005,10,,,"0,45678912345678A-1234"
01.10.2016,01.10.2016,Dauerauftrag,DKB VISACARD,9876543210987654 12.34  MR FOO BAR  DKB INTERNET BANKING,DE12345678901234567890,BYLADEM1006,20,,,NOTPROVIDED
01.10.2017,01.10.2017,Lastschrift,FOO-Versicherung,Foo-Verwendung,DE12345678901234567890,BYLADEM1007,-10,DE123XX1234567891254ZZZ00000028679,12ASDF12345678912345600MREF000000016244411 ,"0,45678912345678A-1234"
01.10.2018,01.10.2018,Kartenzahlung/-abrechnung,FOOHAUSEN,Foo-Verwendung,DE12345678901234567890,BYLADEM1008,-20,,,"0,45678912345678A-1235"
01.10.2019,01.10.2019,Lastschrift,FOO-Versicherung,Foo-Verwendung,DE12345678901234567890,BYLADEM1009,-20,DE45XXX1234567891296ZZZ00000031124,"0,45678912345678A-1231,50724557388865E+019","0,45678912345678A-1236"

Ablesius · 2019-10-12T19:19:33Z

Ah, just saw it. delimiter and quotechar are fixed in your code and not derived from the input. I'll try converting and running the script again.

Ablesius · 2019-10-12T19:26:36Z

OK, I'm still not getting it.
These are the settings I'm using in libreoffice to convert my table to csv now:

The output csv looks like this:

"Buchungstag";"Wertstellung";"Buchungstext";"Auftraggeber / Begünstigter";"Verwendungszweck";"Kontonummer";"BLZ";"Betrag (EUR)";"Gläubiger-ID";"Mandatsreferenz";"Kundenreferenz"
"01.10.2011";"01.10.2011";"GUTSCHR. UEBERWEISUNG";"MR FOO BAR";"Foo-Verwendung";"DE12345678901234567890";"BYLADEM1001";10;;;
"01.10.2012";"01.10.2012";"Kartenzahlung/-abrechnung";"FOOMARKT";"Foo-Verwendung";"DE12345678901234567890";"BYLADEM1002";20;;;"0,45678912345678A-1231"
"01.10.2013";"01.10.2013";"Kartenzahlung/-abrechnung";"CAFE FOO";"Foo-Verwendung";"DE12345678901234567890";"BYLADEM1003";-10;;;"0,45678912345678A-1232"
"01.10.2014";"01.10.2014";"Kartenzahlung/-abrechnung";"CAFE FOO";"Foo-Verwendung";"DE12345678901234567890";"BYLADEM1004";-20;;;"0,45678912345678A-1233"
"01.10.2015";"01.10.2015";"Kartenzahlung/-abrechnung";"CAFE FOO";"Foo-Verwendung";"DE12345678901234567890";"BYLADEM1005";10;;;"0,45678912345678A-1234"
"01.10.2016";"01.10.2016";"Dauerauftrag";"DKB VISACARD";"9876543210987654 12.34  MR FOO BAR  DKB INTERNET BANKING";"DE12345678901234567890";"BYLADEM1006";20;;;"NOTPROVIDED"
"01.10.2017";"01.10.2017";"Lastschrift";"FOO-Versicherung";"Foo-Verwendung";"DE12345678901234567890";"BYLADEM1007";-10;"DE123XX1234567891254ZZZ00000028679";"12ASDF12345678912345600MREF000000016244411 ";"0,45678912345678A-1234"
"01.10.2018";"01.10.2018";"Kartenzahlung/-abrechnung";"FOOHAUSEN";"Foo-Verwendung";"DE12345678901234567890";"BYLADEM1008";-20;;;"0,45678912345678A-1235"
"01.10.2019";"01.10.2019";"Lastschrift";"FOO-Versicherung";"Foo-Verwendung";"DE12345678901234567890";"BYLADEM1009";-20;"DE45XXX1234567891296ZZZ00000031124";"0,45678912345678A-1231,50724557388865E+019";"0,45678912345678A-1236"

But Homebank still doesn't take it.

hamvocke · 2019-10-14T07:01:59Z

Hmm... this is weird.

If I take your provided input file:

Buchungstag,Wertstellung,Buchungstext,Auftraggeber / Begünstigter,Verwendungszweck,Kontonummer,BLZ,Betrag (EUR),Gläubiger-ID,Mandatsreferenz,Kundenreferenz
01.10.2011,01.10.2011,GUTSCHR. UEBERWEISUNG,MR FOO BAR,Foo-Verwendung,DE12345678901234567890,BYLADEM1001,10,,,
01.10.2012,01.10.2012,Kartenzahlung/-abrechnung,FOOMARKT,Foo-Verwendung,DE12345678901234567890,BYLADEM1002,20,,,"0,45678912345678A-1231"
01.10.2013,01.10.2013,Kartenzahlung/-abrechnung,CAFE FOO,Foo-Verwendung,DE12345678901234567890,BYLADEM1003,-10,,,"0,45678912345678A-1232"
01.10.2014,01.10.2014,Kartenzahlung/-abrechnung,CAFE FOO,Foo-Verwendung,DE12345678901234567890,BYLADEM1004,-20,,,"0,45678912345678A-1233"
01.10.2015,01.10.2015,Kartenzahlung/-abrechnung,CAFE FOO,Foo-Verwendung,DE12345678901234567890,BYLADEM1005,10,,,"0,45678912345678A-1234"
01.10.2016,01.10.2016,Dauerauftrag,DKB VISACARD,9876543210987654 12.34  MR FOO BAR  DKB INTERNET BANKING,DE12345678901234567890,BYLADEM1006,20,,,NOTPROVIDED
01.10.2017,01.10.2017,Lastschrift,FOO-Versicherung,Foo-Verwendung,DE12345678901234567890,BYLADEM1007,-10,DE123XX1234567891254ZZZ00000028679,12ASDF12345678912345600MREF000000016244411 ,"0,45678912345678A-1234"
01.10.2018,01.10.2018,Kartenzahlung/-abrechnung,FOOHAUSEN,Foo-Verwendung,DE12345678901234567890,BYLADEM1008,-20,,,"0,45678912345678A-1235"
01.10.2019,01.10.2019,Lastschrift,FOO-Versicherung,Foo-Verwendung,DE12345678901234567890,BYLADEM1009,-20,DE45XXX1234567891296ZZZ00000031124,"0,45678912345678A-1231,50724557388865E+019","0,45678912345678A-1236"

and convert it via ./dkb2homebank.py --cash myfile.csv I'm getting an output file that Homebank will happily import. Important note is that you'll have to use "File > Import", not "File > Open" but I assume that's what you're doing?

Tested with Homebank 5.2.8 and Python 3.7.4.

For completeness, this is what the converted csv output file looks like for me:

01-10-2011;8;;MR FOO BAR;Foo-Verwendung;10;;
01-10-2012;8;;FOOMARKT;Foo-Verwendung;20;;
01-10-2013;8;;CAFE FOO;Foo-Verwendung;-10;;
01-10-2014;8;;CAFE FOO;Foo-Verwendung;-20;;
01-10-2015;8;;CAFE FOO;Foo-Verwendung;10;;
01-10-2016;8;;DKB VISACARD;9876543210987654 12.34  MR FOO BAR  DKB INTERNET BANKING;20;;
01-10-2017;8;;FOO-Versicherung;Foo-Verwendung;-10;;
01-10-2018;8;;FOOHAUSEN;Foo-Verwendung;-20;;
01-10-2019;8;;FOO-Versicherung;Foo-Verwendung;-20;;

Ablesius · 2019-10-20T20:04:08Z

Hi @hamvocke,

I tried it again by copying the exact input you provided here and used the exact outfile for the Import (not Open, like you mentioned), but I get this one-way street sign indicating an error, and can't click "Next" in the Import dialogue.
Also re-checked my versions of both Python and Homebank and got the same.

I also tried converting the files from the testfiles directory. The conversion works for all but the empty Cash one (which makes sense), but the created files can't be imported either. Right now I'm a bit confused, not sure what else I could test.

Ablesius · 2019-11-12T10:21:48Z

hi @hamvocke,

I was finally able to figure out the problem. I had changed the setting for Separator in Homebank to "Comma" instead of "Semicolon". Turns out this is the problem. Thanks for investigating with me.

hamvocke · 2019-11-12T10:34:44Z

Glad that you worked it out. Let me add this as a hint to the readme, just in case someone will run into this issue in the future. Thanks for reporting back!

Ablesius closed this as completed Nov 12, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

created cash.csv is shown as invalid in Homebank's import menu #3

created cash.csv is shown as invalid in Homebank's import menu #3

Ablesius commented Oct 12, 2019 •

edited

Loading

Ablesius commented Oct 12, 2019

hamvocke commented Oct 12, 2019 •

edited

Loading

Ablesius commented Oct 12, 2019

Ablesius commented Oct 12, 2019 •

edited

Loading

Ablesius commented Oct 12, 2019

Ablesius commented Oct 12, 2019

hamvocke commented Oct 14, 2019

Ablesius commented Oct 20, 2019

Ablesius commented Nov 12, 2019

hamvocke commented Nov 12, 2019

created cash.csv is shown as invalid in Homebank's import menu #3

created cash.csv is shown as invalid in Homebank's import menu #3

Comments

Ablesius commented Oct 12, 2019 • edited Loading

Ablesius commented Oct 12, 2019

hamvocke commented Oct 12, 2019 • edited Loading

Ablesius commented Oct 12, 2019

Ablesius commented Oct 12, 2019 • edited Loading

Ablesius commented Oct 12, 2019

Ablesius commented Oct 12, 2019

hamvocke commented Oct 14, 2019

Ablesius commented Oct 20, 2019

Ablesius commented Nov 12, 2019

hamvocke commented Nov 12, 2019

Ablesius commented Oct 12, 2019 •

edited

Loading

hamvocke commented Oct 12, 2019 •

edited

Loading

Ablesius commented Oct 12, 2019 •

edited

Loading