Pre-processing of CSV

[elChapoSing]

Hello,

Great job on the library it's incredibly useful.

Would it make sense to add a pre-processing sequence the same way you have different mappings ?

My bank has incredibly badly formed CSV so I need to pre-process it to have it match the format that csv2ofx can understand. I am assuming this is something that a lot of people have encountered.

The code would look something like :

from __future__ import absolute_import, print_function

import itertools as it

from meza.io import read_csv, IterStringIO
from csv2ofx import utils
from csv2ofx.ofx import OFX
from csv2ofx.mappings.default import mapping

#new import
from csv2ofx.processing.default import pre_process
#new import

ofx = OFX(mapping)

#to be used here
records = read_csv(pre_process('path/to/file.csv'))
#to be used here

groups = ofx.gen_groups(records)
trxns = ofx.gen_trxns(groups)
cleaned_trxns = ofx.clean_trxns(trxns)
data = utils.gen_data(cleaned_trxns)
content = it.chain([ofx.header(), ofx.gen_body(data), ofx.footer()])

for line in IterStringIO(content):
    print(line)

where pre_process would be a function taking a path and returning a StringIO.
You could then have pre-processing for a bunch of counterparties that do not know how to output proper CSV.
Does any of this make sense ?

[reubano]

CR #83

[reubano]

I'd rather not complicate this script by doing additional pre-processing. There are various commands you can run your file through before piping to csv2ofx. E.g. to strip the last 2 lines of a csv file, head -n -2 transactions.csv | csv2ofx -x /path/to/mapping.py