Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
Line tokenizer: Invalid quote char [BATCH-2820] #794
I'm using a FlatFileItemReaderBuilder to transfer some CSV files (in Brazilian Portuguese) to a MongoDB Database. In one of my records I have a line formatted as an example bellow:
03.730.263/0001-44;INSTITUCIONAL "T' FUNDO DE INVESTIMENTO EM AÇÕES;2005-03-22;2000-03-23..........
Because the only one quote char (") in the line, the tokenizer will not create all tokens. It occurs because he tries to find another quote char e there's no another char in the line.
The file that I'm trying to load: http://dados.cvm.gov.br/dados/FI/CAD/DADOS/inf_cadastral_fi_20190503.csv
In my case, there is an error in the csv file (i.e. only one char "). I tried to create more unit tests in the file +DelimitedLineTokenizerTests.java+ but this situation is kind a paradox, see below:
In my case, I put a invalid break line as a quoted char, that works me.
val delimiter = DelimitedLineTokenizer(";") delimiter.setQuoteCharacter('\n')