Windows newlines part of last column

[tonyfischetti]

Hi,
When I use read_csv on a file with dos/windows line endings a carriage return "\r" becomes a part of the last column.

To replicate: I made a csv that looks like this
first,second,
1,2
3,4
then I used unix2dox to convert to dos line endings.

[tonyfischetti]

I'm having trouble following the code flow but it you point me to the general file/files to manipulate I'd be happy to see if I can patch this myself.

[hadley]

@tonyfischetti unfortunately I don't think is a simple fix - the way the tokenizer is currently written would require some new states (STATE_NL, STATE_CR, I think) to do it correctly. (I think this is why people often write a lexer, then a tokenizer).

[tonyfischetti]

Yeah, I was looking into it just now and that's what it seemed like :(

[jayhesselberth]

Wow readr is fast. But this issue is unfortunate because it means readr::read_csv is not a drop-in replacement for utils::read.csv in existing code. I had to fix up csv files with dos2unix first and then works as expected.

[kevinykuo]

+1 this is an issue for us since quite a bit of the data we get come in Excel spreadsheets...

[coolbutuseless]

Just came across this problem as well.

[JeffreySarnoff]

.. and the final column name has '"' postpended