Windows line endings not working in CSVInputFormat

[markus-h]

A user reported this issue, who is running stratosphere under windows. After converting the input file to UNIX line endings it worked.

[rmetzger]

I'm currently preparing a pull request to ease debugging these issues.

[markus-h]

I had a look at this issue and for me CsvInputFormat seems to work as intended. The problem probably is, that the DEFAULT_LINE_DELIMITER is set to "\n", but files created under windows are using "\r\n".
I am not sure how to deal with this. We could set the default delimiter to System.getProperty("line.separator"); but then for example files checkout out from git would not be running by default because git is keeping them in linux format.
Another possibility would be to accept both by default and dynamically remove a tailing "\r" if it is present.

[rmetzger]

I think its okay to accept both IF the user is Windows.

[StephanEwen]

We had a fix for this. Did it get lost during some merge?

[rmetzger]

Thats the pull request: #582

[StephanEwen]

Have a look at #582