Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fread may sometimes detect incorrect newline character #1343

Closed
st-pasha opened this issue Oct 12, 2018 · 0 comments · Fixed by #2436
Closed

fread may sometimes detect incorrect newline character #1343

st-pasha opened this issue Oct 12, 2018 · 0 comments · Fixed by #2436
Assignees
@st-pasha
Labels
bug Any bugs / errors in datatable; however for severe bugs use [segfault] label fread Issues related to parsing any input files via fread function low priority Low priority tasks
Projects
fread
Milestone
Release 0.11.0

Comments

@st-pasha
Copy link
Contributor

st-pasha commented Oct 12, 2018 •
edited
Loading

File having \r newline characters, but also \n in one of the fields (correctly quoted) is not parsed correctly:

>>> dt.fread("a,b,c\r1,4,foo\r7,99,\"wha\ntt?\"\r4,-1,nvm")
   | a         b  c.1   4  foo.7  99  "wha
-- + -------  --  ---  --  -----  --  ----
 0 | tt?"\r4  -1  nvm  NA     NA  NA    NA

[1 row x 7 columns]
@st-pasha st-pasha added bug Any bugs / errors in datatable; however for severe bugs use [segfault] label fread Issues related to parsing any input files via fread function low priority Low priority tasks labels Oct 12, 2018
@st-pasha st-pasha self-assigned this Oct 12, 2018
@st-pasha st-pasha added this to To Do in fread Nov 15, 2018
@st-pasha st-pasha mentioned this issue Oct 10, 2019
Closed
@st-pasha st-pasha mentioned this issue Jan 4, 2020
Closed
27 tasks
@st-pasha st-pasha mentioned this issue May 13, 2020
Merged
fread automation moved this from To Do to Done May 14, 2020
st-pasha added a commit that referenced this issue May 14, 2020
@st-pasha
Improve fread EOL detection (#2436)
8193162 
Improved logic for newline detection in fread. Now files with \r-newlines are recognized more reliably.

Closes #1343
Closes #2074
@st-pasha st-pasha added this to the Release 0.11.0 milestone Sep 18, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@st-pasha st-pasha

Labels
bug Any bugs / errors in datatable; however for severe bugs use [segfault] label fread Issues related to parsing any input files via fread function low priority Low priority tasks
Projects
fread
  
Done
Milestone
Release 0.11.0
Development

Successfully merging a pull request may close this issue.

Improve fread EOL detection h2oai/datatable
1 participant
@st-pasha