Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CSV read stops #1375

Closed
joohokim1 opened this issue Feb 1, 2019 · 1 comment
Closed

CSV read stops #1375

joohokim1 opened this issue Feb 1, 2019 · 1 comment
Assignees
Labels
bug Something isn't working @dataprep Component Name : Data preparation
Milestone

Comments

@joohokim1
Copy link
Contributor

joohokim1 commented Feb 1, 2019

Describe the bug
When an exception occurs from Apache Common CSV library, the reader stops.
Someone might continue work with the shorter result, without noticing the shortage.

To Reproduce
Steps to reproduce the behavior:

  1. Import finefood.sample.txt
  2. Set delimiter as colon.
  3. Click "Next", then "Done"
  4. I.DS row count is 8000. (Correct!)
  5. Import again, with comma delimiter.
  6. I.DS row count is far less than 8000 (4183 in the screenshot)

This is because the CSV parser has stopped when it ran into wrong quotations between delimiters.

Expected behavior
I.DS row count should be more than 7000 skipping the ill-quoted lines.

Screenshots
2019-02-01 4 09 41

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Additional context
BTW, whole empty lines are skipped by default.

@joohokim1 joohokim1 added bug Something isn't working @dataprep Component Name : Data preparation labels Feb 1, 2019
@joohokim1 joohokim1 added this to the 3.2.0 milestone Feb 1, 2019
@joohokim1 joohokim1 self-assigned this Feb 1, 2019
@joohokim1
Copy link
Contributor Author

1st integrated test passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working @dataprep Component Name : Data preparation
Projects
None yet
Development

No branches or pull requests

1 participant