Skip to content
This repository has been archived by the owner on Dec 21, 2023. It is now read-only.

More CSV Parsing Improvements #1266

Merged
merged 5 commits into from Jan 10, 2019

Conversation

ylow
Copy link
Collaborator

@ylow ylow commented Dec 19, 2018

  • Allow escape_char to be optional: push a use_escape_char option all the
    way through all code.
  • Add csv parsing options true_values and false_values which accept a list of strings to interpret as the numeric value 1, and the numeric value 0 respectively.
  • pushed through na_values to the recursive parsers so that missing values inside of dict/lists are handled correctly.
  • Filled in the read_json parsing options to correctly map na_values = ["null"], true_values=["true"] and false_values=["false"]
  • Better parser diagnostics on parser failure. For instance:
 Unable to interpret ""hello"" as a integer
Parse failed at token ending at:
	2,"hello"^
Successfully parsed 1 tokens:
	0: 2

 - Allow escape_char to be optional: push a use_escape_char option all the
  way through all code.
 - Better parser diagnostics on parser failure. For instance:

```
 Unable to interpret ""hello"" as a integer
Parse failed at token ending at:
	2,"hello"^
Successfully parsed 1 tokens:
	0: 2
```
@ylow ylow requested a review from znation December 19, 2018 23:29
@ylow
Copy link
Collaborator Author

ylow commented Jan 4, 2019

Do not merge. Fixing a CSV issue in #1263 when there are a lot of CPUs.

@ylow
Copy link
Collaborator Author

ylow commented Jan 4, 2019

ok. ready for review.

@ylow ylow merged commit 2e1e61c into apple:master Jan 10, 2019
@nickjong nickjong added this to the 5.3 milestone Jan 17, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants