Skip to content

Output eol between ndjson lines#113

Merged
Keyang merged 2 commits intoKeyang:masterfrom
blakeblackshear:master
Sep 12, 2016
Merged

Output eol between ndjson lines#113
Keyang merged 2 commits intoKeyang:masterfrom
blakeblackshear:master

Conversation

@blakeblackshear
Copy link
Copy Markdown
Contributor

While trying to stream a large csv to ndjson, there aren't newlines between each line. Here is an example of the cli command that reproduces the issue:

cat large.csv | csvtojson --noheader=true --flatKeys=true --workerNum=8 --maxRowLength=0 --toArrayString=false --delimiter='|'

Let me know if there is a better way to handle this.

@blakeblackshear
Copy link
Copy Markdown
Contributor Author

Looks like my first attempt had some unintended consequences. This approach passes all tests, but has a side effect of an extra EOL before the ].

@Keyang
Copy link
Copy Markdown
Owner

Keyang commented Sep 11, 2016

Hi,
I dont understand why you would like to add a line breaks after each json and i think this should not be a default behaviour.
I think you better write a simple script for the purpose of yours.

~Keyang

@blakeblackshear
Copy link
Copy Markdown
Contributor Author

For the same reasons stated in this open issue: #87
I am trying to get output in ndjson format

Data in ndjson format can be easily imported into AWS Redshift, etc. It also would be consistent with the compact output of jq.

@Keyang Keyang merged commit a2ba0fb into Keyang:master Sep 12, 2016
Keyang added a commit that referenced this pull request Sep 12, 2016
@Keyang
Copy link
Copy Markdown
Owner

Keyang commented Sep 12, 2016

Looks good. thanks very much.

~Keyang

Keyang added a commit that referenced this pull request Dec 24, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants