New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
csvjson Supporting streaming json documents #355
Comments
jennifersmith
pushed a commit
to jennifersmith/csvkit
that referenced
this issue
Nov 4, 2014
This is to address issue wireservice#355 and to allow json to be 'streamed' - that is, each object in the array be written out separated by a newline, rather than output as one single array. Allowing this means we can convert large csv streams to json without waiting until the end to see the output. Many json-processing tools can deal with streaming json (for instance jq).
jennifersmith
pushed a commit
to jennifersmith/csvkit
that referenced
this issue
Nov 4, 2014
This is to address issue wireservice#355 and to allow json to be 'streamed' - that is, each object in the array be written out separated by a newline, rather than output as one single array. Allowing this means we can convert large csv streams to json without waiting until the end to see the output. Many json-processing tools can deal with streaming json (for instance jq).
jennifersmith
pushed a commit
to jennifersmith/csvkit
that referenced
this issue
Nov 4, 2014
This is to address issue wireservice#355 and to allow json to be 'streamed' - that is, each object in the array be written out separated by a newline, rather than output as one single array. Allowing this means we can convert large csv streams to json without waiting until the end to see the output. Many json-processing tools can deal with streaming json (for instance jq).
Hi Jennifer, nice add! And 👍 on supporting large documents. |
Merged pull request. Thanks! |
Cool - thanks for that! |
lcorbasson
pushed a commit
to lcorbasson/csvkit
that referenced
this issue
Sep 7, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi there,
Great set of utilities!
I was just wondering if there would be a problem with supporting 'streaming' json from the csv2json utility.
Similar to #275 , I would like to emit json without the surrounding array notation, so :
This would mean that for a large csv file, there isn't the necessity to pull it all into memory first.
Ideally this would be done via an optional --stream option that defaults to false.
Happy to work on a PR if that sounds like something csvkit could/should do.
The text was updated successfully, but these errors were encountered: