add to ndjson and to jsonl to the standard library
#10519
Conversation
amtoine
added
std-library
|
|
||
| # Convert structured data to JSONL. | ||
| def "to jsonl" []: any -> string { | ||
| each { to json --raw } | to text |
There was a problem hiding this comment.
Does it make sense to define this in terms of to ndjson? Or do we want to avoid the small performance hit in the std-lib
There was a problem hiding this comment.
tbh, i just copied the same idea from the from ... commands, i.e. they do not call the other one 😋
crates/nu-std/std/formats.nu
Outdated
| @@ -9,12 +9,22 @@ | |||
| # These functions help `open` the files with unsupported extensions such as ndjson. | |||
| # | |||
|
|
|||
| # Convert from ndjson to structured data. | |||
| # Convert from NDJSON to structured data. | |||
There was a problem hiding this comment.
Should the help be more explicit?
In theory we could point to the "spec" http://ndjson.org/
| # Convert from NDJSON to structured data. | |
| # Convert from newline-delimited JSON to structured data. |
There was a problem hiding this comment.
i went with a259805, is that ok? 😇
|
Suspicious test failures around the Weird: |
|
Marking as draft until the custom |
this commit - adds space in the expected string output of `to ...` - adds explicite newlines for the same reason > **Note** > we could go with `str trim`, but i wanted to be the output as explicit > and clear as possible 😌
|
@sholderbach |
|
ooh no.... not windows again 😱 |
|
OK so a fix to harmonize line ending and we should be good to go? Does #10429 still persist after the |
|
not really 🤔 after a use std formats "to ndjson"the conversion itself is working fine > ls | first 2 | to ndjson
{"name": "CODE_OF_CONDUCT.md","type": "file","size": 3444,"modified": "2023-04-20 19:41:30.434104143 +02:00"}
{"name": "CONTRIBUTING.md","type": "file","size": 18786,"modified": "2023-09-10 18:34:32.906018569 +02:00"}but > ls | first 2 | save ls.ndjson
Error: × Internal error: can't run custom command with 'run', use block_id |
|
and about the failing test, i think i know what's happening... 😮💨 |
this will prevent having to do a distinction between `\n` on linux and `\r\n` on windows and should fix the CI.
|
that's such a pain... |
|
with 8d5f21c, i see that all standard library tests have passed 🙏 this is then ready for review and CI should go full green very soon 💪 |
follow up to - nushell#10283 # Description even though it appears defining `to foo` does not allow to do `save x.foo` for free (see nushell#10429), because nushell#10283 did add `from ndjson` and `from jsonl` to the standard library, i thought adding their `to ...` counterpart would make sense :yum: # User-Facing Changes users can now convert structured data back to NDJSON and JSONL :ok_hand: # Tests + Formatting this PR adds the exact same tests as for the `from ...` commands - structured data is in `result` and the string is now the expected - the two invalid `from ...` tests cannot be reproduced for `to ...` afaik # After Submitting
follow up to
Description
even though it appears defining
to foodoes not allow to dosave x.foofor free (see #10429), because #10283 did addfrom ndjsonandfrom jsonlto the standard library, i thought adding theirto ...counterpart would make sense 😋User-Facing Changes
users can now convert structured data back to NDJSON and JSONL 👌
Tests + Formatting
this PR adds the exact same tests as for the
from ...commandsresultand the string is now the expectedfrom ...tests cannot be reproduced forto ...afaikAfter Submitting