Relationship to jsonlines.org

[mcast]

I found http://ndjson.org/ while looking for http://jsonlines.org/ and now I'm puzzled about how they are related to each other.

Thanks for being clear about the fact that ndjson is a fork of jsonlines. Please could you also explain a little about what motivated you to fork? Or how you expect jsonlines to evolve in the presence of ndjson, if that is appropriate?

To a large extent this is just my curiosity & it won't matter whether my next app is throwing .jsonl or .ndjson extensions. However as the standards show signs of diverging (comments on lines, file extension, outer wrapping object to make valid .json) it would be nice to have a clear way to choose, for applications which interact.

Thanks,

Matthew

[notslang]

👍 - I'm wondering the same thing

[finnp]

Hey the website currently is a bit misleading. Only the website styles are a fork, not the standard itself. JSONLines is not a spec, but a website while NDJSON is currently already a living spec. They generally compatible. You can have a look at the spec here: http://dataprotocols.org/ndjson/

[mcast]

@finnp thanks for that clarification, my original question contains fuzziness.

The re-use of website stuff suggested to me a time ordering of the two flavours of multi-object-per-stream JSON, and explains some shared content. It isn't what puzzled me, but is partly why I ask here not there.

The (non)existence of a standard on each side is more significant - for the future of that way of doing it, and awareness of the flavour. It seems like a big plus for ndjson.

Practically, I would appreciate a summary table like

	ndjson	JSON Lines	LDJSON
Name	Newline Delimited JSON	JSON Lines	Line Delimited JSON
Standardised	1.0.0 2013-07-05	No?
Project lifeline	2013-07-05 --	2013-10-02 -- ?	2013-07-02 -- ?
Encoding	UTF8 {M}	UTF8 {M}	UTF8 {M}
Escape non-ASCII characters		If preferred
Object separator	\n {M} or \r\n	\n {M} or \r\n	\r\n {M} or \n or \r
Empty lines (\n\n)	Optional {?}	No {M}
Comments			(discussion)
Line may be		any JSON value
Separator at EOF		Optional
Separator allowed inside object	No {M}	No {M}	Yes incl. pretty-printed
Media type	application/x-ndjson {s}		application/x-ldjson {s} application/json; boundary=NL {c}??
File extension	.ndjson {s}	.jsonl {c} (Formerly .jl but see Julia)	.ldjson or .ldj {s}
Stream compression		.jsonl.gz or .jsonl.bz2 {r}
Object size limits			16MiB {?} (1KiB for embedded)
When to error	...	...

{M} = MUST
{s} = SHOULD
{?} = MAY / Check spec for details.
{r} = recommended
{c} = suggested convention

I pulled values together from what I could find. Some, especially project lifetimes, could be plain wrong. Also there are interesting rows to be added.

Maybe it's useful to show nearby alternatives, e.g.

• Websockets - more complex
• Iteratively parsing off a stream with a standard JSON parser - more permissive, e.g. does allow \n inside object serialisations. Oh, that seems to be LDJSON..?

I also have some interest in the history of each - how they arose and subsequent project evolution; and tools known to use the style.

[pmackay]

I also think that a bit more detail on the history and differences would be useful, as @mcast has outlined in a table.

[josefbetancourt]

There is also the JSON Text Sequences RFC at https://tools.ietf.org/html/rfc7464 This seems to be similar, but JSON is preceded by a record separator: RS JSON LF. I'm not sure how the RS adds better fault tolerance.

[alexshpilkin]

@josefbetancourt: The issue is not exactly fault tolerance, it’s that valid JSON is allowed to contain LFs inside (though only as syntactic whitespace and not inside strings), while RS characters are completely forbidden. So skipping to the next RS is a more-or-less foolproof way to resync corrupted input. Personally, I also find it more convenient to be able to prettyprint my entries instead of scrolling or wrapping 300-character lines, but this is admittedly situational.

FD: Wrote a jsonseq library.

[hattesen]

YAML Streams and JSON Sequences json-ld/yaml-ld#63

@mcast, add to the table...

MIME type

• ndjson: application/x-ndjson.
• JSON Lines: (no official MIME type) - used: application/x-jsonlines, application/jsonlines, application/jsonlines+json. See discussion.
• LDJSON: ´application/x-ldjson`.

Content edited to correct mistakes

[alexshpilkin]

@hattesen If by LDJSON you mean line-delimited JSON, then the MIME type you have quoted is unrelated: application/ld+json is for JSON-LD (“JSON for linked data”), which is ordinary JSON (not a sequence) annotated so as to give it RDF-style (“linked data”) semantics. The plus sign in application/ld+json means “encoded as”, compare image/svg+xml, application/epub+zip etc.

[hattesen]

@alexshpilkin, my bad.
The LDJSON reference was taken from the table in #1 (comment) where the MIME type is listed as `application/x-ldjson´.

I have edited the comment accordingly.

[ssg]

@mcast As I understand, NDJSON is just a new name for LDJSON because LDJSON's name got confused with JSON-LD spec.

Here's the reference:

ndjson/ndjson-spec@c658c26

[ssg]

Now I checked the issues, the author (@finnp) stated that the further efforts should be focused on JSON Lines instead of NDJSON.

[pauldraper]

application/jsonlines+json

is just wrong FYI

[pauldraper]

the author (@finnp) ndjson/ndjson-spec#35 (comment) the further efforts should be focused on JSON Lines instead of NDJSON

That's not what finnp said.

He said:

I'm open to the idea to deprecate the ndjson repository and redirect the website to jsonlines.org (since some places link it)

But the fact that ndjson.org is dead (#24) does suggest JSON Lines is the path forward.