Summary
A line containing null ends the stream. Every record after it is silently dropped, and the stream emits stream.push() after EOF instead of anything that points at the cause.
null is valid JSON and valid NDJSON, so this is data loss on well-formed input.
Reproduction
import { Readable } from "node:stream";
import ndjson from "ndjson";
const input = '{"id":1}\n{"id":2}\nnull\n{"id":3}\n{"id":4}\n{"id":5}\n';
const rows = [];
Readable.from([input])
.pipe(ndjson.parse())
.on("data", (d) => rows.push(d))
.on("error", (e) => console.log("error :", e.message))
.on("end", () => console.log("rows :", rows.length));
error : stream.push() after EOF
rows : 2
Five valid records in, two out. {"id":3}, {"id":4} and {"id":5} are gone.
Cause
parseRow returns whatever JSON.parse produced, and split2 pushes that value downstream:
function parseRow (row) {
try {
if (row) return JSON.parse(row)
} catch (e) {
...
}
}
return split(parseRow, opts)
For the row null, JSON.parse returns null — and in Node object-mode streams push(null) is the end-of-stream signal. So the stream terminates early, and the next row that arrives triggers stream.push() after EOF.
The empty-string case is already guarded by if (row); null just isn't covered by it.
Suggested fix
null carries no record, so skipping it matches how blank lines are already handled. Returning undefined makes split2 skip the row:
function parseRow (row) {
try {
if (row) {
const parsed = JSON.parse(row)
return parsed === null ? undefined : parsed
}
} catch (e) {
if (opts.strict) {
this.emit('error', new Error('Could not parse row ' + row.slice(0, 50) + '...'))
}
}
}
With that, the repro above emits all five records.
If you'd rather surface it than skip it, emitting a parse-style error under opts.strict would work too — either is better than truncating the stream. Happy to send a PR for whichever you prefer.
Two smaller things found alongside
Not data loss, but same area, in case they're useful:
- A whitespace-only line (
" ") throws Could not parse row, while a truly empty line is skipped. Files written with trailing spaces hit this.
- A leading UTF-8 BOM makes the first row fail to parse. Common with files produced by Windows tooling.
Tested against ndjson@2.0.0 on Node 24.
Summary
A line containing
nullends the stream. Every record after it is silently dropped, and the stream emitsstream.push() after EOFinstead of anything that points at the cause.nullis valid JSON and valid NDJSON, so this is data loss on well-formed input.Reproduction
Five valid records in, two out.
{"id":3},{"id":4}and{"id":5}are gone.Cause
parseRowreturns whateverJSON.parseproduced, and split2 pushes that value downstream:For the row
null,JSON.parsereturnsnull— and in Node object-mode streamspush(null)is the end-of-stream signal. So the stream terminates early, and the next row that arrives triggersstream.push() after EOF.The empty-string case is already guarded by
if (row);nulljust isn't covered by it.Suggested fix
nullcarries no record, so skipping it matches how blank lines are already handled. Returningundefinedmakes split2 skip the row:With that, the repro above emits all five records.
If you'd rather surface it than skip it, emitting a parse-style error under
opts.strictwould work too — either is better than truncating the stream. Happy to send a PR for whichever you prefer.Two smaller things found alongside
Not data loss, but same area, in case they're useful:
" ") throwsCould not parse row, while a truly empty line is skipped. Files written with trailing spaces hit this.Tested against
ndjson@2.0.0on Node 24.