Skip to content

v2.5.6: Fix reading gzipped JSONL files with blank lines

Choose a tag to compare

@github-actions github-actions released this 03 Nov 23:34
· 11 commits to master since this release

Backport of the v3.0.1 fixes to the v2.x series.

🔴 Bug fixes

  • Fix reading gzipped JSONL files with blank lines (#125). read_gzip_jsonl reads the file in binary mode, so blank lines arrived as b"" and weren't skipped. Instead it raised ValueError: Invalid JSON on line N: b''. This included files written by write_gzip_jsonl(..., append=True), which adds a blank separator line, so a srsly write-then-append round trip couldn't be read back. The bug was present since read_gzip_jsonl was added in v2.4.6.
  • Fix line numbers in JSONL errors. The line number in Invalid JSON on line N errors from read_jsonl and read_gzip_jsonl now matches the line in the file. Previously, blank lines and lines skipped as invalid weren't counted.
  • Show the line text, not a bytes repr, in gzipped JSONL errors. Errors from read_gzip_jsonl now show the offending line as text instead of b'...'.
  • Fix json_dumps(data, indent=None), which raised TypeError unless sort_keys=True was also set. indent=None now gives compact output, the same as indent=0.

👥 Contributors

@sueun-dev, @honnibal