Skip to content

Ignore blank lines when reading JSONL#1479

Merged
romanlutz merged 1 commit intomicrosoft:mainfrom
biefan:fix-jsonl-blank-lines
Mar 18, 2026
Merged

Ignore blank lines when reading JSONL#1479
romanlutz merged 1 commit intomicrosoft:mainfrom
biefan:fix-jsonl-blank-lines

Conversation

@biefan
Copy link
Copy Markdown
Contributor

@biefan biefan commented Mar 16, 2026

Summary

This makes read_jsonl() tolerate blank lines in JSONL input.

Problem

read_jsonl() currently does a direct json.loads(line) for every line in the file.

That means a JSONL file with an empty line or a whitespace-only separator fails immediately with JSONDecodeError, even when every non-empty line is valid JSON.

In practice, blank lines are common in generated or hand-edited JSONL files, especially at the end of a file or between entries during review.

Fix

Skip blank / whitespace-only lines before decoding:

json.loads(line) for line in file if line.strip()

This preserves the existing behavior for valid JSONL records while making the reader robust to common formatting noise.

Tests

Added regression coverage for a JSONL stream containing a blank line between two valid entries.

Validation command:

.venv/bin/pytest tests/unit/common/test_json_helper.py -q

Validation result:

1 passed in 0.01s

@romanlutz romanlutz merged commit 7ab138b into microsoft:main Mar 18, 2026
35 of 38 checks passed
riyosha pushed a commit to riyosha/PyRIT that referenced this pull request Mar 24, 2026
jbolor21 pushed a commit to jbolor21/jbolor-PyRIT that referenced this pull request Mar 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants