fix(llm): make JSON extraction string-aware so braces in text cannot drop findings#151
Conversation
…drop findings
parse_llm_json counted braces without tracking string context, so any
'}' inside a JSON string value truncated the object mid-string; the
resulting decode error was swallowed upstream and the whole structured
result (all findings) was silently discarded, leaving only raw text.
Use json.JSONDecoder().raw_decode, trying candidate '{' positions in
order — string/escape aware, and also recovers when a stray '{'
precedes the real object.
Fixes #136
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W4svt5QTs4WUMSy4HwiVt9
|
Warning Review limit reached
Next review available in: 39 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Summary
parse_llm_jsoncounted braces without tracking string context, so any}inside a JSON string value — entirely plausible in an LLM's free-textoverall_impression/findings— truncated the object mid-string. The resulting decode error was swallowed byparse_llm_response, which fell back to treating the whole response as plain text: all structured findings silently discarded.Fixes #136
Changes
src/medcheck/llm/base.py:parse_llm_jsonnow usesjson.JSONDecoder().raw_decode(string/escape-aware), trying candidate{positions in order until one decodes to an object — this also recovers when a stray{appears in prose before the real JSON, which the old scanner could nottests/unit/test_llm/test_base.py: 3 new tests — braces inside string values, stray{before the object, markdown-fenced JSONTesting
uv run pytest) — 175 passed (all existing parser tests unchanged and green)uv run pytest --cov-fail-under=85) — 87.23%uv run ruff check .)uv run mypy src/medcheck --strict)Additional Notes
Error behavior is unchanged for genuinely JSON-free responses:
parse_llm_jsonstill raisesValueErrorandparse_llm_responsestill falls back to raw text.🤖 Generated with Claude Code
https://claude.ai/code/session_01W4svt5QTs4WUMSy4HwiVt9
Generated by Claude Code