Skip to content

fix(llm): accept list payloads for single-list schemas - #363

Merged
guangyu-reflexio merged 1 commit into
mainfrom
fix/structured-output-list-wrapper
Jul 18, 2026
Merged

fix(llm): accept list payloads for single-list schemas#363
guangyu-reflexio merged 1 commit into
mainfrom
fix/structured-output-list-wrapper

Conversation

@guangyu-reflexio

@guangyu-reflexio guangyu-reflexio commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Accept top-level JSON arrays for structured-output schemas that are only a single list wrapper field
  • Keep multi-field schemas strict so malformed outputs still surface as parse failures
  • Add unit coverage for both accepted and rejected top-level list payloads

Finding

test-backend-pipeline surfaced a playbook extraction parse failure from MiniMax where the model returned a semantically valid list of playbook entries while the schema expected the canonical object wrapper ({"playbooks": [...]}). Layer: codebase under test. Root cause: the structured-output parser was too brittle for single-list wrapper schemas, causing one extraction batch to fail and leaving too few user playbooks for aggregation.

Fix

Normalize only the safe case: when the target Pydantic response model has exactly one list[...] field and the parsed JSON root is a list, validate {"<field>": parsed_list}. All other schemas keep the existing strict validation behavior.

Verification

  • Review Code PR Readiness: clean
  • uv run --no-sync ruff check open_source/reflexio/reflexio/server/llm/_litellm_structured_output.py open_source/reflexio/tests/server/llm/test_litellm_client_unit.py
  • uv run --no-sync pytest open_source/reflexio/tests/server/llm/test_litellm_client_unit.py -q -o 'addopts=' (220 passed)

Summary by CodeRabbit

  • Bug Fixes
    • Improved structured-output parsing when responses are returned as top-level JSON arrays.
    • Automatically wraps array payloads to match schemas shaped as a single list field.
    • Maintains correct failures when the schema expects additional fields beyond a single list wrapper.
  • Tests
    • Added/expanded unit coverage for single-list, bare-list, and multi-field list schema handling for structured parsing.

@coderabbitai

coderabbitai Bot commented Jul 18, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 7c666141-7e04-4429-a016-9584198de55d

📥 Commits

Reviewing files that changed from the base of the PR and between 2fd0bdc and 541db60.

📒 Files selected for processing (2)
  • reflexio/server/llm/_litellm_structured_output.py
  • tests/server/llm/test_litellm_client_unit.py

📝 Walkthrough

Walkthrough

Structured-output parsing now wraps top-level JSON arrays for Pydantic models containing exactly one list field. Primary, sanitized, and repaired parsing paths share this validation behavior, with tests covering accepted single-list and rejected multi-field schemas.

Changes

Structured output normalization

Layer / File(s) Summary
Single-list payload normalization
reflexio/server/llm/_litellm_structured_output.py
Adds helpers to detect single-list Pydantic models and wrap top-level arrays under the corresponding field name.
Parser integration and coverage
reflexio/server/llm/_litellm_structured_output.py, tests/server/llm/test_litellm_client_unit.py
Routes primary, sanitized, and repaired payloads through normalized validation and tests accepted single-list and rejected multi-field schemas.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant LLMParser
  participant _validate_structured_payload
  participant PydanticResponseModel
  LLMParser->>_validate_structured_payload: pass parsed JSON or repaired payload
  _validate_structured_payload->>_validate_structured_payload: detect single-list schema and wrap top-level array
  _validate_structured_payload->>PydanticResponseModel: call model_validate
  PydanticResponseModel-->>LLMParser: return validated response
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: accepting list payloads for single-list schemas.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/structured-output-list-wrapper

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@reflexio/server/llm/_litellm_structured_output.py`:
- Line 51: Update the list-detection condition in the structured-output schema
handling to recognize both a raw unsubscripted list annotation and parameterized
lists via get_origin(field.annotation). Preserve the existing wrapping behavior
for list[str]-style annotations while ensuring fields annotated simply as list
are also treated as top-level arrays.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: b7c6c9fc-9d37-4530-906d-728637f5325c

📥 Commits

Reviewing files that changed from the base of the PR and between 9b3ef6a and 2fd0bdc.

📒 Files selected for processing (2)
  • reflexio/server/llm/_litellm_structured_output.py
  • tests/server/llm/test_litellm_client_unit.py

Comment thread reflexio/server/llm/_litellm_structured_output.py Outdated
@guangyu-reflexio
guangyu-reflexio force-pushed the fix/structured-output-list-wrapper branch from 2fd0bdc to 541db60 Compare July 18, 2026 18:24
@guangyu-reflexio

Copy link
Copy Markdown
Contributor Author

Fixed in 541db60: _single_list_field_name now recognizes both field.annotation is list and get_origin(field.annotation) is list, with BareListResponse regression coverage. Verification: ruff check on touched files and test_litellm_client_unit.py (221 passed).

@guangyu-reflexio
guangyu-reflexio merged commit 06012b1 into main Jul 18, 2026
1 check passed
@guangyu-reflexio
guangyu-reflexio deleted the fix/structured-output-list-wrapper branch July 18, 2026 18:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant