Skip to content

fix(evaluation): skip invocations without user events to prevent ValidationError#5929

Open
Oxygen56 wants to merge 3 commits into
google:mainfrom
Oxygen56:fix/skip-invocations-without-user-events-3760
Open

fix(evaluation): skip invocations without user events to prevent ValidationError#5929
Oxygen56 wants to merge 3 commits into
google:mainfrom
Oxygen56:fix/skip-invocations-without-user-events-3760

Conversation

@Oxygen56
Copy link
Copy Markdown

@Oxygen56 Oxygen56 commented Jun 1, 2026

Summary

Changes user_content initialization from Content(parts=[]) to None and skips invocations that have no user-authored events.

Previously, internal/system-driven invocations lacking a user event would produce semantically incorrect eval cases with empty content objects, causing a ValidationError.

Changes

  • Initialize user_content = None instead of Content(parts=[])
  • Add guard to skip invocations where no user event was found (continue to next invocation)
  • Prevents ValidationError from empty Content objects

Fixes #3760

…dationError

Change user_content initialization from Content(parts=[]) to None and
skip invocations that have no user-authored events. This prevents
ValidationError from semantically incorrect empty Content objects.

Fixes google#3760
@rohityan rohityan self-assigned this Jun 1, 2026
@rohityan rohityan added the eval [Component] This issue is related to evaluation label Jun 2, 2026
@adk-bot
Copy link
Copy Markdown
Collaborator

adk-bot commented Jun 5, 2026

I have successfully analyzed GitHub Pull Request #5929 and generated a detailed PR Analysis Report under findings artifact:

👉 analysis_results.md

Summary of Analysis Work

  1. Compliance Verified: Confirmed that the Contributor License Agreement (cla/google) check is verified and has successfully passed for PR fix(evaluation): skip invocations without user events to prevent ValidationError #5929.
  2. Context Retrieved: Fetched complete PR details & changes, and linked Issue #3760, verifying the underlying Pydantic ValidationError crash during evaluation session addition.
  3. Workspace Inspection: Inspected evaluation_generator.py to assess correctness and structural soundness of the fix.
  4. Outcome / Recommendation: Approve with Nits. The fix correctly skips empty invocations without crashing, but the contributor has not supplied any unit tests. I have proposed a concrete unit test to add to test_evaluation_generator.py in the report.

Next Steps & Decisions

  • Would you like to request the test addition directly on the PR, or should we checkout the branch and add the suggested unit test ourselves?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

eval [Component] This issue is related to evaluation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Evaluation Invocation.user_content ValidationError when session has invocations without user events

3 participants