Skip to content

feat(cli): expose structured violation data in policy develop eval output#2969

Merged
migmartri merged 1 commit intochainloop-dev:mainfrom
migmartri:feat/policy-develop-eval-structured-output
Mar 31, 2026
Merged

feat(cli): expose structured violation data in policy develop eval output#2969
migmartri merged 1 commit intochainloop-dev:mainfrom
migmartri:feat/policy-develop-eval-structured-output

Conversation

@migmartri
Copy link
Copy Markdown
Member

@migmartri migmartri commented Mar 30, 2026

Summary

  • Automatically populate structured_violations in policy develop eval output when the policy declares a finding_type and violations contain structured finding data
  • Uses protojson to serialize proto violations directly, no custom types needed
  • The violations field (string messages) remains always populated for backward compatibility

Closes #2968

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 5 files

@migmartri migmartri force-pushed the feat/policy-develop-eval-structured-output branch 2 times, most recently from b135567 to 7046780 Compare March 30, 2026 22:29
…tput

Automatically populate structured_violations in the eval output when
the policy returns structured finding data. Uses protojson to serialize
the proto violations directly — no custom types needed. The violations
field (string messages) remains populated for backward compatibility.

Closes chainloop-dev#2968

Signed-off-by: Miguel Martinez <migmartri@gmail.com>
Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev>
@migmartri migmartri force-pushed the feat/policy-develop-eval-structured-output branch from 7046780 to 1135617 Compare March 30, 2026 22:33
@migmartri migmartri changed the title feat(cli): add --structured flag to policy develop eval feat(cli): expose structured violation data in policy develop eval output Mar 30, 2026
@migmartri migmartri merged commit b417baf into chainloop-dev:main Mar 31, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

policy develop eval: expose structured violation data in output

2 participants