MAINT Breaking: Convert ScenarioResult to Pydantic#1908
Merged
ValbuenaVC merged 6 commits intoJun 3, 2026
Merged
Conversation
Relocate get_scorer_evaluation_metrics to pyrit.score.scenario_metrics.get_scenario_metrics, make display_group_map a public field, and keep to_dict/from_dict as deprecated shims preserving the legacy wire shape. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The helper had no production callers (the original ScenarioResult method also had none), so relocating it in Phase 7 just moved dead code. Remove the module, its test, and the pyrit.score export. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…nario-result-pydantic # Conflicts: # pyrit/models/scenario_result.py # tests/unit/models/test_import_boundary.py
romanlutz
reviewed
Jun 3, 2026
romanlutz
reviewed
Jun 3, 2026
romanlutz
approved these changes
Jun 3, 2026
- Convert ScenarioRunState from a Literal alias to a str-backed Enum (matches house style; backward-compatible at all string call sites). - Move ScenarioResult.normalize_scenario_name to the end (static methods last). - Guard ScoreEntry.__init__ against a None scorer_class_identifier and drop a now-unused type: ignore, fixing ty errors surfaced after merging origin/main. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…efault - Move inline imports in test_scenario_result.py to the top of the file - Simplify pyrit_version from default_factory to a plain default in ScenarioIdentifier and ComponentIdentifier (sibling identifiers) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
romanlutz
added a commit
to romanlutz/PyRIT
that referenced
this pull request
Jun 4, 2026
Merge 26 commits from main, including: - MAINT Breaking: Convert ScenarioResult to Pydantic (microsoft#1908) - MAINT: Migrating Seed classes to Pydantic (microsoft#1898) - MAINT: Migrating AttackResult to Pydantic (microsoft#1899) - MAINT: Bump ty-pre-commit v0.0.32 -> 0.0.43 (microsoft#1919) - FEAT: Realtime streaming session support and server-side barge-in attack (microsoft#1766) - FEAT text adaptive scenario (microsoft#1760) - FIX: Integration Test Fixes (microsoft#1907) - DOC: Scoring Docs Refactor (microsoft#1892) - Various dependency bumps Conflicts (15 files) resolved by taking main's version + re-running ruff --fix to re-apply PEP 604 typing modernization on the incoming code (177 violations auto-fixed). All resolved files re-staged. Local verification: - ruff check: All checks passed - ruff format: clean - pytest tests/unit -n 8: 9550 passed, 6 skipped Known issue (pre-existing on main, not caused by this merge): - ty 0.0.43 enabled missing-override-decorator rule, which flags hundreds of pre-existing methods across the codebase. Main's own CI is currently failing on this. Our PR will inherit the same failure since touched files come into pre-commit scope. Fixing this rule globally is a separate, large mechanical change orthogonal to typing modernization. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This converts ScenarioResult to a Pydantic BaseModel. It is phase 7 of the pyrit.models refactor (and the last model move remaining!):
https://gist.github.com/rlundeen2/3e8daa8e12a11b4b6e52587b3c9b1dca
This one is breaking. I want to remove
get_scorer_evaluation_metrics. There were no callers in our code, and it relies on scorer, so it's something we need to get rid of to make our dependency footprint small. Check out phase 9 where we'd want to evict this from pyrit.models to understand some of the annoying things about deprecation. But not a huge deal if we do want to keep as is.