FEAT: Propagating Scenario/Attack Errors#1720
Merged
rlundeen2 merged 9 commits intoMay 13, 2026
Merged
Conversation
- Add RetryEvent dataclass for capturing Tenacity retry attempts - Add RetryCollector (contextvar-based) to accumulate retry events per-attack - Hook log_exception() to record retries to active collector - Add error_message, error_type, error_traceback, retry_events, total_retries fields to AttackResult and AttackResultEntry - Add error_attack_result_ids pointer to ScenarioResult/ScenarioResultEntry - Create ON_ERROR handler in attack strategy to persist failed AttackResults with error details, even when the attack crashes before returning - Wire RetryCollector lifecycle in pre/post execute event handlers - Link failed attack results to scenario result on incomplete objectives - Add RetryEventResponse, error/retry fields to REST API models (AttackSummary, ScenarioRunSummary, AtomicAttackResults) - Update service layer to populate error/retry info from persisted data - Add Alembic migration for new database columns - Add update_scenario_error_attacks() to MemoryInterface All new fields are nullable/optional for backward compatibility. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- RetryEvent: serialization, round-trip, defaults, missing fields - RetryCollector: contextvar lifecycle, record(), asyncio task isolation - AttackResult: error field defaults, storage, DB round-trip, truncation - ScenarioResult: error_attack_result_ids defaults and storage Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
hannahwestra25
approved these changes
May 12, 2026
- Replace placeholder Alembic revision ID (a1b2c3d4e5f6) with proper generated hash (4f9db4b0a77f)
- Swap persist-before-set order in _on_error_async so context only gets error_attack_result_id after DB write succeeds; clear stale ID at handler entry
- Rewrite update_scenario_error_attacks to do read-modify-write in a single DB session
- Extract retry_events_to_response() shared helper in attack_mappers, replacing duplicate code in scenario_run_service
- Remove inline import from loop body in scenario_run_service
- Replace __import__("uuid") hack with proper top-level import
- Rename _on_error to _on_error_async per style guide
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds the ability to structurally keep track of errors and retries in
ScenarioResultandAttackResult. This will be important to surface these from the backend to front end, and useful for analytics (e.g. we can view retries to targets over time)