Skip to content

chore(redteam): added strategies round-trips#269

Merged
poshinchen merged 1 commit into
strands-agents:mainfrom
poshinchen:chore/strategies-round-trips
Jun 15, 2026
Merged

chore(redteam): added strategies round-trips#269
poshinchen merged 1 commit into
strands-agents:mainfrom
poshinchen:chore/strategies-round-trips

Conversation

@poshinchen

@poshinchen poshinchen commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Description

  1. Adds to_dict / from_dict round-trip support for the four built-in attack strategies that were previously only persisted with strategy_type + label: BadLikertJudgeStrategy, GoatStrategy, PairStrategy, and SequentialBreakStrategy.
  • Each now serializes its fields (max_turns, success_threshold, strategy-specific params) and its model so a saved experiment reloads with the same configuration.
  1. Light cleanup along the way: updated the function and renamed _serialize_model to _put_model_field. Behavior is unchanged.

Notes

AttackStrategy.from_dict already centralized the registry lookup; the four classes are added there alongside CrescendoStrategy / PromptStrategy so reload resolves them without custom_strategies=.

Related Issues

N/A

Documentation PR

N/A

Type of Change

New feature

Testing

  • Added per-strategy to_dict assertions for BLJ / Goat / Pair / SequentialBreak.
  • Added test_round_trip_preserves_all_builtin_strategies: builds an experiment with all six built-ins, writes via to_file, reloads via from_file, and asserts type and label preservation.
  • Existing test_from_dict_round_trip_runs_after_setting_agent continues to pass.
  • tests/strands_evals/experimental/redteam/ — 312 passed.
  • I ran hatch run prepare

Checklist

  • I have read the CONTRIBUTING document
  • I have reviewed and understand every line of code in this PR, including any generated by AI tools, and I can explain why it works
  • My change is focused and reasonably small; I have split unrelated work into separate PRs
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@github-actions github-actions Bot added chore Maintenance tasks, dependency updates, CI changes, refactoring with no user-facing impact area-redteam Red teaming: adversarial generation, attack strategies, attack success evaluation strands-running labels Jun 15, 2026
Comment thread tests/strands_evals/experimental/redteam/test_experiment.py
@github-actions

Copy link
Copy Markdown

Assessment: Approve

Clean, focused change. I verified that every new to_dict key maps exactly to its __init__ parameter name (refine_rounds, success_threshold, max_turns, variants, store_reasoning, model), so the from_dicttarget_cls(**payload) reconstruction is correct for all four strategies. The _put_model_field helper is a sound de-dup of the repeated idiom with behavior unchanged, and the registry additions follow the existing Crescendo/Prompt pattern. Tests pass locally.

Review notes
  • Correctness: to_dict/from_dict symmetry holds for BLJ, Goat, Pair, and SequentialBreak — confirmed against each constructor signature.
  • Refactor: _put_model_field correctly preserves the "drop field on None → fall back to experiment model" semantic.
  • Testing: One non-blocking gap — the all-built-ins round-trip test asserts only type/label, not config-field fidelity through reconstruction. Left an inline suggestion to extend the symmetric from_dict(to_dict()).to_dict() == to_dict() check to all six strategies.
  • API: No public-API/bar-raising concern; experimental redteam internals following an established serialization pattern.

Nicely scoped PR with thorough docstrings explaining the design intent.

@poshinchen poshinchen merged commit fdacbff into strands-agents:main Jun 15, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-redteam Red teaming: adversarial generation, attack strategies, attack success evaluation chore Maintenance tasks, dependency updates, CI changes, refactoring with no user-facing impact

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants