Skip to content

MAINT Breaking: Rename Scenario Strategies to Techniques#2153

Merged
rlundeen2 merged 3 commits into
microsoft:mainfrom
rlundeen2:rlundeen2-rename-scenario-strategy-to-technique
Jul 9, 2026
Merged

MAINT Breaking: Rename Scenario Strategies to Techniques#2153
rlundeen2 merged 3 commits into
microsoft:mainfrom
rlundeen2:rlundeen2-rename-scenario-strategy-to-technique

Conversation

@rlundeen2

@rlundeen2 rlundeen2 commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Renames the scenario-layer strategytechnique vocabulary for the 1.0 release. The CLI already selects techniques (the --strategies flag chose techniques), and the codebase already uses AttackTechnique / AttackTechniqueRegistryScenarioStrategy was the lone holdout. This aligns the naming end-to-end.

This is a hard break with no backward-compatibility aliases (intended for 1.0).

Key changes

  • ScenarioStrategyScenarioTechnique (base enum + module rename)
  • Scenario technique enums: Foundry/Cyber/Leakage/RapidResponse/Benchmark/Adversarial/TextAdaptive/Psychosocial*Strategy*Technique
  • Params/registry: scenario_strategies/strategy_convertersscenario_techniques/technique_converters; _resolve_strategies_and_converters_resolve_techniques_and_converters
  • Models/API (JSON fields): strategies, strategies_used, default_strategy, aggregate_strategies, all_strategiestechniques, techniques_used, default_technique, aggregate_techniques, all_techniques; ScenarioResult.get_strategies_used()get_techniques_used()
  • CLI: --strategies/-s--techniques/-t
  • Docs (scanner + scenarios + resiliency notebooks) and scenarios.instructions.md updated

Explicitly NOT touched (deferred)

Executor AttackStrategy/attack_strategy, WorkflowStrategy, PromptGeneratorStrategy, converter *SelectionStrategy, and StrategyResult are unrelated concepts and are preserved.

Verification

  • In-scope unit suite green; executor/converter safety suite green
  • ruff lint + format clean; ty check pyrit clean
  • CLI --help shows --techniques, -t; repo-wide sweep confirms no dangling old scenario symbols

End-to-end / live testing (nothing broken)

Ran the scanner path against a live OpenAI target and exercised the renamed API surface:

  • doc/scanner/1_pyrit_scan.ipynb — executed end-to-end (start-server → --list-scenarios/--list-initializers → real foundry.red_team_agent --techniques base64 scan → custom ScenarioTechnique subclass → stop-server). 0 cell errors; output shows the renamed vocabulary (scenario_techniques, technique_converters, techniques: N/N progress).
  • Direct CLI scan foundry.red_team_agent --target openai_chat --initializers target --techniques base64 — completed cleanly (Total Techniques: 2, per-group baseline / base64 breakdown).
  • doc/scanner/foundry.ipynb — executed the programmatic scenario + output_scenario_async rendering path. 0 cell errors (Scenario: RedTeamAgent, Atomic attacks: 2, Total Techniques: 2).
  • Renamed-enum smoke check — with the registry initialized, resolved every renamed technique enum used across the scanner notebooks and confirmed get_all_techniques() / get_aggregate_techniques() work: FoundryTechnique, RapidResponseTechnique, PsychosocialTechnique, CyberTechnique, JailbreakTechnique, LeakageTechnique, ScamTechnique, EncodingTechnique, DoctorTechnique, WebInjectionTechnique.

Also fixed a stale scanner-doc flag surfaced during testing: --scenario-techniques--techniques (the doc referenced a non-existent flag).

rlundeen2 and others added 2 commits July 9, 2026 12:03
Renames the scenario-layer 'strategy' vocabulary to 'technique' for 1.0, aligning with the pre-existing AttackTechnique/AttackTechniqueRegistry naming and the CLI (--techniques selects techniques). This is a hard break with no backward-compat aliases.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The scanner walkthrough referenced a non-existent --scenario-techniques
flag (carried forward from the earlier non-existent --scenario-strategies
example). The actual CLI flag is --techniques / -t. Fixes the markdown
examples in the pyrit_scan scanner guide.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Comment thread doc/scanner/1_pyrit_scan.ipynb Outdated
Co-authored-by: jsong468 <songjustin@microsoft.com>
@rlundeen2 rlundeen2 enabled auto-merge July 9, 2026 22:47
@rlundeen2 rlundeen2 added this pull request to the merge queue Jul 9, 2026
Merged via the queue into microsoft:main with commit e151cce Jul 9, 2026
53 checks passed
@rlundeen2 rlundeen2 deleted the rlundeen2-rename-scenario-strategy-to-technique branch July 9, 2026 23:19
varunj-msft pushed a commit to varunj-msft/PyRIT that referenced this pull request Jul 10, 2026
… collisions

Re-migrated onto current main (post technique rename microsoft#2153 / technique
organization microsoft#2155). garak.encoding was already seam-migrated on main to
EncodingTechnique(ScenarioTechnique); this layers the remaining #8380
standardization on top.

Add an EncodingTechnique.DEFAULT aggregate (curated 8-scheme spread across
base-N, cipher, and symbolic families) with a get_aggregate_tags() override,
make it the default technique, and bump VERSION to 2. Follows the hand-written
enum DEFAULT pattern used by scam/web_injection (not the factory
default_technique_names path, which applies only to the core catalog).

Fix atomic-attack name collisions by naming each converter variant
f"{variant_slug}_{config_suffix}" and grouping variants under
display_group=encoding_name so per-encoding results aggregate. Trim
near-duplicate base64 variants (standard_b64encode is byte-identical to the
default; b2a_base64 only appends a newline).

Keep the default per-dataset cap at main's max_dataset_size=3 (no timing change).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants