FEAT expand TargetCapabilities

[hannahwestra25]

Description

This PR builds on https://github.com/Azure/PyRIT/pull/1433 to expand the TargetCapabilities class and consolidate the logic in Targets to use the TargetCapabilities class rather than misc variables. This is the first of at least 2 more PRs which will allow users to query target capabilities and add validation to attacks, converters, scorers, etc which have requirements for targets.

TargetCapabilities dataclass — expanded fields:

• supports_multi_turn (existing)
• supports_multi_message_pieces — rejects messages with >1 piece when False
• supports_json_response — whether JSON response format is supported
• input_modalities — allowed input data types (text, image_path, audio_path, …)
• output_modalities — produced output data types

Added assert_satisfies() to validate one TargetCapabilities against another. This will be useful later when validating whether a target satisfies the requirements of an attack / scorer / converter

---

PromptTarget._validate_request() — converted from abstract to a concrete base
implementation that auto-enforces capabilities:

• Rejects multi-piece messages when supports_multi_message_pieces=False
• Rejects unsupported converted_value_data_type against input_modalities
• Rejects follow-up turns when supports_multi_turn=False

All ad-hoc inline validation in individual targets was deleted.
Renamed constructor param capabilities → custom_capabilities.
Added is_json_response_supported() delegating to capabilities.supports_json_response.

---

Per-target _DEFAULT_CAPABILITIES declarations added/updated:

Target	Notable capabilities
OpenAIChatTarget	multi-turn, JSON, text/image/audio in, text/audio out
OpenAICompletionTarget	supports_multi_message_pieces=False
OpenAITTSTarget	no multi-turn, single-piece, audio out
OpenAIImageTarget	no multi-turn, text/image in, image out
OpenAIVideoTarget	no multi-turn, text/image in, video out
RealtimeTarget	multi-turn, single-piece, text/audio in+out
PlaywrightTarget	multi-turn, text/image in
PlaywrightCopilotTarget	multi-turn, text/image in+out
WebSocketCopilotTarget	multi-turn, text/image in
PromptShieldTarget	single-piece
HuggingFaceChatTarget	multi-turn, single-piece
CrucibleTarget	single-piece

---

Tests and Documentation

• New test_target_capabilities.py covering modality declarations, assert_satisfies, and per-target defaults
• test_supports_multi_turn.py extended with constructor override tests
• All pytest.raises error message patterns updated to the new unified format
• Mock memory calls updated: get_conversation → get_message_pieces
• patch_central_database fixture added to HTTP target tests that were missing it

[Copilot]

Pull request overview

Expands TargetCapabilities and consolidates target-side request validation to use a unified, capability-driven PromptTarget._validate_request, enabling future capability queries/validation across attacks, scorers, and converters.

Changes:

• Expanded TargetCapabilities (modalities + JSON/schema/editable-history flags) and added capability comparison helper.
• Implemented capability-based base request validation in PromptTarget, removing ad-hoc per-target validation in many targets.
• Updated/added unit tests and docs to reflect capability-driven behavior and new error messages.

Reviewed changes

Copilot reviewed 56 out of 59 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
tests/unit/target/test_video_target.py	Updates video target tests for new capability validation and memory API changes
tests/unit/target/test_tts_target.py	Updates TTS tests to match unified validation/error messages and memory API changes
tests/unit/target/test_target_capabilities.py	Adds coverage for modalities, known profiles, and default capability resolution
tests/unit/target/test_supports_multi_turn.py	Shifts assertions to target.capabilities.supports_multi_turn and constructor override rename
tests/unit/target/test_realtime_target.py	Updates validation expectation to new unified error message
tests/unit/target/test_prompt_target_azure_blob_storage.py	Updates expected validation message
tests/unit/target/test_playwright_target.py	Updates expected validation message
tests/unit/target/test_playwright_copilot_target.py	Updates expected validation message
tests/unit/target/test_openai_target_auth.py	Removes is_json_response_supported override from mock target
tests/unit/target/test_openai_response_target.py	Updates validation assertions and removes JSON support tests
tests/unit/target/test_openai_chat_target.py	Removes JSON support tests; adjusts tests to use custom capabilities
tests/unit/target/test_image_target.py	Updates tests for capability validation and memory API changes
tests/unit/target/test_huggingface_chat_target.py	Removes JSON support test for HF chat target
tests/unit/target/test_http_target.py	Ensures mocked message pieces include converted_value_data_type; adds missing fixture use
tests/unit/target/test_gandalf_target.py	Updates expected validation message
tests/unit/target/test_crucible_target.py	Updates expected validation message
tests/unit/target/test_azure_openai_completion_target.py	Updates expected validation message
tests/unit/target/test_azure_ml_chat_target.py	Removes JSON support test
tests/unit/registry/test_target_registry.py	Removes JSON support method from test target stub
tests/unit/mocks.py	Removes JSON support method from mock target
tests/unit/executor/attack/multi_turn/test_supports_multi_turn_attacks.py	Adjusts mocks to use target.capabilities.supports_multi_turn
tests/integration/mocks.py	Removes JSON support method from integration mock
pyrit/prompt_target/websocket_copilot_target.py	Replaces ad-hoc validation with capability defaults + super()._validate_request
pyrit/prompt_target/text_target.py	Adds custom_capabilities plumbing to constructor
pyrit/prompt_target/prompt_shield_target.py	Plumbs custom_capabilities; removes ad-hoc request validation
pyrit/prompt_target/playwright_target.py	Adds modality defaults + custom_capabilities; removes ad-hoc request validation
pyrit/prompt_target/playwright_copilot_target.py	Adds modality defaults + custom_capabilities; removes ad-hoc request validation
pyrit/prompt_target/openai/openai_video_target.py	Adds modality defaults + custom_capabilities; calls base request validation
pyrit/prompt_target/openai/openai_tts_target.py	Adds output modality defaults + custom_capabilities; removes ad-hoc request validation
pyrit/prompt_target/openai/openai_target.py	Adds default capability baseline and implements JSON support check via capabilities
pyrit/prompt_target/openai/openai_response_target.py	Declares detailed default capabilities; plumbs custom_capabilities; removes ad-hoc validation
pyrit/prompt_target/openai/openai_realtime_target.py	Declares realtime modality defaults; plumbs custom_capabilities; removes ad-hoc validation
pyrit/prompt_target/openai/openai_image_target.py	Declares image target default capabilities; plumbs custom_capabilities; calls base validation
pyrit/prompt_target/openai/openai_completion_target.py	Plumbs custom_capabilities; removes ad-hoc request validation
pyrit/prompt_target/openai/openai_chat_target.py	Declares chat target default capability flags; merges deprecated is_json_supported into capabilities
pyrit/prompt_target/hugging_face/hugging_face_endpoint_target.py	Introduces default capabilities; trims validation logic
pyrit/prompt_target/hugging_face/hugging_face_chat_target.py	Declares default capabilities; plumbs custom_capabilities; ties JSON support to capabilities
pyrit/prompt_target/http_target/httpx_api_target.py	Adds default capabilities + custom_capabilities plumbing
pyrit/prompt_target/http_target/http_target.py	Adds custom_capabilities plumbing; removes ad-hoc request validation
pyrit/prompt_target/gandalf_target.py	Adds custom_capabilities plumbing; removes ad-hoc request validation
pyrit/prompt_target/crucible_target.py	Adds custom_capabilities plumbing; removes ad-hoc request validation
pyrit/prompt_target/common/target_capabilities.py	Expands capabilities model and adds known profiles + assertion helper
pyrit/prompt_target/common/prompt_target.py	Implements capability-based base request validation and default-capability resolution
pyrit/prompt_target/common/prompt_chat_target.py	Switches JSON-format gating to capabilities.supports_json_output
pyrit/prompt_target/azure_ml_chat_target.py	Adds default capabilities + custom_capabilities plumbing; removes JSON support method
pyrit/prompt_target/azure_blob_storage_target.py	Declares blob target modalities and custom_capabilities; removes ad-hoc request validation
pyrit/executor/attack/multi_turn/tree_of_attacks.py	Updates multi-turn branching logic to use capabilities
pyrit/executor/attack/multi_turn/multi_turn_attack_strategy.py	Updates rotation logic to use capabilities
pyrit/executor/attack/multi_turn/multi_prompt_sending.py	Updates guard to use capabilities
pyrit/executor/attack/multi_turn/crescendo.py	Updates guard to use capabilities
pyrit/executor/attack/multi_turn/chunked_request.py	Updates guard to use capabilities
doc/code/targets/3_openai_image_target.py	Updates docs to show capability overrides in examples
doc/code/targets/1_openai_chat_target.py	Updates docs to show capability overrides in examples
doc/code/executor/attack/2_red_teaming_attack.py	Updates docs to show capability overrides in scorer target example
doc/code/converters/3_image_converters.py	Updates docs to show capability overrides for multimodal usage
doc/code/converters/3_image_converters.ipynb	Updates notebook example to include capability overrides

---

You can also share your feedback on Copilot code review. Take the survey.

[jsong468]

Description

This PR builds on #1433 to expand the TargetCapabilities class and consolidate the logic in Targets to use the TargetCapabilities class rather than misc variables. This is the first of at least 2 more PRs which will allow users to query target capabilities and add validation to attacks, converters, scorers, etc which have requirements for targets.

TargetCapabilities dataclass — expanded fields:

• supports_multi_turn (existing)
• supports_multi_message_pieces — rejects messages with >1 piece when False
• supports_json_response — whether JSON response format is supported
• input_modalities — allowed input data types (text, image_path, audio_path, …)
• output_modalities — produced output data types

Added assert_satisfies() to validate one TargetCapabilities against another. This will be useful later when validating whether a target satisfies the requirements of an attack / scorer / converter

PromptTarget._validate_request() — converted from abstract to a concrete base implementation that auto-enforces capabilities:

• Rejects multi-piece messages when supports_multi_message_pieces=False
• Rejects unsupported converted_value_data_type against input_modalities
• Rejects follow-up turns when supports_multi_turn=False

All ad-hoc inline validation in individual targets was deleted. Renamed constructor param capabilities → custom_capabilities. Added is_json_response_supported() delegating to capabilities.supports_json_response.

Per-target _DEFAULT_CAPABILITIES declarations added/updated:

Target Notable capabilities
OpenAIChatTarget multi-turn, JSON, text/image/audio in, text/audio out
OpenAICompletionTarget supports_multi_message_pieces=False
OpenAITTSTarget no multi-turn, single-piece, audio out
OpenAIImageTarget no multi-turn, text/image in, image out
OpenAIVideoTarget no multi-turn, text/image in, video out
RealtimeTarget multi-turn, single-piece, text/audio in+out
PlaywrightTarget multi-turn, text/image in
PlaywrightCopilotTarget multi-turn, text/image in+out
WebSocketCopilotTarget multi-turn, text/image in
PromptShieldTarget single-piece
HuggingFaceChatTarget multi-turn, single-piece
CrucibleTarget single-piece

Tests and Documentation

• New test_target_capabilities.py covering modality declarations, assert_satisfies, and per-target defaults
• test_supports_multi_turn.py extended with constructor override tests
• All pytest.raises error message patterns updated to the new unified format
• Mock memory calls updated: get_conversation → get_message_pieces
• patch_central_database fixture added to HTTP target tests that were missing it

side note for the future PRs, do we want to be able to have necessary target capabilities attached to each scenario?

[jsong468]

small comments/questions, but looks good!