MAINT remove PromptChatTarget#1532
Conversation
…emove_prompt_chat_target
…om/hannahwestra25/PyRIT into hawestra/remove_prompt_chat_target
| new_item="AttackParameters.prepended_conversation", | ||
| removed_in="0.14.0", | ||
| ) | ||
| target._set_target_system_prompt( |
There was a problem hiding this comment.
The one thing I want to check is that this flow still works in the few places you can have a system prompt but are not editable_history, like realtime_target.
Because prepended_conversation is essentially how we edit the conversation. But then we also need a way to set the system prompt for that target specifically.
At the same time, we've had these really nasty bugs where attacks will call set_system_prompt on a target and it messes everything up. So I LIKE that we're setting it via conversation history. But we just need to figure out these special cases.
| ) | ||
|
|
||
| self._adversarial_chat.set_system_prompt( | ||
| self._adversarial_chat._set_target_system_prompt( |
There was a problem hiding this comment.
this may already be the fix for what I mentioned earlier about realtime. But we want to verify.
| self._adversarial_chat = attack_adversarial_config.target | ||
| if not isinstance(self._adversarial_chat, PromptChatTarget): | ||
| raise ValueError("The adversarial target must be a PromptChatTarget for TAP attack.") | ||
| self._adversarial_chat.capabilities.validate( |
There was a problem hiding this comment.
validate centrally for adversarial_chat
There was a problem hiding this comment.
We could probably do this in the adversarial_chat_config object? Or a base attack class
|
|
||
| # Adversarial chat target for the attack | ||
| target: PromptChatTarget | ||
| target: PromptTarget |
There was a problem hiding this comment.
yeah, this needs to be validated to be multi-turn and editable; then we could potentially take out the validation from TAP
| ) | ||
|
|
||
| # Validate that the adversarial chat target supports required capabilities | ||
| if adversarial_chat: |
There was a problem hiding this comment.
nit: validate centrally for adversarial_chat
|
|
||
| # Configure the target with the system prompt | ||
| self._objective_target.set_system_prompt( | ||
| self._objective_target._set_target_system_prompt( |
There was a problem hiding this comment.
likely want to validate _objective_target here
| "Use RedTeamingAttack or TreeOfAttacksWithPruning instead." | ||
| ) | ||
| self._objective_target.capabilities.validate( | ||
| required={"supports_multi_turn"}, |
There was a problem hiding this comment.
also needs editable_history
| @@ -1293,7 +1293,7 @@ def __init__( | |||
|
|
|||
| Raises: | |||
There was a problem hiding this comment.
objective_target also needs to validate multi_turn and editable history
| objective_target: PromptChatTarget = REQUIRED_VALUE, # type: ignore[assignment] | ||
| objective_target: PromptTarget = REQUIRED_VALUE, # type: ignore[assignment] | ||
| attack_adversarial_config: AttackAdversarialConfig, | ||
| attack_converter_config: Optional[AttackConverterConfig] = None, |
There was a problem hiding this comment.
need to validate objective_target supports multi-turn and editable history
| """ | ||
| Args: | ||
| objective_target (PromptChatTarget): The target system to attack. | ||
| objective_target (PromptTarget): The target system to attack. |
There was a problem hiding this comment.
need to validate objective_target supports multi-turn and editable history
| @@ -56,7 +56,7 @@ async def convert_async(self, *, prompt: str, input_type: PromptDataType = "text | |||
|
|
|||
| conversation_id = str(uuid.uuid4()) | |||
There was a problem hiding this comment.
need to validate objective_target supports setting the system prompt
| conversation_id = str(uuid.uuid4()) | ||
|
|
||
| self.converter_target.set_system_prompt( | ||
| self.converter_target._set_target_system_prompt( |
There was a problem hiding this comment.
if we're calling this outside of target, it should be a public method
…orts, add seed model structural tests - Update PromptChatTarget to PromptTarget per PR microsoft#1532 deprecation - Move ScenarioStrategy import to top-level in test_foundry_contract.py - Add rationale for explicit inheritance checks in test_import_smoke.py - Expand seed model tests with structural validation (value, data_type, harm_categories, role, metadata, SeedGroup composition) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Description
PromptChatTarget has historically was an interface for a target that supports multi-turn conversations with editable history. This PR decouples those concerns by moving capability signaling to TargetCapabilities making PromptChatTarget an empty deprecated subclass.
Also, deprecate set_system_prompt and directing users to use prepended messages. Targets still set system prompts via an internal method _set_target_system_prompt
Changes
All attack strategies, scenarios, scorers, and converters that previously required a PromptChatTarget now accept any PromptTarget Compatibility is enforced at construction time through TargetCapabilities validate function with supports_multi_turn & supports_editable_history instead of a type check.
Tests and Documentation
[in progress] running all target notebooks & integration tests