Skip to content

Conversation

@bashirpartovi
Copy link
Contributor

@bashirpartovi bashirpartovi commented Jun 26, 2025

Description

  • Refactored TreeOfAttacksWithPruningOrchestrator and TreeOfAttacksNode from pyrit/orchestrator module
  • Added auxiliary scoring capability to the attack
  • Added response converters
  • Improved the performance by batching and parallelization among the nodes
  • Ability to pass in the objective scorer
  • Fixed the issue with the previous implementation where only boolean scores were accepted from the objective scorer, now it works with both float scale and boolean scales

Tests and Documentation

  • Tested through unit tests as well as manual testing

@bashirpartovi bashirpartovi requested a review from Copilot June 26, 2025 15:17
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request refactors the TreeOfAttacks components with the new AttackStrategy, adds auxiliary scoring and response converter capabilities, and improves performance via batching and parallelization. Key changes include updates to the TreeOfAttacksNode and TAPAttack classes, additions to the scoring and prompt loading logic, and adjustments to the associated tests.

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/unit/attacks/test_tree_of_attacks.py Expanded test coverage for various TAP attack behaviors, including node pruning, branching, and error handling.
pyrit/attacks/multi_turn/tree_of_attacks_node.py Refactored node implementation for improved modularity, error handling, and integration of prompt and score conversions.
pyrit/attacks/multi_turn/tree_of_attacks.py Implemented the TAP attack strategy with tree exploration, parallel prompt execution, and result aggregation.
pyrit/attacks/base/attack_context.py Updated context classes to include prepended conversation for multi- and single-turn attacks.
pyrit/attacks/init.py Exposed the new TAP attack related classes through the module interface.

Copy link
Contributor

@romanlutz romanlutz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! A few smaller comments but I love how this turned out!

@bashirpartovi bashirpartovi merged commit f242758 into Azure:main Jul 2, 2025
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants