Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Task/jailbreak adv sim #3455

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Task/jailbreak adv sim #3455

wants to merge 10 commits into from

Conversation

nagkumar91
Copy link
Member

@nagkumar91 nagkumar91 commented Jun 21, 2024

Description

Adding an additional way to utilize the jailbreak feature of the adversarial simulator.

Our customers who use the Adversarial Simulator with the jailbreak=True option, always try to run the simulator twice, to simulate with and without jailbreak.

This PR introduces a new JailbreakAdversarialSimulator which runs the simulator twice. Once with jailbreak=True and once with jailbreak=False to accomplish our customers' goals of running the simulator for an adversarial dataset.

All Promptflow Contribution checklist:

  • The pull request does not introduce [breaking changes].
  • CHANGELOG is updated for new features, bug fixes or other significant changes.
  • I have read the contribution guidelines.
  • Create an issue and link to the pull request to get dedicated review from promptflow team. Learn more: suggested workflow.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

  • Pull request includes test coverage for the included changes.

Copy link

github-actions bot commented Jun 21, 2024

promptflow-evals test result

   12 files     12 suites   20m 6s ⏱️
   84 tests    84 ✅ 0 💤 0 ❌
1 008 runs  1 008 ✅ 0 💤 0 ❌

Results for commit 2eb4e6c.

♻️ This comment has been updated with latest results.

@nagkumar91 nagkumar91 marked this pull request as ready for review June 27, 2024 20:50
@nagkumar91 nagkumar91 requested a review from a team as a code owner June 27, 2024 20:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants