Skip to content

feat: add tap vs goat eval#183

Merged
rdheekonda merged 2 commits into
mainfrom
feat/airt-examples
Sep 17, 2025
Merged

feat: add tap vs goat eval#183
rdheekonda merged 2 commits into
mainfrom
feat/airt-examples

Conversation

@rdheekonda
Copy link
Copy Markdown
Contributor

@rdheekonda rdheekonda commented Sep 17, 2025

[[[# [Title of Your PR]

Key Changes:

  • List major changes and core updates
  • Keep each line under 80 characters
  • Focus on the "what" and "why"

Added:

  • New features/functionality
  • New files/configurations
  • New dependencies

Changed:

  • Updates to existing code
  • Configuration changes
  • Dependency updates

Removed:

  • Deleted files/code
  • Removed dependencies
  • Cleaned up configurations

Generated Summary:

  • Added a new dataset adversarial_benchmark_subset.csv containing 70 adversarial examples categorized by their intent and associated responses.
  • Introduced the tap_vs_goat_eval.ipynb notebook which evaluates and compares the effectiveness of two adversarial attack methods: TAP (Tree of Attacks) and GOAT (Graph of Attacks).
  • Implemented comprehensive logging and metrics capturing for both TAP and GOAT attacks including success indicators, execution times, and trial statistics.
  • This addition aims to enable detailed evaluation and analysis of adversarial strategies against the provided dataset, facilitating improvements in security measures and AI resilience.
  • The notebook showcases setup for running attacks, evaluates the performance, and aggregates results into a structured output, aiding in further research and analysis of adversarial AI.

This summary was generated with ❤️ by rigging

](feat: add tap vs goat eval)](feat: add tap vs goat eval)](feat: add tap vs goat eval)

@dreadnode-renovate-bot dreadnode-renovate-bot Bot added the area/examples Changes to example code and demonstrations label Sep 17, 2025
@rdheekonda rdheekonda merged commit 428ef03 into main Sep 17, 2025
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/examples Changes to example code and demonstrations

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant