Skip to content

v0.14.0

Latest
Latest

Choose a tag to compare

View all tags
@ValbuenaVC ValbuenaVC released this 05 Jun 22:22
v0.14.0
9a47100

What's changed?

Welcome to PyRIT v0.14.0! We've continued to expand the library with lots of new features. This release has a significant number of renames and refactors, so read the "Breaking Changes" section below carefully.

⚠️ Breaking Changes

Please review the deprecation notes and migration guidance before upgrading.

  • Core models migrated to Pydantic v2 — Message (#1885), MessagePiece (#1871), Score (#1891), AttackResult (#1899), ScenarioResult (#1908), the Seed* classes (#1898), Identifier classes (#1881), and other leaf types (#1769). Construction is now stricter (keyword-only, extra fields rejected) — update any positional or ad-hoc construction.
  • _async suffix enforced on all async functions across pyrit/; some async helpers were renamed (#1889, #1744).
  • Output/printer refactor — printers consolidated into a lightweight pyrit.output module; use await output_attack_async(result, ...) (with the new blur_images flag) instead of the old printer objects (#1732, #1768).
  • Renames: SeedDatasetProvider.fetch_dataset → fetch_dataset_async (#1735); BASELINE_POLICY → BASELINE_ATTACK_POLICY (#1763).
  • Removed all functionality previously deprecated for v0.14.0 (#1748).
  • GCG is now experimental (emits ExperimentalWarning) with a new GCG/GCGConfig public API; fastchat dependency dropped (#1792, #1049, #1861).
  • New deprecations (still functional, slated for removal): PromptChatTarget (#1678), Dall-E-specific image params (#1661), use_entra_auth on Azure Speech converters (#1634), MemoryExporter/export_conversations (#1870), display_image_response_async (#1930), label on MessagePiece (#1670), raise in PrependedConversationConfig (#1731), split kwarg on 8 single-split HF loaders (#1901), AtomicAttack(attack=...) (#1743, for v0.16.0), and ContentHarms/Originator aliases (#1816).

🎯 Targets & Attacks

  • Round Robin Target (#1761) and Realtime streaming session support + server-side barge-in attack (#1766).
  • TAP gains image functionality (#1036); PAIRAttack added as a TAP alias with PAIR-definitional defaults (#1822); StrategySequenceAttack compound primitive (#1819).
  • single_turn_crescendo technique with adversarial config (#1665) and crescendo persona techniques — movie director, history lecture, journalist interview (#1677).
  • Runtime capability discovery for prompt targets + migration to TargetConfiguration checks (#1699, #1645, #1778); an additional Microsoft target for api-version-sensitive Azure AI endpoints (#1730); HuggingFace reproducibility params + endpoint-target deprecation (#1672).
  • New converters: image color-saturation/resize/rotate (#1633), Image Filter (#1669), ImageOverlay (#1764), and an Arabic adversarial set — Bidi (#1832), Tatweel/kashida (#1869), ArabicPresentationForm (#1888), Arabizi (#1906); plus generalized AddTextImage/AddImageConverter (#1591) and Translation/Variation/Persuasion now inheriting LLMGenericTextConverter (#1714).

📚 Datasets

New loaders: VLGuard (#1447), ComicJailbreak (#1591), MOSSBench (#1786), FigStep/SafeBench (#1787), MSTS (#1757), SGXSTest (#1754), HiXSTest (#1755), DangerousQA (#1751), CategoricalHarmfulQA/CatQA (#1749), CoCoNot (#1802), SIUO (#1799), StrongREJECT (#1800), DecodingTrust Toxicity (#1821), JailbreakV-28k (#1098), Agent Threat Rules/ATR (#1715), SALT-NLP MIC (#1831), and 0DIN JEF n-day sets (#1398). Plus class-level metadata backfill + author/affiliation YAML enrichment across datasets (#1780, #1834) and a shared multimodal image-fetch helper (#1776).

📊 Scoring

  • RegexScorer + CredentialLeakScorer for regex-based secret detection (#1704).
  • PromptInjectionScorer (OWASP LLM01) (#1774) and an OWASP LLM02 output-side pack — XSS / SQLi / Shell / Path (#1868); 0DIN JEF keyword scorers (#1398).
  • Score partial content from content-filtered responses (#1689); unified error/blocked-response scoring across scorers (#1770).

🖥️ CoPyRIT (GUI) & CLI

  • Isolated GUI deployment automation + guide, storage-account provisioning, inline Container App secret with Key Vault lockdown, and AKV-referenced secrets (#1655, #1658, #1693, #1721, #1836).
  • REST API for scenarios: listing endpoints, run, parameters/initializers, and initializer scripts (#1666, #1696, #1724, #1728); custom scenario parameters via CLI/YAML (#1680).
  • GUI UX: signed-in user display (#1636), searchable Attack History filters (#1643), Entra auth for new targets (#1762), AzureML target support (#1681), modality-aware send blocking (#1692), Home landing page (#1750), chat ribbon redesign + contrast/label fixes (#1736, #1708, #1711), structured capability columns (#1691), pretty-printed JSON responses (#1706), and a frontend core refactor (#1753).

🧩 Framework internals

  • DB schema tracking via Alembic, including a safe upgrade path from 0.13.0 (#1631, #1772, #1895).
  • Memory-interface batching (#1325); lazy imports for startup performance (#1668); eliminated blocking I/O on async paths (#1878).
  • Migration mypy → ty with strict typing (#1319, #1515); Python 3.14 support (#1130); pathlib + JSON-serialization standardization and to_dict/from_dict roundtrips (#1877, #1815, #1813, #1738); defined pyrit.models boundary and moved Identifiers into it (#1771, #1858).

🔃 Scenarios

  • New Rapid Response (#1622), Adversarial Benchmark (#1662, refactor #1765), and text-adaptive (#1760) scenarios.
  • Better scenario tracking (#1758), parallel atomic-attack execution (#1783), per-group success-rate sorting (#1809), scenario/attack error propagation (#1720), technique-registry consolidation (#1654, #1785), Leakage scenario refactor (#1687), and unified default adversarial/scorer target wiring (#1695).

🔒 Security

🧪 Tests & Tooling

  • Added Dependabot configuration (#1835) and a broad ruff/ty/CI dependency-bump rollout; merge-queue CI support (#1663).
  • GCG unit/integration tests no longer hit HuggingFace (#1684, #1886); unit-test speedups and flaky-test fixes (#1872, #1874, #1716, #1795, #1797); build-script tests (#1642); kernelspec-stripping pre-commit hook (#1637).

📖 Docs

  • Multi-version docs on GitHub Pages (replaces ReadTheDocs) (#1866); auto-linked API symbol references (#1823); scoring docs refactor (#1892); citation/bibliography cleanup (#1686, #1747); MyST cross-reference polish + reST-role removal (#1824, #1782); cross-OS troubleshooting + per-OS ODBC steps (#1932); simplified Getting Started (#1727); dataset-loader contributor guide (#1775); Scoring Evaluations blog (#1617).

🐛 Bug Fixes and Improvements

See the full changelog below for everything else.

Full list of changes

New Contributors

Full Changelog: v0.13.0...v0.14.0

Contributors

athal7, cclauss, and 29 other contributors
Assets 2
Loading