Skip to content

v1.2.0

Latest
Latest

Choose a tag to compare

View all tags
@hannahwestra25 hannahwestra25 released this 13 Nov 19:28
· 136 commits to main since this release
v1.2.0
5398ad1

PyRIT v1.2.0

What's changed?

PyRIT v1.2.0 delivers a major upgrade to the CoPyRIT GUI, making it easier to launch, monitor, compare, and report on security assessments. Reusable scenario configurations, grouped launches, persistent engagements and findings, and richer result analytics turn individual scans into repeatable, reviewable workflows. The release also expands attack techniques, targets, converters, datasets, and scoring options.

Python support is now >=3.11, <3.15.

🌟 Highlights

  • A major CoPyRIT GUI update brings scan launch, live progress, history, scoring, comparison, and review into one workflow.
  • Scenario presets and suites make assessments reusable and allow related scenario configurations to be launched together.
  • Operations and Findings organize engagements, capture human-reviewed issues and evidence, and support reporting.
  • New attacks and targets expand coverage with GOAT, role-play and jailbreak techniques, A2A, HackAPrompt, and GitHub Copilot.
  • More scoring and benchmark options add ASSERT, Microsoft decision-model scoring, local scorers, and several major datasets.
  • Improved reliability and security strengthen execution cleanup, resume behavior, validation, authorization, and persistence.

🤝 Community contributions

Community contributors drove a substantial part of this release: 108 external contributors authored 507 merged pull requests—58% of the 882 release pull requests identified from the release cut. Thank you to everyone who contributed features, datasets, tests, documentation, bug fixes, and reviews to PyRIT v1.2.0. See the New Contributors section below.

⚠️ Breaking changes

  • Python 3.10 is no longer supported. Upgrade to Python 3.11–3.14 before installing PyRIT 1.2.0.
  • Backend and client artifacts must match exactly. Compatibility-enforcing CoPyRIT backends reject CLI or browser clients built from a different source commit, even when the package version matches. Deploy the backend, frontend, and CLI from the same source commit, and do not roll back below the first compatibility-enforcing backend while newer clients remain active (#2829).
  • Scenario construction and dataset configuration changed. Custom scenarios must use the current parameter/context APIs and argument-free initialize_async(). Dataset selection now uses explicit sources, fetch policies, and limits; use "all" rather than None for unlimited selection (#3404, #2956, #2849, #2904, #3143).
  • Backend scenario reads are async-only. Integrations using synchronous scenario-read methods must migrate to and await the asynchronous APIs (#2953).
  • Custom MessageScorer subclasses must use the expectation-aware scoring hook. Replace overrides of _score_piece_async(..., objective=...) with _score_piece_with_expectation_async(..., expectation=...) and read scoring context from the expectation (#3312).
  • The Gandalf target and scorer were removed. Remove imports and configuration references, and migrate to another supported or custom target/scorer (#3467).
  • Some API target creation now requires administrator access. Non-admin users can no longer create targets that use server-stored credentials or execute model-repository code; use an administrator-created target or caller-provided credentials where supported (#3247, #3173).
  • API-created HTTP target uploads now use a server-selected folder. Callers must use the returned server location rather than relying on a requested upload path (#3172).

🧩 Framework, scenarios, and analytics

  • Added scenario presets—named, reusable configurations for one scenario—and scenario suites, which group ordered preset references for one-click launch and GUI editing (#3490, #2931, #3407).
  • Added explicit scenario dataset sources, limits, and generation providers across the SDK, CLI, and GUI (#2956, #3116, #3143).
  • Added structured run-size bounds, clearer progress roles, role-aware attempt accounting, and parent/child result browsing (#2739, #2997, #3062, #3611).
  • Added persistent Operations for grouping assessment activity and Findings for recording human-reviewed issues with linked attack evidence. Added attack analytics, saved-result analysis in CoPyRIT, and findings export to Word and Markdown (#3066, #3059, #3149, #3168, #3493).
  • Added technique.execute for running one technique directly against an objective (#3118).

⚔️ Attacks, targets, and converters

  • Added GOAT, context-wrap, character-split, tool role-play, and multi-turn jailbreak-delivery techniques (#2763, #3549, #3381, #3584, #3657).
  • Added GCG capability preflight checks before workers start (#3338).
  • Added A2A and HackAPrompt targets, a GitHub Copilot SDK target, and pluggable tools for the OpenAI Responses target (#2771, #2801, #2828, #2718).
  • Added a no-key authentication mode for OpenAI-compatible targets (#3809).
  • Added PromptTemplateConverter, nested and multiple conversion markers, multimodal chat pipelines, grid composition, image steganography, and configurable URL safety (#2845, #3002, #2699, #2936, #3256, #3402).

📊 Scoring and evaluation

  • Added seed scoring expectations and an observation lifecycle for surface scoring (#2738, #3156).
  • Added standalone ASSERT conversation scoring, native ASSERT prompt generation, Microsoft decision-model scoring, and explicit true/false verdict setting (#3539, #3271, #3188, #3317).
  • Added local CPU refusal and violence scorers, agent-threat rules, known-bad signature detection, and message tool-call scoring (#2772, #2626, #2624, #3009, #2854).
  • Expanded credential-leak and shell-command detection and improved scorer evaluation reporting for abstentions and coverage (#3286, #3832, #2860).
  • Fixed score inversion, rationale preservation, category matching, invalid thresholds, and malformed or out-of-range judge scores.

📚 Datasets and benchmarks

  • Added CyberSecEval prompt-injection and MITRE datasets, Anthropic model-written evaluations, MASK, HarmEval, FORTRESS, and SemGuard Arabic datasets (#3419, #3028, #2986, #1904, #2794, #1805, #2468).
  • Expanded Garak and privacy-leakage benchmark coverage.
  • Added dataset catalog navigation, paginated seed browsing, media previews, and import of dataset prompts and conversations into chat (#3071, #2989, #3251, #3670).

🖥️ GUI, CLI, and backend

  • Added a workflow-focused Home experience with scan results, live chats, latest-response scoring, consolidated history, contextual help, automatic refresh, and an expandable sidebar (#3642, #3643, #3644, #3645, #3646, #3649, #3650).
  • Added paired outcome comparison, scorer and technique registries, improved conversation branching, and saved chat drafts (#3337, #3408, #3187).
  • Added JSON and HTML report formats and output-file generation (#2733).
  • Added dataset summary APIs and exposed the scorer registry through the backend API (#2762, #2990).
  • Improved GUI result consistency, draft preservation, preset-save behavior, loading states, and history navigation.

🛠️ Reliability and operations

  • Improved scenario cancellation, resume behavior, worker shutdown, nested task cleanup, and concurrent-run/attempt accounting (#2851, #2857, #2856, #2695, #3774).
  • Added request-size, field-length, and path-parameter validation for backend-created components (#2960, #3594, #3595).
  • Improved SQLite cancellation cleanup, findings-export pagination safety, media conversion cleanup, and image/audio/video edge-case handling (#2982, #4008, #3836, #3240).
  • Declared aiohttp as a runtime dependency to prevent installation-time failures (#4174).

🗄️ Upgrade notes

This release includes Alembic migrations affecting persisted scores, scenarios, conversations, target identities, observations, and attack results. Back up production databases and verify the migration path before upgrading. Treat rollback as a coordinated application-and-database operation.

For compatibility-enforcing CoPyRIT deployments, stage matching frontend, CLI, and backend artifacts before switching traffic. Keep those matching client artifacts available and do not roll back the backend below the first compatibility-enforcing release while newer clients remain active.

🔒 Security

No known high-priority security issues.

  • Hardened API authorization for target creation that uses server-stored credentials or executes model-repository code (#3247, #3173).
  • Restricted API-created HTTP target uploads to server-selected folders and added backend request-size, field-length, and path-parameter validation (#3172, #2960, #3594, #3595).
  • Updated dependencies with available security patches.

New Contributors

Full changelog

See every merged change in v1.1.0...v1.2.0.

Contributors

pratikgx, jay23606, and 74 other contributors
Assets 2
Loading