FEAT Backend attack API: conversation-centric redesign with multi-conversation workspaces and media serving by romanlutz · Pull Request #1419 · microsoft/PyRIT

romanlutz · 2026-03-01T13:28:27Z

Backend Attack API: conversation-centric redesign

Why

The original backend API modeled attacks as single-conversation entities: one attack, one conversation, one target. This doesn't match how red-teaming actually works. Operators need to branch
conversations mid-attack to explore alternative paths, compare outcomes, or retry with different system prompts — all without losing the original conversation history.

This PR restructures the API around the idea that an attack is a workspace containing multiple conversations. A new attack still starts with one "main" conversation, but operators can now branch from
any point in that conversation, send messages to specific branches, and swap which branch is considered "main."

What changed

Conversation management (`/attacks/{id}/conversations`)

List, create (with branching via source_conversation_id + cutoff_index), and swap main conversations within an attack
Messages can be routed to any conversation via target_conversation_id
source_conversation_id is validated against the attack's known conversations to prevent cross-attack history leakage
Related conversations are tracked with correct lifecycle states (ADVERSARIAL → PRUNED on swap)

Media serving (`/api/media`)

New endpoint serves local media files (image_path, audio_path, video_path) via FileResponse with proper MIME types
Replaces inline base64 data-URI encoding, eliminating memory spikes from large files (especially video)
Azure Blob Storage URLs are signed with SAS tokens (cached with TTL derived from token expiry)
Path traversal protection: only files under the configured results_path are served

Attack list & discovery

/attacks returns paginated summaries with aggregated ConversationStats (turn counts, labels, preview) instead of loading full message pieces
Conversation IDs are de-duplicated before batch stats queries
/attack-options and /converter-options populate UI dropdowns from database history

Score model enhancements

Score DTO includes score_type and score_category
true_false scores expose score_value_as_str for readable display

Video target improvements

_validate_video_remix_pieces enforces video_id presence and consistency across text + video_path pieces
Remix validation logic moved from AttackService into OpenAIVideoTarget where it belongs
prompt_metadata field added to MessagePieceRequest DTO for passing video_id through the API

Backend models & exports

backend/models/__init__.py consolidates all DTO exports; no missing re-exports
PrependedMessageRequest, TargetInfo, AttackOptionsResponse, ConverterOptionsResponse, and other models properly exported

CLI

pyrit_backend CLI gains --config-file argument and uses FrontendCore for initialization (consistent with pyrit_scan / pyrit_shell)
frontend_core.py imports moved to module level per style guide

Version endpoint (`/api/version`)

Reports PyRIT version and active database backend for debugging

What didn't change

The memory layer (MemoryInterface, SQLiteMemory, models) is unchanged — those pieces landed in earlier PRs. This PR builds the API surface on top of the existing memory primitives.

Test coverage

432 tests across backend, CLI, and video target (429 passing, 3 skipped for Azure-only paths)
99% line coverage on pyrit/backend/ (1137 statements, 13 uncovered — all Azure credential or DB-detection paths)
New test files: test_media_route.py (media endpoint), test_pyrit_backend.py (CLI), expanded test_video_target.py (remix validation)

- Add run_initializers_async to pyrit.setup for programmatic initialization - Switch AIRTInitializer to Entra (Azure AD) auth, removing API key requirements - Add --config-file flag to pyrit_backend CLI - Use PyRIT configuration loader in FrontendCore and pyrit_backend - Update AIRTTargetInitializer with new target types Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Add conversation_stats model and attack_result extensions - Add get_attack_results with filtering by harm categories, labels, attack type, and converter types to memory interface - Implement SQLite-specific JSON filtering for attack results - Add memory_models field for targeted_harm_categories - Add prompt_metadata support to openai image/video/response targets - Fix missing return statements in SQLite harm_category and label filters Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…nd-attack-api

- Add attack CRUD routes with conversation management - Add message sending with target dispatch and response handling - Add attack mappers for domain-to-DTO conversion with signed blob URLs - Add attack service with video remix support and piece persistence - Expand target service and routes with registry-based target management - Add version endpoint with database info Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Builds out the PyRIT backend from a scaffold into an attack-centric API, adding multi-conversation attack support, conversation summarization, target registry interactions, and memory-layer enhancements (including Entra-auth-related initializer updates).

Changes:

Adds attack/conversation backend endpoints keyed by attack_result_id, plus message sending that dispatches to targets by registry name.
Introduces ConversationStats and new memory APIs to efficiently summarize conversations without loading full message pieces.
Updates setup/CLI and initializers (incl. run_initializers_async + config-file support) and expands OpenAI target/mapping behavior for richer metadata/media handling.

Reviewed changes

Copilot reviewed 43 out of 43 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
tests/unit/target/test_video_target.py	Adds regression test for single-turn enforcement on video target requests.
tests/unit/target/test_image_target.py	Adds async regression test for single-turn enforcement on image target requests.
tests/unit/setup/test_airt_targets_initializer.py	Adds test coverage for GPT-5 high-reasoning Responses target extra body parameters.
tests/unit/setup/test_airt_initializer.py	Updates AIRT initializer tests to reflect Entra/token auth (no API key env vars).
tests/unit/memory/test_sqlite_memory.py	Adds unit tests for `get_conversation_stats` aggregation behavior.
tests/unit/memory/memory_interface/test_interface_attack_results.py	Expands tests for attack result dedupe + update semantics and renames attack/converter filters.
tests/unit/cli/test_pyrit_backend.py	Adds CLI tests for `pyrit_backend` arg parsing and initialization wiring.
tests/unit/cli/test_frontend_core.py	Updates FrontendCore expectations + patches to new initialization flow and defaults.
tests/unit/backend/test_target_service.py	Updates tests for renamed target fields (`target_registry_name`).
tests/unit/backend/test_mappers.py	Adds extensive mapper tests for media handling, blob URL signing/fetching, and summary mapping changes.
tests/unit/backend/test_main.py	Updates lifespan tests to reflect CLI-driven initialization instead of in-app initialization.
tests/unit/backend/test_api_routes.py	Updates route tests for new endpoints, request/response shapes, and renamed filters/fields.
pyrit/setup/initializers/airt_targets.py	Adds target config `extra_kwargs` and new GPT-5 high-reasoning Responses target; adjusts video target config naming/env vars.
pyrit/setup/initializers/airt.py	Switches AIRTInitializer to Entra/token auth and updates scorer setup accordingly.
pyrit/setup/initialization.py	Adds `run_initializers_async` helper to run initializers without reinitializing memory.
pyrit/setup/init.py	Re-exports `run_initializers_async`.
pyrit/prompt_target/openai/openai_video_target.py	Adds validation to enforce single-turn conversations for video target usage.
pyrit/prompt_target/openai/openai_target.py	Refactors OpenAITarget base inheritance/initialization behavior.
pyrit/prompt_target/openai/openai_response_target.py	Includes `extra_body_parameters` in target identifiers for richer provenance.
pyrit/prompt_target/openai/openai_realtime_target.py	Adjusts realtime target inheritance to ensure chat-target semantics.
pyrit/prompt_target/openai/openai_image_target.py	Adds validation to enforce single-turn conversations for image target usage.
pyrit/models/conversation_stats.py	Introduces `ConversationStats` dataclass for lightweight conversation summaries.
pyrit/models/attack_result.py	Adds `attack_result_id` to expose persisted identifier to callers.
pyrit/models/init.py	Exports `ConversationStats`.
pyrit/memory/sqlite_memory.py	Implements `get_conversation_stats`, improves update semantics, and renames attack/converter filter helpers.
pyrit/memory/memory_models.py	Populates `attack_result_id` when constructing `AttackResult` from DB entry.
pyrit/memory/memory_interface.py	Adds `get_conversation_stats`, exposes `duplicate_messages`, and updates attack/converter filter APIs + update helpers.
pyrit/memory/azure_sql_memory.py	Mirrors SQLite changes for attack/converter filters and implements `get_conversation_stats`.
pyrit/cli/pyrit_backend.py	Refactors backend CLI to use FrontendCore + adds `--config-file`.
pyrit/cli/frontend_core.py	Splits “initialize” vs “run initializers” behavior and reuses `run_initializers_async`.
pyrit/backend/services/target_service.py	Renames API surface to `target_registry_name` and adjusts pagination cursor semantics.
pyrit/backend/services/attack_service.py	Major refactor: `attack_result_id`-keyed operations, multi-conversation support, branching, message routing, and async DTO mapping.
pyrit/backend/routes/version.py	Adds optional database backend info to version response.
pyrit/backend/routes/targets.py	Renames target route params to `target_registry_name`.
pyrit/backend/routes/attacks.py	Adds conversation management endpoints and refactors message endpoints to support per-conversation routing.
pyrit/backend/models/targets.py	Updates DTO fields (registry naming) and adds `supports_multiturn_chat`.
pyrit/backend/models/attacks.py	Refactors attack DTOs for `attack_result_id`, target info nesting, conversation endpoints, and message routing fields.
pyrit/backend/models/init.py	Updates exported backend models to reflect renamed/removed DTOs.
pyrit/backend/mappers/target_mappers.py	Updates target mapping to registry naming and adds multiturn capability detection.
pyrit/backend/mappers/attack_mappers.py	Refactors summary mapping to use `ConversationStats` and adds media/blob URL signing/fetching logic + async message DTO mapping.
pyrit/backend/mappers/init.py	Exports `pyrit_messages_to_dto_async` instead of sync variant.
pyrit/backend/main.py	Removes implicit initialization in lifespan; warns when started without CLI initialization.

pyrit/cli/pyrit_backend.py

pyrit/backend/routes/attacks.py

pyrit/backend/mappers/attack_mappers.py

pyrit/memory/memory_interface.py

pyrit/backend/services/attack_service.py

…ack-api

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 25 out of 25 changed files in this pull request and generated 1 comment.

tests/unit/memory/memory_interface/test_interface_attack_results.py

…ack-api

Copilot

Pull request overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated 7 comments.

pyrit/backend/routes/attacks.py

tests/unit/memory/memory_interface/test_interface_attack_results.py

pyrit/backend/services/attack_service.py

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…ack-api

Copilot

Pull request overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated 5 comments.

tests/unit/backend/test_attack_service.py

pyrit/backend/services/attack_service.py

pyrit/backend/mappers/attack_mappers.py

pyrit/backend/models/attacks.py

Resolve merge conflicts favoring backend-attack-api (PR microsoft#1419) changes: - Use set-based deduplication for conversation IDs - Use FrontendCore in pyrit_backend CLI - Remove old video_id injection helpers (moved to video target) - Use ConversationStats-based aggregation - Fix SAS cache TTL, score DTOs, version endpoint - Add Path import to pyrit_backend.py - Fix created_at type (datetime, not str) in list_conversations Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…ack-api

- Add 400 bad request handling to create_related_conversation endpoint with validation for source_conversation_id belonging to the attack - Add HTTPException 500 to serve_media_async docstring - Increase filename hash truncation from 8 to 12 characters - Rename initialize_and_run to initialize_and_run_async per convention - Handle /api/media?path= URLs and existing file paths in _persist_base64_pieces_async to prevent base64 decode errors - Replace attack_result_id or '' fallbacks with RuntimeError assertions - Merge latest main Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 1 comment.

You can also share your feedback on Copilot code review. Take the survey.

pyrit/backend/routes/media.py

…itive extensions Prevents exfiltration of database files and other sensitive data by: - Only serving files from prompt-memory-entries/ and seed-prompt-entries/ - Blocking .db, .sqlite, .json, .yaml, .env and other sensitive extensions - Rejecting files in the results_path root directory Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

pyrit/backend/services/attack_service.py

pyrit/backend/models/attacks.py

pyrit/backend/routes/media.py

pyrit/backend/routes/version.py

- Fix phase numbering in list_attacks_async (Phase 2 -> Phase 3) - Add TODO comment on RuntimeError for attack_result_id assertion - Split add_message_async into focused helpers: _validate_target_match, _validate_operator_match, _resolve_labels, _update_attack_after_message_async - Rename op_name label to operator_name for clarity - Add comment explaining results[0] usage with unique ID queries - Remove duplicate source_conversation_id validation - Rename ChangeMainConversation -> UpdateMainConversation (request, response, route, service method) - Add updated_at field to UpdateMainConversationResponse Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Instead of leaving attack_result_id as None and relying on DB backfill, generate a UUID in the constructor when not provided. This ensures attack_result_id is always populated, eliminating the need for None checks and RuntimeError assertions. - AttackResult.attack_result_id: Optional[str]=None -> str=uuid4() - AttackResultEntry uses entry.attack_result_id instead of new uuid - Remove backfill loop from add_attack_results_to_memory - Remove RuntimeError checks in attack_service.py and attack_mappers.py Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The frontend uses 'operator' and 'operation' as label keys. The previous rename from op_name to operator_name was incorrect - op_name referred to 'operation', not 'operator'. Fix the operator validation to use the correct 'operator' label key. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Rename all label key occurrences across codebase: - op_name -> operation - user_name/username -> operator Updated in backend service, tests, and documentation files (.py and .ipynb) for consistency with frontend LabelsBar defaults. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Switch media endpoint from extension blocklist to allowlist - Use consistent db_type format 'Type (None)' when db_name is absent Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…ack-api

Replace uuid-based test_operation/test_operator with readable examples 'op_trash_panda' and 'roakey' matching frontend defaults. Remove unused uuid imports. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…bject Add methods to AttackResult: - includes_conversation(): check if a conversation belongs to this attack - get_all_conversation_ids(): main + all related - get_active_conversation_ids(): main + pruned (user-visible) - get_pruned_conversation_ids(): pruned only Refactor AttackService to delegate to these methods instead of inline set comprehensions, reducing duplication across 5 methods. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…ack-api

…versation workspaces and media serving (microsoft#1419) Co-authored-by: Roman Lutz <romanlutz@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

romanlutz and others added 4 commits February 28, 2026 14:49

Merge branch 'romanlutz/memory-models-expansion' into romanlutz/backe…

792dc8c

…nd-attack-api

Copilot AI review requested due to automatic review settings March 1, 2026 13:28

Copilot started reviewing on behalf of romanlutz March 1, 2026 13:28 View session

Copilot AI reviewed Mar 1, 2026

View reviewed changes

romanlutz changed the title ~~FEAT backend attack api~~ FEAT backend attack API - revisiting attacks & conversations Mar 1, 2026

romanlutz mentioned this pull request Mar 2, 2026

FEAT Add TargetCapabilities with supports_multi_turn and adapt attacks accordingly #1433

Merged

romanlutz and others added 5 commits March 2, 2026 14:52

Merge remote-tracking branch 'origin/main' into romanlutz/backend-att…

c08b76f

…ack-api

fix: auto ruff format fixes

0a4280d

fix: address ruff and mypy lint issues

96d5839

fix: ruff format

14fb53f

fix: resolve merge conflicts, align service->memory naming

ad31d5b

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings March 3, 2026 00:53

Copilot started reviewing on behalf of romanlutz March 3, 2026 00:54 View session

Copilot AI reviewed Mar 3, 2026

View reviewed changes

tests/unit/memory/memory_interface/test_interface_attack_results.py Outdated Show resolved Hide resolved

romanlutz added 2 commits March 2, 2026 20:50

Merge remote-tracking branch 'origin/main' into romanlutz/backend-att…

84f2c9d

…ack-api

Merge remote-tracking branch 'origin/main' into romanlutz/backend-att…

4440cfd

…ack-api

Copilot AI review requested due to automatic review settings March 3, 2026 05:01

Copilot started reviewing on behalf of romanlutz March 3, 2026 05:01 View session

Copilot AI reviewed Mar 3, 2026

View reviewed changes

romanlutz and others added 2 commits March 2, 2026 21:26

fix: address copilot comments - security, cleanup, API contract

b48e23f

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Merge remote-tracking branch 'origin/main' into romanlutz/backend-att…

6bd4737

…ack-api

Copilot AI review requested due to automatic review settings March 3, 2026 05:33

Copilot started reviewing on behalf of romanlutz March 3, 2026 05:33 View session

Copilot AI reviewed Mar 3, 2026

View reviewed changes

hannahwestra25 reviewed Mar 3, 2026

View reviewed changes

pyrit/backend/mappers/attack_mappers.py Show resolved Hide resolved

hannahwestra25 reviewed Mar 3, 2026

View reviewed changes

pyrit/backend/mappers/attack_mappers.py Outdated Show resolved Hide resolved

hannahwestra25 reviewed Mar 3, 2026

View reviewed changes

pyrit/backend/models/attacks.py Show resolved Hide resolved

romanlutz and others added 3 commits March 5, 2026 21:40

Merge remote-tracking branch 'origin/main' into romanlutz/backend-att…

1513745

…ack-api

fix: add type ignore for ContainerSasPermissions across mypy versions

f3617f2

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings March 6, 2026 06:04

Copilot started reviewing on behalf of romanlutz March 6, 2026 06:05 View session

Copilot AI reviewed Mar 6, 2026

View reviewed changes

pyrit/backend/routes/media.py Show resolved Hide resolved