diff --git a/docs/guardrails.md b/docs/guardrails.md index 70bb0d7e3b..9b258e824d 100644 --- a/docs/guardrails.md +++ b/docs/guardrails.md @@ -51,6 +51,8 @@ Output guardrails run in 3 steps: Output guardrails always run after the agent completes, so they don't support the `run_in_parallel` parameter. +An output tripwire and an exception raised by the guardrail function have different session behavior. A tripwire rejects the candidate final output. When a tripwire fires, the runner asks the configured session to persist already-completed tool call and tool output items, together with any reasoning context required to replay those calls, while excluding the rejected candidate final output. The runner applies this tripwire rule to both streaming and non-streaming runs. When the guardrail function raises an exception instead of returning a tripwire result, the runner treats the verdict as unknown and asks the configured session to persist the completed final-turn items before surfacing the guardrail exception. If that session write also fails, the session write error takes precedence. Streaming runs use the same persistence ordering as non-streaming runs and raise the terminal exception from `stream_events()`. An immediate [`RunResultStreaming.cancel()`][agents.result.RunResultStreaming.cancel] call while the output guardrail is running cancels the in-flight guardrail and does not start a final-turn session write. + ## Tool guardrails Tool guardrails wrap **`FunctionTool` instances** and let you validate or block calls to those tools before and after execution. They are configured on the tool itself and run every time that tool is invoked. diff --git a/docs/human_in_the_loop.md b/docs/human_in_the_loop.md index 17b4e89200..c153fe4b11 100644 --- a/docs/human_in_the_loop.md +++ b/docs/human_in_the_loop.md @@ -45,13 +45,15 @@ agent = Agent( ## How the approval flow works 1. When the model emits a tool call, the runner evaluates its approval rule (`needs_approval`, `require_approval`, or the hosted MCP equivalent). -2. If an approval decision for that tool call is already stored in the [`RunContextWrapper`][agents.run_context.RunContextWrapper], the runner proceeds without prompting. Per-call approvals are scoped to the specific call ID; pass `always_approve=True` or `always_reject=True` to persist the same decision for future calls to that tool during the rest of the run. +2. If an approval decision for that tool call is already stored in the [`RunContextWrapper`][agents.run_context.RunContextWrapper], the runner proceeds without prompting. Per-call approvals are scoped to the specific call ID; pass `always_approve=True` or `always_reject=True` to persist the same decision for future calls to the same tool identity during the rest of the run. 3. If the approval rule requires approval and no decision for that tool call is stored, execution pauses, and `RunResult.interruptions` (or `RunResultStreaming.interruptions`) contains [`ToolApprovalItem`][agents.items.ToolApprovalItem] entries with details such as `agent.name`, `tool_name`, and `arguments`. This includes approvals raised after a handoff or inside nested `Agent.as_tool()` executions. 4. Convert the result to a `RunState` with `result.to_state()`, call `state.approve(...)` or `state.reject(...)`, and then resume with `Runner.run(agent, state)` or `Runner.run_streamed(agent, state)`, where `agent` is the original top-level agent for the run. 5. The resumed run continues where it left off and will re-enter this flow if new approvals are needed. Sticky decisions created with `always_approve=True` or `always_reject=True` are stored in the run state, so they survive `state.to_string()` / `RunState.from_string(...)` and `state.to_json()` / `RunState.from_json(...)` when you resume the same paused run later. +For approval requests from [`HostedMCPTool`][agents.tool.HostedMCPTool], the Agents SDK identifies a sticky tool decision by the combination of `server_label` and tool name. An always-approve decision for `lookup_account` on one hosted MCP server does not approve a tool with the same name on another server. The Agents SDK persists an always-approve or always-reject decision only when the hosted MCP approval request includes both non-empty identity fields. + You do not need to resolve every pending approval in the same pass. `interruptions` can contain a mix of regular function tools, hosted MCP approvals, and nested `Agent.as_tool()` approvals. If you rerun after approving or rejecting only some items, those resolved calls can continue while unresolved ones remain in `interruptions` and pause the run again. ## Custom rejection messages diff --git a/docs/mcp.md b/docs/mcp.md index 38255d3d8f..0ba85e381e 100644 --- a/docs/mcp.md +++ b/docs/mcp.md @@ -26,6 +26,32 @@ Before wiring an MCP server into an agent decide where the tool calls should exe The sections below walk through each option, how to configure it, and when to prefer one transport over another. +## MCP Python SDK v1 and v2 + +The Agents SDK supports both major versions of the `mcp` Python package through the dependency range `mcp>=1.19.0,<3`. The installed `mcp` package version is separate from the MCP protocol version negotiated with a server. The Agents SDK detects the installed package major version and adapts stdio, SSE, and Streamable HTTP connections automatically, so ordinary server configuration does not need a version switch. + +Most applications should let their dependency resolver select a compatible version. If your application must stay on one major version, add an explicit constraint alongside `openai-agents`: + +```bash +# MCP Python SDK v1 +pip install "mcp>=1.19.0,<2" + +# MCP Python SDK v2 +pip install "mcp>=2,<3" +``` + +HTTP transport customization must use the HTTP stack owned by the installed MCP package: + +| Customization | MCP Python SDK v1 | MCP Python SDK v2 | +| --- | --- | --- | +| `params["auth"]` | `httpx.Auth` | `httpx2.Auth` | +| `params["httpx_client_factory"]` return value | `httpx.AsyncClient` | `httpx2.AsyncClient` | +| `MCPServerStreamableHttp` `params["ignore_initialized_notification_failure"] = True` | Supported | Not supported; rejected before connecting | + +Use an `Authorization` header when possible, as shown in the Streamable HTTP example below; an `Authorization` header works unchanged with both package versions. When an application supplies `params["auth"]` or `params["httpx_client_factory"]`, those values must use the HTTP types for the installed `mcp` package major version. When an application sets `MCPServerStreamableHttp`'s `params["ignore_initialized_notification_failure"] = True`, the application must keep `mcp<2` or disable the option before upgrading. + +These local `mcp` dependency requirements do not apply to [`HostedMCPTool`][agents.tool.HostedMCPTool] because the OpenAI Responses API owns the remote MCP connection. + ## Agent-level MCP configuration In addition to choosing a transport, you can tune how MCP tools are prepared by setting `Agent.mcp_config`. diff --git a/docs/models/index.md b/docs/models/index.md index 447e206150..d547d04c7e 100644 --- a/docs/models/index.md +++ b/docs/models/index.md @@ -23,7 +23,7 @@ Start with the simplest path that fits your setup: For most OpenAI-only apps, the recommended path is to use string model names with the default OpenAI provider and stay on the Responses model path. -When you don't specify a model when initializing an `Agent`, the default model will be used. The default is currently [`gpt-5.4-mini`](https://developers.openai.com/api/docs/models/gpt-5.4-mini) with `reasoning.effort="none"` and `verbosity="low"` for low-latency agent workflows. If you have access, we recommend setting your agents to `gpt-5.6-sol` for higher quality while keeping explicit `model_settings`. +When an [`Agent`][agents.agent.Agent] does not specify a model, the Agents SDK uses [`gpt-5.6-luna`](https://developers.openai.com/api/docs/models/gpt-5.6-luna) with `reasoning.effort="none"` and `verbosity="low"` by default for cost-sensitive, high-volume agent workflows. Applications that need frontier capability can explicitly set `model="gpt-5.6-sol"` and choose `model_settings` that are appropriate for the workload. If you want to switch to other models like `gpt-5.6-sol`, there are two ways to configure your agents. @@ -545,11 +545,12 @@ A retry policy receives a [`RetryPolicyContext`][agents.retry.RetryPolicyContext - `error` for raw inspection. - `normalized` facts such as `status_code`, `retry_after`, `error_code`, `is_network_error`, `is_timeout`, and `is_abort`. - `provider_advice` when the underlying model adapter can supply retry guidance. +- `response_started`, `replay_safety`, and `stateful_request` as stable replay-safety facts captured before the policy runs. `replay_safety` is `"safe"`, `"unsafe"`, or `"unknown"`; `stateful_request` is true when the request uses `previous_response_id` or `conversation_id`. The policy can return either: - `True` / `False` for a simple retry decision. -- A [`RetryDecision`][agents.retry.RetryDecision] when you want to override the delay or attach a diagnostic reason. +- A [`RetryDecision`][agents.retry.RetryDecision] when you want to override the delay, attach a diagnostic reason, or explicitly approve a narrowly scoped unsafe replay. The SDK exports ready-made helpers on `retry_policies`: @@ -567,13 +568,15 @@ When you compose policies, `provider_suggested()` is the safest first building b ##### Safety boundaries -Some failures are never retried automatically: +Some failures are never retried: - Abort errors. -- Requests where provider advice marks replay as unsafe. - Streamed runs after output has already started in a way that would make replay unsafe. +- Requests with a separate local-side-effect replay veto, including Programmatic Tool Calling requests, unless the provider has independently marked the replay safe. -Stateful follow-up requests using `previous_response_id` or `conversation_id` are also treated more conservatively. For those requests, non-provider predicates such as `network_error()` or `http_status([500])` are not enough by themselves. The retry policy should include a replay-safe approval from the provider, typically via `retry_policies.provider_suggested()`. +Provider-marked unsafe failures are also blocked by default. For a non-streaming request without a separate local-side-effect veto, an application can accept the provider-side replay risk by returning `RetryDecision(retry=True, approve_unsafe_replay=True)`. Check `context.response_started`, `context.replay_safety`, and `context.stateful_request` before granting this approval, and grant it only when repeating provider-side work is acceptable. An ordinary `RetryDecision(retry=True)` never bypasses replay protection, and `approve_unsafe_replay=True` cannot authorize streamed retries or local side effects. + +Stateful follow-up requests using `previous_response_id` or `conversation_id` fail closed when replay safety is unknown. For those requests, non-provider predicates such as `network_error()` or `http_status([500])` are not enough by themselves. Include a replay-safe approval from the provider, typically via `retry_policies.provider_suggested()`, or explicitly approve a non-streaming failure that the provider marked unsafe as described above. ##### Runner and agent merge behavior @@ -624,6 +627,8 @@ result = await Runner.run( If you use [`MultiProvider`][agents.MultiProvider], pass `openai_strict_feature_validation=True` instead. +The OpenAI Chat Completions API can return audio output, but [`OpenAIChatCompletionsModel`][agents.models.openai_chatcompletions.OpenAIChatCompletionsModel] does not currently convert audio output into Agents SDK run items. If a non-streaming message or streaming delta contains audio output, the adapter raises `AgentsException("Audio is not currently supported")` instead of returning a partial or empty result. Use [Realtime agents](../realtime/guide.md) or [Voice agents](../voice/quickstart.md) for SDK-managed audio workflows. + Some OpenAI-compatible Chat Completions providers stream tool-call deltas in chunks that are not reliable enough for incremental SDK processing. In that case, enable streamed tool-call buffering so the SDK emits tool calls only after the provider stream finishes: ```python diff --git a/docs/release.md b/docs/release.md index 369d8f9219..718208ebd2 100644 --- a/docs/release.md +++ b/docs/release.md @@ -19,6 +19,20 @@ We will increment `Z` for non-breaking changes: ## Breaking change changelog +### 0.20.0 + +Version 0.20.0 includes a potentially breaking MCP dependency migration for applications that customize local MCP HTTP transports. It also updates the SDK default model used when an agent or run does not explicitly select one. + +Highlights: + +- The SDK default model is now `gpt-5.6-luna` instead of `gpt-5.4-mini`. The default `reasoning.effort="none"` and `verbosity="low"` settings are unchanged. +- Explicit agent models, run-level model overrides, and the `OPENAI_DEFAULT_MODEL` environment variable continue to take precedence over the SDK default. +- Local MCP connections created by the Agents SDK now support MCP Python SDK v2 while retaining v1 compatibility through `mcp>=1.19.0,<3`. The Agents SDK adapts ordinary stdio, SSE, and Streamable HTTP connections automatically. If dependency resolution selects MCP v2, applications that supply custom `httpx.Auth` objects or `httpx.AsyncClient` factories must migrate those values to `httpx2`, or pin `mcp<2` to retain the v1 HTTP stack. `MCPServerStreamableHttp`'s `params["ignore_initialized_notification_failure"] = True` option also remains v1-only. See [MCP Python SDK v1 and v2](mcp.md#mcp-python-sdk-v1-and-v2) for migration details. +- Sandbox mount validation now rejects unsafe credential placement before sandbox or mount-helper side effects. Trusted applications can acknowledge mount-scoped or broad credential exposure for an exact in-container mount path without changing the storage capability tables. These acknowledgements are runtime-only and serialized sandbox state never grants credential authority by itself. See [Mounts and remote storage](sandbox/clients.md#mounts-and-remote-storage) and [Resume from session state](sandbox/guide.md#resume-from-session-state). +- Retry policies can inspect stable replay-safety facts and explicitly set `RetryDecision(approve_unsafe_replay=True)` for a non-streaming request that the provider marked unsafe. This approval does not bypass aborts, emitted streamed output, or separate local-side-effect vetoes such as Programmatic Tool Calling. See [Runner-managed retries](models/index.md#runner-managed-retries). +- Resumable `RunState` objects can now stage durable user input with `add_input()` before the next model call. Staged input survives serialization, runs through input guardrails, and produces one durable SDK input occurrence across local sessions and server-managed conversations. An explicitly approved unsafe replay can still resend the input to the provider and repeat provider-side work. See [Add input before resuming](results.md#add-input-before-resuming). +- Runtime reliability fixes align streamed and non-streamed [output-guardrail session persistence](guardrails.md#output-guardrails), preserve `FunctionTool` subclasses during copying and namespacing, and raise an explicit error for [unsupported Chat Completions audio output](models/index.md#chat-completions-compatibility-options) instead of silently completing an empty stream. The `OpenAIResponsesCompactionSession` wrapper attempts and awaits [pre-compaction history recovery](sessions/index.md#auto-compaction-can-block-streaming) before cancellation reaches the caller. `RunState` round trips now preserve local shell output, acknowledged computer safety checks, and default-valued tool output fields; MCP conversion preserves free-form object schemas; and model replay removes server-owned `created_by` metadata from output items before using them as input. + ### 0.19.0 This minor release does **not** introduce a breaking change. The minor version bump reflects a significant new OpenAI Responses feature area: Programmatic Tool Calling. diff --git a/docs/results.md b/docs/results.md index d6c6985a31..97fe28c4f1 100644 --- a/docs/results.md +++ b/docs/results.md @@ -67,6 +67,7 @@ Resubmitting computer-tool items as conversation input uses the raw Responses pa [`new_items`][agents.result.RunResultBase.new_items] gives you the richest view of what happened during the run. Common item types are: +- [`InputItem`][agents.items.InputItem] for input admitted from `RunState.pending_input` immediately before a resumed model call - [`MessageOutputItem`][agents.items.MessageOutputItem] for assistant messages - [`ReasoningItem`][agents.items.ReasoningItem] for reasoning items - [`ToolSearchCallItem`][agents.items.ToolSearchCallItem] and [`ToolSearchOutputItem`][agents.items.ToolSearchOutputItem] for Responses tool search requests and loaded tool-search results @@ -132,6 +133,24 @@ if result.interruptions: result = await Runner.run(agent, state) ``` +#### Add input before resuming + +Use [`RunState.add_input()`][agents.run_state.RunState.add_input] when new user input arrives after a run pauses or stops after a completed turn, but before the unfinished run reaches its next model call. A string becomes a user message, and multiple calls preserve insertion order. The staged input is part of serialized `RunState`, so it survives `to_json()` / `from_json()` and `to_string()` / `from_string()` round trips. + +```python +state = result.to_state() +state.add_input("Also keep the generated report in the project folder.") + +for interruption in state.get_interruptions(): + state.approve(interruption) + +result = await Runner.run(agent, state) +``` + +On resume, the runner applies both the current agent's input guardrails and the input guardrails from [`RunConfig`][agents.run.RunConfig] only to the staged input. When a client-managed [`Session`][agents.memory.session.Session] is configured, the runner converts the accepted staged input into a durable [`InputItem`][agents.items.InputItem] and awaits the session write before issuing the model request. Without a client-managed session or server-managed conversation, the runner converts the accepted staged input into an `InputItem` before issuing the model request. For a server-managed conversation, the input remains pending until the server request accepts it. Across serialization, resume, and replay-safe retries, the SDK preserves one durable `InputItem` occurrence. This SDK occurrence guarantee is not a provider-delivery guarantee: if a retry policy returns `RetryDecision(approve_unsafe_replay=True)` after a request may have reached the provider, the runner can resend the staged input and provider-side work can repeat. Successfully admitted input appears in `new_items` as an `InputItem`. Read [`RunState.pending_input`][agents.run_state.RunState.pending_input] for a detached copy, or call [`RunState.clear_pending_input()`][agents.run_state.RunState.clear_pending_input] to discard all staged input before resuming. + +`RunState.add_input()` rejects a terminal state, a state with no remaining model turns, a state in which an accepted model response is awaiting local processing, and an interrupted state whose pending tool result may end the run before another model call. In those cases, finish the current run and start a new user turn instead. + For streaming runs, finish consuming [`stream_events()`][agents.result.RunResultStreaming.stream_events] first, then inspect `result.interruptions` and resume from `result.to_state()`. For the full approval flow, see [Human-in-the-loop](human_in_the_loop.md). ### Server-managed continuation @@ -175,6 +194,13 @@ Python does not expose a separate streamed `completed` promise or `error` proper [`last_response_id`][agents.result.RunResultBase.last_response_id] is just the ID from the last entry in `raw_responses`. +Each [`ModelResponse`][agents.items.ModelResponse] also exposes two diagnostics that apply to that individual model call: + +- [`request_id`][agents.items.ModelResponse.request_id] is the transport request ID when the model adapter and transport propagate one. The built-in `OpenAIResponsesModel` and `OpenAIChatCompletionsModel` propagate an available server-generated `x-request-id` on their HTTP and SSE transport paths. When the configured endpoint is the OpenAI API, log a non-`None` value in production so you can correlate failures with OpenAI support; for an OpenAI-compatible provider or proxy, use that service's support channel instead. `OpenAIResponsesWSModel` currently leaves `request_id` as `None`. Third-party adapters do not guarantee request ID propagation. The AnyLLM Chat Completions adapter and `LitellmModel` currently leave `request_id` as `None`. The Agents SDK AnyLLM Responses adapter may also leave `request_id` as `None` when it normalizes a provider response without preserving the transport request ID. +- [`raw_usage`][agents.items.ModelResponse.raw_usage] is an opt-in, JSON-compatible snapshot of the provider's usage payload before the Agents SDK normalizes the payload. Enable `raw_usage` with `ModelSettings(preserve_raw_usage=True)`; see [Preserving provider usage payloads](usage.md#preserving-provider-usage-payloads). + +`ModelResponse.request_id` and `ModelResponse.raw_usage` can each be `None`, so handle these values as optional diagnostics rather than conversation state. + ### Guardrail results Agent-level guardrails are exposed as [`input_guardrail_results`][agents.result.RunResultBase.input_guardrail_results] and [`output_guardrail_results`][agents.result.RunResultBase.output_guardrail_results]. diff --git a/docs/running_agents.md b/docs/running_agents.md index 32bd335219..9dcf4d1f4a 100644 --- a/docs/running_agents.md +++ b/docs/running_agents.md @@ -29,7 +29,7 @@ When you call any of the three `Runner` methods above, you pass in a starting ag - a string (treated as a user message), - a list of input items in the OpenAI Responses API format, or -- a [`RunState`][agents.run_state.RunState] when resuming an interrupted run. +- a [`RunState`][agents.run_state.RunState] when resuming a paused run or a run stopped with `cancel(mode="after_turn")`. The state can also carry [input staged for the next resumed model call](results.md#add-input-before-resuming). The runner then runs a loop: diff --git a/docs/sandbox/clients.md b/docs/sandbox/clients.md index 7102eb917c..001ff33e0f 100644 --- a/docs/sandbox/clients.md +++ b/docs/sandbox/clients.md @@ -119,6 +119,24 @@ Hosted sandbox clients expose provider-specific mount strategies. Choose the bac +The mount tables describe which storage types each backend can execute. A check mark does not bypass the credential boundary for a mount helper that runs inside a model-controlled sandbox, and it does not mean that every strategy can operate without credentials. The Agents SDK accepts an in-container mount without an acknowledgement only when the selected helper can operate without protected authority. It rejects a mount that requires protected authority before starting the sandbox or mount helper unless trusted application code explicitly acknowledges the exposure for the exact mount path. + +Credentialless `rclone` mounts are limited to S3, GCS, R2, and Azure Blob. An in-container Box mount requires a non-interactive authentication source and the acknowledgement that matches that source. `FuseMountPattern` requires broad acknowledgement because `blobfuse2` discovers ambient Azure authority, even when no inline credential is configured. `S3FilesMountPattern` likewise requires broad acknowledgement because `mount.s3files` uses ambient IAM authority. These requirements also apply when Docker is the backend; the check marks below indicate that Docker can execute the mount after the applicable authority boundary is satisfied. + +For a mount entry named `"data"`, retain the copied `Manifest` returned by the acknowledgement that matches the configured authority: + +```python +# Mount-scoped values such as inline access keys. +manifest = manifest.with_in_container_mount_credential_exposure_acknowledged("data") + +# Broader authority such as managed or workload identity and external credential files. +manifest = manifest.with_in_container_mount_broad_credential_exposure_acknowledged("data") +``` + +Pass every exact mount path that needs the acknowledgement. A mount that uses both authority classes requires both acknowledgements. The acknowledgements are runtime-only, are not serialized, and permit the helper to receive credentials without confining credential use to the mounted path. Prefer an external or provider-native strategy when available, and otherwise use sandbox-scoped, short-lived, least-privilege credentials. + +`VercelSandboxClientOptions(allow_s3_credential_exposure=True)` remains a compatibility option for create-time Vercel S3 mounts with inline mount-scoped credentials. It does not authorize broad credential authority. + The table below summarizes which remote storage entries each backend can mount directly.
diff --git a/docs/sandbox/guide.md b/docs/sandbox/guide.md index 42733d630d..90abb88c35 100644 --- a/docs/sandbox/guide.md +++ b/docs/sandbox/guide.md @@ -649,6 +649,8 @@ Use this when sandbox state lives in your own storage or job system and you want Session-state serialization omits native `host_path` values. To resume host-backed grants, provide the current trusted manifest through `SandboxRunConfig.manifest` or `agent.default_manifest`; otherwise resume fails before the sandbox starts. Never derive host paths from serialized or other untrusted input. +Session-state and `RunState` serialization also remove cloud mount credentials, credential-bearing helper configuration, and in-container credential-exposure acknowledgements. For a backend that supports resuming mounted sessions, provide the current trusted manifest through `SandboxRunConfig.manifest` or `agent.default_manifest` when the state contains redacted mount authority. When the mount entry named `"data"` needs mount-scoped acknowledgement, retain the copied manifest with `trusted_manifest = trusted_manifest.with_in_container_mount_credential_exposure_acknowledged("data")` before resuming. Use `trusted_manifest = trusted_manifest.with_in_container_mount_broad_credential_exposure_acknowledged("data")` for broad authority, and call both methods when the mount uses both authority classes. Pass every exact mount path that needs an acknowledgement. The Agents SDK restores credentials only when the current trusted manifest has exactly the same credential-free mount topology as the persisted state. Missing or mismatched trusted configuration causes resume to fail before the sandbox starts; serialized state never grants authority by itself. `VercelSandboxClient` cannot resume a mounted session, so start a new sandbox with the trusted manifest instead. + ### Start from a snapshot Seed a new sandbox from saved files and artifacts: diff --git a/docs/sessions/index.md b/docs/sessions/index.md index b9acdaf9cd..dee68240b1 100644 --- a/docs/sessions/index.md +++ b/docs/sessions/index.md @@ -284,6 +284,8 @@ If your agent runs with `ModelSettings(store=False)`, the Responses API does not Compaction clears and rewrites the session history, so the SDK waits for compaction to finish before considering the run complete. In streaming mode, this means `run.stream_events()` can stay open for a few seconds after the last output token if compaction is heavy. +`OpenAIResponsesCompactionSession.run_compaction()` treats the clear-and-rewrite operation as a recoverable replacement at the wrapper boundary. If replacement fails or is cancelled after the underlying history changes, the wrapper attempts to restore the previous history and waits for that recovery attempt to settle before the original exception or cancellation reaches the caller. If the underlying backend also fails during recovery, the previous history can remain unrestored and the SDK logs the recovery failure. The wrapper serializes calls to `add_items()`, `pop_item()`, and `clear_session()` with the locked replacement and recovery phase, but a mutation can complete while the remote compaction request is still in flight and then be overwritten by successful replacement. Run manual compaction between turns without concurrent wrapper mutations, and do not mutate the underlying session directly while compaction is running. + If you want low-latency streaming or fast turn-taking, disable auto-compaction and call `run_compaction()` yourself between turns (or during idle time). You can decide when to force compaction based on your own criteria. ```python @@ -641,39 +643,38 @@ if __name__ == "__main__": ## Custom session implementations -You can implement your own session memory by creating a class that follows the [`Session`][agents.memory.session.Session] protocol: +You can implement your own session memory by creating a class that structurally follows the [`Session`][agents.memory.session.Session] protocol. You do not need to inherit from `SessionABC`; define `session_id` and `session_settings`, and implement the four history methods directly: ```python -from agents.memory.session import SessionABC +from agents import Agent, Runner, SessionSettings from agents.items import TResponseInputItem -from typing import List -class MyCustomSession(SessionABC): + +class MyCustomSession: """Custom session implementation following the Session protocol.""" - def __init__(self, session_id: str): + session_settings: SessionSettings | None = None + + def __init__(self, session_id: str) -> None: self.session_id = session_id - # Your initialization here + self.items: list[TResponseInputItem] = [] - async def get_items(self, limit: int | None = None) -> List[TResponseInputItem]: - """Retrieve conversation history for this session.""" - # Your implementation here - pass + async def get_items(self, limit: int | None = None) -> list[TResponseInputItem]: + if limit is None: + return list(self.items) + if limit <= 0: + return [] + return list(self.items[-limit:]) - async def add_items(self, items: List[TResponseInputItem]) -> None: - """Store new items for this session.""" - # Your implementation here - pass + async def add_items(self, items: list[TResponseInputItem]) -> None: + self.items.extend(items) async def pop_item(self) -> TResponseInputItem | None: - """Remove and return the most recent item from this session.""" - # Your implementation here - pass + return self.items.pop() if self.items else None async def clear_session(self) -> None: - """Clear all items for this session.""" - # Your implementation here - pass + self.items.clear() + # Use your custom session agent = Agent(name="Assistant") @@ -684,6 +685,47 @@ result = await Runner.run( ) ``` +### Accessing run context from a custom session + +The Agents SDK can pass the active [`RunContextWrapper`][agents.run_context.RunContextWrapper] to a custom session for tenant routing, authorization, or other app-specific storage decisions. For the Agents SDK to pass the wrapper, add an explicitly named, keyword-compatible `wrapper` parameter to all four history methods: + +```python +from typing import Any + +from agents import RunContextWrapper +from agents.items import TResponseInputItem + + +class ContextAwareSession: + async def get_items( + self, + limit: int | None = None, + *, + wrapper: RunContextWrapper[Any] | None = None, + ) -> list[TResponseInputItem]: ... + + async def add_items( + self, + items: list[TResponseInputItem], + *, + wrapper: RunContextWrapper[Any] | None = None, + ) -> None: ... + + async def pop_item( + self, + *, + wrapper: RunContextWrapper[Any] | None = None, + ) -> TResponseInputItem | None: ... + + async def clear_session( + self, + *, + wrapper: RunContextWrapper[Any] | None = None, + ) -> None: ... +``` + +The Agents SDK enables this integration only when `get_items`, `add_items`, `pop_item`, and `clear_session` all declare `wrapper`. A generic `**kwargs` parameter does not satisfy this signature check. Existing session implementations that omit `wrapper` keep their released call shape and continue to work without changes. + ## Community session implementations The community has developed additional session implementations: diff --git a/docs/streaming.md b/docs/streaming.md index f4abec97cc..57632d6cdc 100644 --- a/docs/streaming.md +++ b/docs/streaming.md @@ -62,6 +62,8 @@ If you need to stop a streaming run in the middle, call [`result.cancel()`][agen A streamed run is not complete until `result.stream_events()` finishes. The SDK may still be persisting session items, finalizing approval state, or compacting history after the last visible token. If you are manually continuing from [`result.to_input_list(mode="normalized")`][agents.result.RunResultBase.to_input_list], and `cancel(mode="after_turn")` stops after a tool turn, rerun `result.last_agent` with that normalized input to continue the unfinished existing user turn instead of appending a fresh user turn right away. + +- If new user input arrives before that unfinished run resumes, convert the drained result with `result.to_state()`, call [`state.add_input(...)`][agents.run_state.RunState.add_input], and resume from the state. The runner admits the staged input immediately before the next model call; see [Add input before resuming](results.md#add-input-before-resuming). - If a streamed run stopped for tool approval, do not treat that as a new turn. Finish draining the stream, inspect `result.interruptions`, and resume from `result.to_state()` instead. - Use [`RunConfig.session_input_callback`][agents.run.RunConfig.session_input_callback] to customize how retrieved session history and the new user input are merged before the next model call. If you rewrite new-turn items there, the rewritten version is what gets persisted for that turn. diff --git a/docs/usage.md b/docs/usage.md index fbeb68e2b7..2d4f1d1987 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -11,6 +11,7 @@ The Agents SDK automatically tracks token usage for every run. You can access it - **request_usage_entries**: list of per-request usage breakdowns - **details**: - `input_tokens_details.cached_tokens` + - `input_tokens_details.cache_write_tokens` - `output_tokens_details.reasoning_tokens` ## Accessing usage from a run @@ -49,6 +50,29 @@ for i, request in enumerate(result.context_wrapper.usage.request_usage_entries): print(f"Request {i + 1}: {request.input_tokens} in, {request.output_tokens} out") ``` +## Preserving provider usage payloads + +The Agents SDK normalizes provider usage into [`Usage`][agents.usage.Usage] fields that provide consistent totals across model providers. Set [`ModelSettings.preserve_raw_usage`][agents.model_settings.ModelSettings.preserve_raw_usage] to `True` when an application must retain provider-specific usage fields or distinguish an omitted field from a provider-reported zero: + +```python +from agents import Agent, ModelSettings, Runner + +agent = Agent( + name="Assistant", + model_settings=ModelSettings(preserve_raw_usage=True), +) +result = await Runner.run(agent, "What's the weather in Tokyo?") + +for response in result.raw_responses: + print(response.raw_usage) +``` + +The Agents SDK stores each [`ModelResponse.raw_usage`][agents.items.ModelResponse.raw_usage] value as a detached, JSON-compatible snapshot of the provider payload for that model call. The Agents SDK does not aggregate `raw_usage` across the run. The value remains `None` when preservation is disabled, the provider returns no usage payload, or an upstream adapter has already discarded the original field-presence information. + +`preserve_raw_usage` preserves only a usage payload that reaches the model adapter; the setting does not request usage from the provider. When a streaming Chat Completions provider requires an explicit usage request, also set `ModelSettings(include_usage=True)`. + +`LitellmModel` does not currently populate `ModelResponse.raw_usage` in either streaming or non-streaming runs, so `preserve_raw_usage=True` has no effect with that adapter. Continue to use the normalized [`Usage`][agents.usage.Usage] fields when using `LitellmModel`, or choose an adapter that supports raw usage preservation when provider-specific field presence is required. + ## Accessing usage with sessions When you use a `Session` (e.g., `SQLiteSession`), each call to `Runner.run(...)` returns usage for that specific run. Sessions maintain conversation history for context, but each run's usage is independent.