test: harden release compatibility contracts

[seratch]

This pull request adds a layered release-hardening system for v0.20.0. It expands the fast unit suite, verifies the actual wheel and source distribution, exercises historical RunState compatibility, compares streaming and non-streaming semantics, and adds adversarial checks around sandbox credentials and persisted state.

The goal is to catch compatibility, packaging, lifecycle, and credential-containment regressions at the boundary where they matter, while keeping the normal tests/ suite deterministic and fast.

Public API compatibility contracts

• Adds a machine-readable v0.19.4 public API manifest covering:
  • 247 top-level exports.
  • Public modules and supported submodule bindings.
  • Canonical import paths and object identity.
  • Exported classes and functions.
  • Constructor, function, and public method signatures.
  • Inherited SDK-owned public methods.
  • Curated documented properties.
  • Dataclass field order and defaults.
  • Optional-dependency and platform-specific imports.
• Allows compatible API additions while rejecting:
  • Removed or relocated exports.
  • Canonical identity changes.
  • Callable-kind changes.
  • Reordered or removed positional parameters.
  • Required parameter insertions.
  • Parameters inserted before released variadic arguments.
  • Incompatible dataclass field changes.
• Validates the existing contract before promoting newly added APIs, preventing the release update command from accidentally hiding a breaking change.
• Runs the same contract against the source checkout, installed wheel, installed source distribution, and optional-extra environments.
• Adds Python 3.10 and 3.14 packaged-contract CI coverage without credentials.

This gives future releases an explicit compatibility baseline. Newly released APIs can be added during local release preparation, while previously released behavior remains protected.

Historical RunState compatibility corpus

• Adds frozen fixtures for every readable schema version from 1.0 through 1.15.
• Generates fixtures from historical release writers instead of changing the version number on current output.
• Records fixture provenance so each payload can be traced to its original package or commit.
• Includes feature-bearing snapshots for:
  • Reasoning item IDs.
  • Resumed trace state.
  • Request IDs.
  • Tool-search metadata.
  • Approval rejection.
  • Duplicate agent identities and sandbox state.
  • Prompt cache keys.
  • Custom tool calls and tool origins.
  • max_turns=None.
  • Tool output custom data.
  • Input cache-write usage.
  • Programmatic tool calling and nested history ownership.
  • Hosted MCP approval scope.
  • Canonical invocation identity.
• Verifies the complete durable state across historical read, current-schema rewrite, and second read.
• Resumes historical pending approvals through Runner, covering both approval and rejection without duplicate tool execution or session history.
• Adds negative coverage for missing, malformed, and unsupported schema versions.
• Verifies that legacy credential-bearing sandbox state is sanitized, preserves non-authority topology, and requires a trusted rebind before reuse.
• Runs the same corpus against installed wheel and source-distribution artifacts.

This protects resume behavior against actual historical payloads and detects silent data loss that a current-writer-only round trip would miss.

Streaming and non-streaming symmetry

• Adds shared semantic projections for RunResult and RunResultStreaming.
• Compares:
  • Final output and last agent.
  • Generated items, tool names, call IDs, and outputs.
  • Interruptions.
  • Committed session history.
  • Complete usage data, including token details and per-request entries.
  • Trace and span outcomes.
  • Exception classes.
• Covers basic runs, function tools, failures, and approval resumes in both mode transitions:
  • Streaming pause to non-streaming resume.
  • Non-streaming pause to streaming resume.
• Extends live integration coverage for Responses function tools, structured output, SQLite persistence, and serialized approval resumes.
• Avoids assertions against exact model prose, token counts, or stream chunk boundaries.

This prevents streaming and non-streaming execution paths from drifting in billing, persistence, approval, error, or tracing behavior.

Packaged, release, nightly, and security profiles

• Expands the integration runner with explicit profiles for packaging, release, nightly, manual, and security verification.
• Builds wheel and source-distribution artifacts once and reuses them across isolated environments.
• Ensures installed tests import from site-packages, not the repository checkout.
• Adds source-distribution live smoke coverage for release and nightly runs.
• Treats missing required coverage as a failure in strict release and security profiles.
• Deselects intentionally inapplicable optional-extra cases so expected skips do not fail strict profiles.
• Keeps the existing 75-second per-test timeout and does not add automatic reruns that could hide provider or product failures.
• Writes separate JUnit reports per profile and environment.
• Validates declared JUnit counts against actual test cases and outcomes.
• Rejects malformed, truncated, duplicate-outcome, or otherwise impossible reports.
• Removes raw passing-test output and unsafe provider payloads from retained JUnit artifacts.
• Reports passed, failed, errored, skipped, and deselected counts by environment.

This separates fast pull-request feedback from slower artifact, provider, and sandbox verification while ensuring strict release gates cannot become false-green through skips or malformed reports.

Sandbox credential and error hardening

• Adds redaction coverage across:
  • Sandbox manifests and session state.
  • Generated mount commands and stderr.
  • Provider exceptions and exception groups.
  • Exception attributes, chains, traceback frames, and frame locals.
  • Logs and serialized traces.
  • Model-visible instructions, input, settings, tools, schemas, handoffs, continuation identifiers, and prompts.
  • Temporary configuration and persisted RunState.
• Rejects credential-bearing in-container mount fallback before provider side effects.
• Sanitizes legacy persisted authority and requires credentials to be rebound by trusted application code.
• Preserves non-secret structured sandbox diagnostics such as error type, error code, operation, and retryability.
• Preserves process-control behavior using fresh value-free CancelledError, KeyboardInterrupt, and SystemExit instances.
• Adds installed-artifact security contracts for wheel and source-distribution environments.
• Adds a local Docker adversarial test that attempts environment, filesystem, process, and model-request inspection while verifying that a trusted sentinel credential remains inaccessible.
• Requires the local sandbox security contract in strict release and security profiles.

This verifies credential containment at public runtime, persistence, packaging, and sandbox boundaries instead of relying only on sanitized error messages.

Compatibility considerations

No new runtime public symbols are introduced.

The RunState restoration path now validates exact built-in serialized structures without invoking caller-defined mapping behavior. An exact base RunContextWrapper returned by context_override or context_deserializer remains supported and preserves its identity and restored state. RunContextWrapper subclasses are rejected before their custom descriptors or private hooks can run; callers can return the custom context value directly or use the exact base wrapper.

Credential-bearing historical sandbox state remains readable, but authority is removed during migration and must be rebound by trusted application code before the sandbox can resume.

Release workflow

The released API manifest remains a local release-preparation operation:

make update-released-api-contract VERSION=0.20.0
make check-released-api-contract VERSION=0.20.0

The update command first verifies the existing released contract, then adds compatible APIs from the final candidate. After rebasing the release branch, rerun the check command to ensure the committed manifest still matches the candidate.

The primary verification entry points are:

make tests
make integration-tests-packaging
make integration-tests-mcp-v1
make integration-tests-security

The exact release candidate can then be verified with:

OPENAI_AGENTS_INTEGRATION_STRICT=1 \
OPENAI_AGENTS_INTEGRATION_EXTERNAL_PROVIDERS=1 \
OPENAI_AGENTS_INTEGRATION_DIRECT_PROVIDERS=0 \
make integration-tests-release

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5026567c1b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 38dab9dcbd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ab031791bf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4bc56281df

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1d03a15897

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f8f4f62633

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a6557fbfba

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6e8c024c31

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6006cf4631

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 04b775e0d0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3472d0f50a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0c1bcfdaea

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 861cdca6fb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d5126e33b9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: be58ead641

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

openai-agents-python/src/agents/sandbox/entries/mounts/patterns.py

Lines 125 to 131 in 93b82d2

	for value in sensitive_values:
	if not value:
	continue
	redacted = redacted.replace(value, "REDACTED")
	quoted = shlex.quote(value)
	if quoted != value:
	redacted = redacted.replace(quoted, "REDACTED")

Redact longer credential values before their prefixes

When one mount credential is a prefix of another, this replacement order can expose the longer value's suffix in a provider error. For example, redacting "abcdef" with the normal environment order ['abc', 'abcdef'] produces "REDACTEDdef"; MountpointMountPattern.apply() supplies access key, secret key, and session token in that order before applying this helper to stderr. Deduplicate and redact values longest-first so overlapping credentials cannot be partially retained in diagnostics.

AGENTS.md reference: AGENTS.md:L103-L103

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".