Skip to content

fix(run): report output guardrail results when a tripwire aborts the run - #4090

Merged
seratch merged 1 commit into
openai:mainfrom
Kaif10:fix/output-guardrail-results-on-tripwire
Aug 1, 2026
Merged

fix(run): report output guardrail results when a tripwire aborts the run#4090
seratch merged 1 commit into
openai:mainfrom
Kaif10:fix/output-guardrail-results-on-tripwire

Conversation

@Kaif10

@Kaif10 Kaif10 commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Summary

#4071 made input guardrails report the results that had already completed when a tripwire aborts the run. The output-guardrail sibling, directly below it in the same file, was left as it was — so the two exception types still report asymmetrically.

On an output tripwire, run_output_guardrails raises immediately and discards the local guardrail_results it had accumulated, run.py hardcodes output_guardrail_results=[] when building RunErrorDetails, and the streamed path only assigns streamed_result.output_guardrail_results after the guardrail task returns — so a tripwire leaves it at its previous, empty value. Even the tripping result is absent from run_data; it is reachable only via exc.guardrail_result.

Reproduction with two output guardrails ordered by an asyncio.Event barrier (passes, then trips), on main:

run()           run_data.output_guardrail_results = []
run_streamed()  run_data.output_guardrail_results = []
run_streamed()  result.output_guardrail_results   = []
run_sync()      run_data.output_guardrail_results = []

With this change all four report ['passes', 'trips'].

Fix

Mirrors #4071: run_output_guardrails takes a results_sink and records each result as it completes; run.py keeps a run-level accumulator and passes it to the three call sites, using it for RunErrorDetails; the streamed path publishes the completed results onto streamed_result in its except OutputGuardrailTripwireTriggered branch. Behaviour on the success path is unchanged.

Test plan

Five tests appended to tests/test_guardrails.py (the file #4071 used), reusing its _result_names helper and parameterised across run / run_streamed / run_sync: results reported on tripwire for each entry point, results still reported on success, and completed results reported when a guardrail raises a non-tripwire exception.

pytest tests/test_guardrails.py: 55 passed / 2 failed with the change; clean main baseline is 50 passed / the same 2 failed (test_blocking_guardrail_cancels_remaining_on_trigger and its _streaming variant — timing-sensitive, unrelated to this diff). pytest tests/models: 608 passed. ruff check / ruff format --check clean; mypy reports no issues on the changed files.

Verification-script note: .agents/skills/code-change-verification/scripts/run.sh could not run here because make is not installed on this machine, so I ran the underlying commands directly.

Issue number

None — noticed while reading #4071.

Checks

  • I've added new tests, if relevant
  • I've run .agents/skills/code-change-verification/scripts/run.sh — could not: make unavailable locally (see Test plan); ran the equivalent commands directly
  • I've confirmed all verification steps pass (format, lint, typecheck, affected tests; the 2 remaining failures are present on clean main)
  • If using Codex, I've run /review before submitting this PR — n/a

AI assistance was used in preparing this change; I reviewed it, reproduced the behaviour and verified the fix locally.

@seratch seratch added this to the 0.19.x milestone Aug 1, 2026
@seratch
seratch merged commit 855255a into openai:main Aug 1, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants