Skip to content

stagehand-python@4.2.0a0.dev1574

stagehand-python@4.2.0a0.dev1574: Harden Codex CUA facade isolation, evidence, and tool logging (#2907)
@miguelg719 miguelg719 tagged this 04 Oct 07:44 
## Stack

Top-of-stack child of #2906 (`evals/consolidation-17-gemini-cua`).
Review against that immediate parent, not `main`.

## Summary

Port the remaining focused Codex/Stagehand facade hardening from the
experimental
Codex worktree without replacing the consolidation stack's newer shared
runtime.

- Add opt-in facade JSONL tool logging: request/session IDs, actual
arguments/code,
paired starts/ends, timing, errors, bounded result previews and browser
readiness.
Never write logs to MCP stdout; redact known credentials and omit image
payloads.
- Escape Unicode line separators in text tool results, addressing the
stream
  parsing failure observed in the Hostelworld benchmark traces.
- Share isolated HOME/CODEX_HOME creation between the SDK example and
evals;
do not inherit operator plugins/config/thread context. Preserve
file-based auth.
- Abort unexpected MCP servers, await observation capture, and match
evidence by
tool-call ID so unrelated calls/missing final captures do not shift
screenshots.
- Add optional bounded SDK failure artifacts and keep binary payloads
out of
  telemetry without mutating original events.
- Document the difference between Codex event logs and facade server
logs, privacy
  limitations, authentication, and runnable logging commands.

This remains the existing Codex SDK harness with the Stagehand facade,
not a new
native OpenAI computer-use adapter. Preserve the parent's terminal
session-loss
semantics, shared facade API, HardBench dataset, verifier and usage
contracts.
The older experimental `state`/`nodeRepl`/`reset` surface and reconnect
implementation
are not copied wholesale into the newer shared runtime. The original
dirty
worktree remains untouched.

## Validation

- 220 focused tests pass across 23 files (core facade, Codex SDK, eval
adapters).
- 2 standalone Codex example tests pass.
- Built stdio regression verifies the JSONL file is created and contains
paired
  request IDs and error results while the MCP client remains responsive.
- Integration packages and eval ESM/CLI builds; core/Codex/example/evals
typechecks.
- Targeted lint: no errors; warnings remain in existing patterns and
diagnostic
  stringification. `git diff --check` passes.
- No paid model or HardBenchmark rerun; no benchmark score improvement
claimed.

## Operational notes

Logging remains opt-in. Logs/error artifacts may contain sensitive
model/page
content despite best-effort redaction and must be reviewed before
sharing.
Unexpected-server detection aborts observed calls; it is not a security
sandbox.
Keychain-only login is not copied into isolated profiles.

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Hardens Codex facade runs with isolated `HOME`/`CODEX_HOME` profiles,
opt-in tool logging, and evidence keyed to tool-call IDs so unmatched or
unrelated calls no longer misplace screenshots.

**New Features**
- Adds opt-in JSONL facade tool logging with request/session IDs,
arguments, timing, errors, and bounded result previews; logs go only to
file or stderr, never MCP stdout, with known credentials redacted and
image payloads omitted.
- Saves bounded SDK failure artifacts to
`STAGEHAND_CODEX_DIAGNOSTICS_DIR` or `EVAL_CODEX_DIAGNOSTICS_DIR`; a
failed diagnostic write never masks the original SDK failure.
- Shares isolated `HOME`/`CODEX_HOME` creation between the SDK example
and evals via a common helper in
`@browserbasehq/stagehand-integrations-codex-sdk`; only file-based
`auth.json` is copied, never plugins, config, stray `CODEX_*` env, or
inherited thread context, and the example now runs in a temporary
working directory and warns Codex to use only the mounted browser tools.

**Bug Fixes**
- Matches probe evidence by tool-call ID so missing final captures no
longer shift screenshots onto the wrong step.
- Aborts on unexpected MCP servers (detection only, not a sandbox).
- Escapes Unicode line separators in text tool results, fixing the
stream parsing failure seen in Hostelworld traces.

<sup>Written for commit bdc69f2a8fb9a1b2a9c66ef876f37d87dd37af5b.
Summary will update on new commits.</sup>

<a
href="https://cubic.dev/pr/browserbase/stagehand/pull/2907?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>

<!-- End of auto-generated description by cubic. -->

Current open stack: #2893 → #2894 → #2902 → #2903 → #2889 → #2904 →
#2905 → #2906 → #2907.
Assets 2
Loading