Mirrored from external contributor PR #3105 after approval by
@miguelg719.

Original author: @alanagoyal
Original PR: https://github.com/browserbase/stagehand/pull/3105
Approved source head SHA: `b48795840bce28f4c26f48b35183bb035a1ea519`

@alanagoyal, please continue any follow-up discussion on this mirrored
PR. When the external PR gets new commits, this same internal PR will be
marked stale until the latest external commit is approved and refreshed
here.

## Original description
# why

this is a first step toward #2841. today, turning a bug report into a
pinned fixture takes a manually built mirror. this adds a small recorder
so a page state can be saved locally and used in an `extract()` or
`observe()` regression.

# what changed

- added `fixture:record` in `packages/evals`: pass a url, output
directory, and optional playwright setup script
- saves `index.html` plus a manifest with the source url, capture time,
viewport, and limitations
- preserves computed styles, resolved links, and current form state;
removes scripts and remote assets and adds a restrictive csp
- rejects frames/open shadow dom and refuses to overwrite an existing
fixture
- documented capture, replay, refresh, and content review

before: reproduce against the live site or build a mirror by hand.
after: run setup to reach the relevant state, record it, and write an
assertion against the saved page.

this is scoped to static observations. it doesn't replay application
behavior, generate assertions, or migrate the old live-site tasks
mentioned in the issue, which aren't on current v4 main.

proposed follow ups:

1. migrate a few representative current evals to recorded fixtures
2. add a reusable bug-report-to-fixture-and-regression-test workflow

# test plan

- recorder/browser tests cover form state, visibility, links, blocked
resource loads, output protection, unsupported content, and the cli
setup workflow
- added a real stagehand extraction test using a reduced reproduction of
#2624, with a deterministic model adapter and no api keys
- added dom-sensitive validation through real `stagehand.observe()`
calls: source and fixture prompts match, hidden controls stay hidden,
the dynamic accessible name and edited form state survive, and the
returned xpath resolves to the intended button
- replay disables the original page and stylesheet; a negative control
removes captured visibility and confirms that stagehand sees the leaked
hidden control and the validation rejects the lossy fixture
- confirmed both source and recorded pages fail when the historical
schema-key bug is restored, then pass with the upstream fix; their
extraction prompts also match after normalizing element ids

the schema check validates one historical transport regression; the
synthetic checkout checks dom fidelity for the supported patterns above.
neither claims general site fidelity or model reasoning quality.

local checks:

- `just install`, `just build`, `just fmt`, `just check`, and `just
test` passed
- all 10 recorder/extraction/observation browser tests passed separately
- 1,031 eval unit tests, 750 python tests (1 existing skip), and the go
sdk/generator suites passed
- an existing sdk browser smoke test hit its 10-second timeout during
concurrent runs; it passed in isolation, then all typescript tasks
passed with `--concurrency=1` and the final `just test` passed using
those cached results. no timeout or assertion changes

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Adds a static observation task recorder in `packages/evals` so
`extract()` and `observe()` regressions can run against a saved local
page instead of the live site.

- `task:record` takes a URL, output directory, and optional Playwright
setup script, then writes `index.html` plus a manifest with source URL,
capture time, viewport, and limitations.
- Preserves computed styles, resolved links, and current form state;
strips scripts and remote assets and adds a restrictive CSP.
- Rejects frames and open shadow DOM; refuses to overwrite existing
tasks.
- Adds a real Stagehand extraction regression for the #2624 schema-key
bug using a deterministic model adapter, plus an `observe()` fidelity
test on a synthetic checkout page showing edited state, accessible
names, and CSS-hidden controls survive replay.
- Includes a negative control that detects loss of captured visibility
styles, and recording tests covering form state, visibility, links,
blocked resources, output protection, and CLI setup.

<sup>Written for commit 503c421e7b1a928cdbc64b3c5476323928c67ea7.
Summary will update on new commits.</sup>

<a
href="https://cubic.dev/pr/browserbase/stagehand/pull/3106?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<a
href="https://www.cubic.dev/action/auto-fix/pr/browserbase/stagehand/3106?returnTo=https%3A%2F%2Fgithub.com%2Fbrowserbase%2Fstagehand%2Fpull%2F3106&source=description"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/turn-on-auto-fix-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/turn-on-auto-fix-light.svg"><img
alt="Turn on auto-fix"
src="https://www.cubic.dev/buttons/turn-on-auto-fix-dark.svg"></picture></a>

<!-- End of auto-generated description by cubic. -->

<!-- external-contributor-pr:owned source-pr=3105
source-sha=b48795840bce28f4c26f48b35183bb035a1ea519 claimer=miguelg719
-->

---------

Co-authored-by: alanagoyal <alanaan@gmail.com>
Co-authored-by: miguel <miguelg71921@gmail.com>