evals: document the stagehand_code exposure surface and batching guidance

[shriyatheunicorn]

Gives the stagehand_code exposure prompt an accurate, closed API contract and batching guidance.

With explicit contracts, there is a strong reduction in median turns per task.

Follow-up once #2608 lands: a short section introducing _experimental_batch usage.

---

Summary by cubic

Documents a closed API contract for stagehand_code by listing the exact page and page.locator methods and routing anything else through stagehand.act. Adds “always await” guidance, a recommended first step (page.goto(startUrl, { waitUntil: 'domcontentloaded' })), and batching advice; updates tests to assert the expanded surface (e.g., evaluate, scroll, setViewportSize, waitForSelector/waitForTimeout, and locator count/isVisible/textContent/inputValue).

^{Written for commit f981de5. Summary will update on new commits.}

[changeset-bot]

⚠️ No Changeset found

Latest commit: f981de5

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[cubic-dev-ai]

All reported issues were addressed across 2 files

Architecture diagram

sequenceDiagram
    participant Agent as Coding Agent (LLM)
    participant Prompt as Prompt Instructions
    participant RunTool as run_snippet tool
    participant StagehandClient as Stagehand Client (stagehand)
    participant Page as Page Object
    participant Locator as page.locator()
    participant Backend as Browser Backend (CDP)

    Note over Agent,Backend: NEW: Tightened API surface & batching guidance

    Agent->>Prompt: Reads closed contract (page & locator methods)
    Prompt-->>Agent: Exact method list + batching guidance

    Agent->>RunTool: Invoke run() with snippet
    Note over Agent,RunTool: NEW: Prefer chaining multiple steps in one call

    RunTool->>StagehandClient: Execute snippet
    StagehandClient->>Page: Access page methods

    alt Deterministic Page Methods (new explicit list)
        Page->>Page: goto(), reload(), back(), forward()
        Page->>Page: url(), title()
        Page->>Page: screenshot(), setViewportSize()
        Page->>Page: waitForSelector(), waitForTimeout()
        Page->>Backend: async RPC to browser
        Backend-->>Page: Result
    else Locator Methods (new explicit list)
        Page->>Locator: .locator(selector)
        Locator->>Locator: count(), click(), hover()
        Locator->>Locator: fill(value), type(text)
        Locator->>Locator: isVisible(), textContent(), inputValue()
        Locator->>Backend: async RPC to browser
        Backend-->>Locator: Result
    else Unlisted Behavior
        Page->>StagehandClient: Fallback to stagehand.act()
        StagehandClient->>StagehandClient: act('describe the action')
    end

    StagehandClient-->>RunTool: Return final result (JSON-serializable)
    Note over RunTool,Agent: NEW: Return useful JSON for progress inspection

    alt Batching (NEW guidance)
        Agent->>RunTool: Chain act/observe/extract + page steps in one snippet
        RunTool->>StagehandClient: Execute batched sequence
        StagehandClient-->>RunTool: Return aggregated result
        RunTool-->>Agent: Single response with full outcome
    else Traditional (step-by-step)
        Agent->>RunTool: One action per call
        RunTool->>StagehandClient: Execute single step
        StagehandClient-->>RunTool: Partial result
        RunTool-->>Agent: Intermediate state
        Agent->>RunTool: Next action
    end

    Note over Agent,Backend: Batching reduces median turns per task

Loading

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>