feat(core): support modelName "auto" when using the Stagehand API

[miguelg719]

Why

Passing model: "auto" failed at the constructor with:

ERROR: No known environment variable for provider 'auto'

followed by an UnsupportedModelError, because Stagehand always resolved a provider API key and built a local LLM client — even when all inference runs server-side through the Stagehand API.

What changed

model: "auto" is now accepted in the constructor and in per-primitive (act/extract/observe) model overrides when running through the Stagehand API (env: "BROWSERBASE", disableAPI: false, experimental: false). The API resolves "auto" server-side.

• modelUtils.ts: new AUTO_MODEL_NAME constant + isAutoModel() helper.
• v3.ts constructor: for "auto", skip provider API-key lookup and local client creation; throw a clear StagehandInvalidArgumentError when the API won't be used (LOCAL env, disableAPI: true, or experimental: true).
• v3.ts resolveLlmClient: per-call "auto" overrides no-op onto the default client in API mode and throw the same clear error otherwise.
• v3.ts act(): local act-cache replay is skipped for "auto" sessions — there is no local client for self-heal; the server-side cache still applies.
• v3.ts init(): if the API reports available: false, "auto" has no local fallback — the just-created session is ended best-effort and a clear StagehandInitError (with sessionId) is thrown instead of failing later with an undefined-client TypeError.
• api.ts prepareModelConfig: "auto" never inherits the default model's provider config or API key (the server picks the provider); explicit per-call options still pass through.

Agent/CUA mode is intentionally out of scope.

Known limits (intentional, kept surgical)

• middleware on a per-call { modelName: "auto" } override is inert — consistent with middleware being documented local-only and already inert in API mode for concrete models.
• The public llmClient!: LLMClient field is undefined for "auto" sessions; tightening that type ripples across the public surface and is left for a follow-up.

Tests

• New tests/unit/auto-model.test.ts (10 tests): constructor accept/reject matrix (string + object form), per-call resolution guard, and wire-level assertions that "auto" payloads carry no inherited API key.
• Full core unit suite: 869 passed; the 4 flowlogger-eventstore failures reproduce on clean main (pre-existing, unrelated).
• Typecheck, eslint, and prettier clean.

🤖 Generated with Claude Code

---

Summary by cubic

Enables model: "auto" in @browserbasehq/stagehand when using the Stagehand API, delegating model selection to the server and avoiding local provider keys. Outside API mode, "auto" now fails fast with clear errors.

• New Features
  • Accept "auto" in the constructor and per-call overrides only in API mode (env: "BROWSERBASE", disableAPI: false, experimental: false).
  • No local LLM client or provider API key; the API selects the provider and payloads don’t inherit API keys.
  • Clear errors: StagehandInvalidArgumentError when not in API mode or when a custom llmClient is supplied; StagehandInitError if the API is unavailable. On "auto" init failure, only end sessions created by this init and leave caller-supplied browserbaseSessionID sessions untouched.
  • Per-call "auto" reuses the default client in API mode; explicit options pass through. Local act-cache and agent-cache replay are disabled for "auto" sessions.

^{Written for commit 1f9b6aa. Summary will update on new commits.}

[changeset-bot]

🦋 Changeset detected

Latest commit: 1f9b6aa

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages

Name	Type
@browserbasehq/stagehand	Patch
@browserbasehq/stagehand-evals	Patch
@browserbasehq/stagehand-server-v3	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[cubic-dev-ai]

1 issue found across 5 files

Confidence score: 3/5

• In packages/core/lib/v3/v3.ts, the opts.llmClient path appears to bypass the isAutoModel guard, so model: "auto" may be accepted outside the Stagehand API and run with disableAPI = true; this could cause incorrect routing/behavior for callers expecting auto to be blocked in non-API contexts — enforce the isAutoModel check before the opts.llmClient branch (or re-check inside it) before merging.

Architecture diagram

sequenceDiagram
    participant Client as Consumer Code
    participant Stagehand as Stagehand V3
    participant LLMClient as Local LLM Client
    participant APIClient as Stagehand API Client
    participant RemoteAPI as Stagehand API Server
    participant ActCache as Local Act Cache

    Note over Stagehand,ActCache: Constructor Flow

    Client->>Stagehand: new Stagehand({ model: "auto", env: "BROWSERBASE" })
    Stagehand->>Stagehand: isAutoModel("auto")
    
    alt API Mode (env: "BROWSERBASE", disableAPI: false, experimental: false)
        Stagehand->>Stagehand: Skip provider API key lookup
        Stagehand->>Stagehand: Skip local LLM client creation
        Stagehand-->>Client: Instance created (llmClient = undefined)
    else Non-API Mode (LOCAL, disableAPI, or experimental)
        Stagehand-->>Client: throw StagehandInvalidArgumentError
    end

    Note over Stagehand,RemoteAPI: init() Flow

    Client->>Stagehand: init()
    Stagehand->>APIClient: startSession({ modelName: "auto" })
    APIClient->>RemoteAPI: POST /sessions (model: auto)
    RemoteAPI-->>APIClient: { sessionId, available: true }
    APIClient-->>Stagehand: Session started
    Stagehand->>Stagehand: isAutoModel(this.modelName) == true

    alt API Unavailable
        RemoteAPI-->>APIClient: { available: false }
        APIClient->>RemoteAPI: end() [best-effort]
        Stagehand-->>Client: throw StagehandInitError
    end

    Note over Client,RemoteAPI: Per-Call Model Override ("auto")

    Client->>Stagehand: act("instruction", { model: "auto" })
    Stagehand->>Stagehand: resolveLlmClient("auto")

    alt Has API Client
        Stagehand->>Stagehand: Return existing llmClient (or undefined)
        Note over Stagehand: Local client never dereferenced for "auto"
    else No API Client
        Stagehand-->>Client: throw StagehandInvalidArgumentError
    end

    Stagehand->>Stagehand: isAutoModel(this.modelName)

    alt "auto" Session
        Stagehand->>ActCache: Skip local cache replay (no LLM client)
        Note over Stagehand: Rely on server-side cache instead
    else Concrete Model Session
        Stagehand->>ActCache: Normal cache replay
    end

    Stagehand->>APIClient: act({ options: { model: "auto" } })
    APIClient->>APIClient: prepareModelConfig("auto")
    APIClient->>APIClient: Return clean config (no provider/apiKey)
    APIClient->>RemoteAPI: POST /act (model: "auto", no apiKey)
    RemoteAPI-->>APIClient: Result
    APIClient-->>Stagehand: Result
    Stagehand-->>Client: Result

    Note over Client,RemoteAPI: Explicit Per-Call Options with "auto"

    Client->>Stagehand: extract("instruction", { model: { modelName: "auto", temperature: 0.5 } })
    Stagehand->>APIClient: extract({ options: { model: { modelName: "auto", temperature: 0.5 } } })
    APIClient->>APIClient: prepareModelConfig({ modelName: "auto", temperature: 0.5 })
    APIClient->>APIClient: Return explicit options (no inherited apiKey)
    APIClient->>RemoteAPI: POST /extract (model: "auto", temperature: 0.5)
    RemoteAPI-->>APIClient: Result
    APIClient-->>Stagehand: Result
    Stagehand-->>Client: Result

Loading

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[seanmcguire12]

@cubic-dev-ai

[cubic-dev-ai]

@cubic-dev-ai

@seanmcguire12 I have started the AI code review. It will take a few minutes to complete.

[cubic-dev-ai]

1 issue found across 5 files

Confidence score: 3/5

• In packages/core/lib/v3/v3.ts, auto sessions using cacheDir may replay AgentCache steps locally even when no LLM client is available, and selector drift then prevents self-healing; this can leave users with partially applied, unintended page changes before the API fallback path runs. Gate or disable local replay when an LLM client is absent (or force an immediate fallback before any cached step executes) before merging.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/core/lib/v3/v3.ts">

<violation number="1" location="packages/core/lib/v3/v3.ts:1326">
P2: Auto sessions with `cacheDir` can still replay an `AgentCache` entry locally without an LLM client; selector changes make self-heal fail, and prior replayed steps may already have changed the page before the API fallback. Apply the same auto cache exclusion to agents or reject agent mode with a clear API-only error.</violation>
</file>

Architecture diagram

sequenceDiagram
    participant Client as User Code (SDK Consumer)
    participant Stagehand as V3 (Stagehand SDK)
    participant ModelUtils as modelUtils.ts
    participant API as StagehandAPIClient
    participant RemoteAPI as Stagehand API Server
    participant Env as Environment Variables

    Note over Client,Env: Model "auto" Delegation Flow (API Mode)

    Client->>Stagehand: new Stagehand({ model: "auto", env: "BROWSERBASE" })
    Stagehand->>ModelUtils: isAutoModel("auto")
    ModelUtils-->>Stagehand: true
    alt API Mode Check
        Stagehand->>Stagehand: Validate env="BROWSERBASE", disableAPI=false, experimental=false, no llmClient
        Stagehand->>Stagehand: Skip provider API key lookup
        Stagehand->>Stagehand: Skip local LLM client creation
        Stagehand-->>Client: Stagehand instance (llmClient = undefined)
    else Not API Mode
        Stagehand->>Stagehand: Throw StagehandInvalidArgumentError
        Stagehand-->>Client: Error: "auto only supported in API mode"
    end

    Note over Client,RemoteAPI: Session Initialization

    Client->>Stagehand: await stagehand.init()
    Stagehand->>API: init({ modelName: "auto" })
    API->>RemoteAPI: POST /init (model: auto, no apiKey)
    RemoteAPI-->>API: { sessionId, available: true/false }
    alt API Unavailable
        API-->>Stagehand: available: false
        Stagehand->>API: end() (best-effort cleanup)
        Stagehand->>Stagehand: Throw StagehandInitError
        Stagehand-->>Client: Error: "API unavailable for auto model"
    else API Available
        API-->>Stagehand: available: true
        Stagehand-->>Client: Initialized session
    end

    Note over Client,RemoteAPI: Per-Primitive Call (act with model "auto")

    Client->>Stagehand: act({ model: "auto" })
    Stagehand->>ModelUtils: isAutoModel("auto")
    ModelUtils-->>Stagehand: true
    Stagehand->>Stagehand: Skip local act-cache replay (no LLM client for self-heal)
    Stagehand->>Stagehand: resolveLlmClient("auto")
    Stagehand->>Stagehand: Validate apiClient exists
    Stagehand-->>Client: return this.llmClient (undefined for auto session)
    Stagehand->>API: act({ options: { model: "auto" } })
    API->>API: prepareModelConfig({ modelName: "auto" })
    API->>API: Skip default model provider config and API key inheritance
    API->>RemoteAPI: POST /act (model: { modelName: "auto" }, no apiKey)
    RemoteAPI-->>API: { result }
    API-->>Stagehand: Act result
    Stagehand-->>Client: Result

    Note over Client,RemoteAPI: Per-Primitive Call with Explicit Options

    Client->>Stagehand: observe({ model: { modelName: "auto", temperature: 0.5 } })
    Stagehand->>API: observe({ options: { model: { modelName: "auto", temperature: 0.5 } } })
    API->>API: prepareModelConfig({ modelName: "auto", temperature: 0.5 })
    API->>API: Pass through explicit options (temperature), skip API key
    API->>RemoteAPI: POST /observe (model: { modelName: "auto", temperature: 0.5 })
    RemoteAPI-->>API: { result }
    API-->>Stagehand: Observe result
    Stagehand-->>Client: Result

Loading

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P2: Auto sessions with cacheDir can still replay an AgentCache entry locally without an LLM client; selector changes make self-heal fail, and prior replayed steps may already have changed the page before the API fallback. Apply the same auto cache exclusion to agents or reject agent mode with a clear API-only error.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/core/lib/v3/v3.ts, line 1326:

<comment>Auto sessions with `cacheDir` can still replay an `AgentCache` entry locally without an LLM client; selector changes make self-heal fail, and prior replayed steps may already have changed the page before the API fallback. Apply the same auto cache exclusion to agents or reject agent mode with a clear API-only error.</comment>

<file context>
@@ -1289,7 +1320,10 @@ export class V3 {
+        this.actCache.enabled &&
+        // "auto" sessions have no local LLM client for replay self-heal;
+        // rely on the API (and its server-side cache) instead.
+        !isAutoModel(this.modelName);
       if (canUseCache) {
         actCacheContext = await this.actCache.prepareContext(
</file context>

[miguelg719]

Fixed in e187303 — AgentCache.shouldAttemptCache() now returns false for "auto" sessions (single gate covering all three replay call sites), mirroring the act-cache exclusion; the API agent (and its server-side cache) handles those runs instead. Added regression tests for both the auto and concrete-model cases.

[cubic-dev-ai]

The parent comment is addressed here: AgentCache.shouldAttemptCache() now excludes "auto" sessions, so local replay won’t run and the API/server-side cache handles those runs instead. Thanks for the fix and the regression coverage.

Thanks for the feedback! I've saved this as a new learning to improve future reviews.

[miguelg719]

@cubic-dev-ai

[cubic-dev-ai]

@cubic-dev-ai

@miguelg719 I have started the AI code review. It will take a few minutes to complete.

[cubic-dev-ai]

All reported issues were addressed across 6 files

Confidence score: 5/5

• Safe to merge after the addressed issues were fixed.

Architecture diagram

sequenceDiagram
    participant Client
    participant V3 as V3 (Stagehand)
    participant ModelUtils as modelUtils
    participant APIClient as StagehandAPIClient
    participant ActCache as actCache
    participant AgentCache as AgentCache
    participant API as Stagehand API (server)

    Note over Client,API: Constructor: modelName "auto"

    Client->>V3: new Stagehand({ model: "auto", env: "BROWSERBASE" })
    V3->>ModelUtils: isAutoModel("auto")
    ModelUtils-->>V3: true
    V3->>V3: Check env, disableAPI, experimental, llmClient
    alt valid API mode
        V3->>V3: Skip provider API key lookup
        V3->>V3: Skip LLM client creation
        V3-->>Client: Stagehand instance (llmClient = undefined)
    else LOCAL / disableAPI / experimental / custom llmClient
        V3-->>Client: throw StagehandInvalidArgumentError
    end

    Note over Client,API: init() with API availability check

    Client->>V3: init()
    V3->>APIClient: init({ modelName: "auto" })
    APIClient->>API: POST /sessions (modelName: "auto", no apiKey)
    API-->>APIClient: { sessionId, available: true }
    APIClient-->>V3: sessionId
    alt API reports available: false
        API-->>APIClient: { available: false }
        V3->>APIClient: end() best-effort
        V3-->>Client: throw StagehandInitError with sessionId
    end

    Note over Client,API: Per-call act/extract/observe

    Client->>V3: act(input, { model: "auto" })
    V3->>ModelUtils: isAutoModel("auto")
    ModelUtils-->>V3: true
    V3->>V3: resolveLlmClient("auto")
    alt has apiClient
        V3-->>V3: return this.llmClient (undefined for auto)
    else no apiClient
        V3-->>Client: throw StagehandInvalidArgumentError
    end
    V3->>ActCache: check enabled (skipped for auto)
    ActCache-->>V3: skip (no local client for self-heal)
    V3->>APIClient: act({ input, options: { model: { modelName: "auto" } } })
    APIClient->>APIClient: prepareModelConfig("auto")
    APIClient->>APIClient: return { modelName: "auto" } (no inherited apiKey)
    APIClient->>API: POST /act (modelName: "auto")
    API-->>APIClient: result (server-side model selection)
    APIClient-->>V3: result
    V3-->>Client: result

    Note over Client,API: Agent cache interaction

    Client->>V3: agent instruction
    V3->>AgentCache: shouldAttemptCache(instruction)
    AgentCache->>ModelUtils: isAutoModel(baseModelName)
    ModelUtils-->>AgentCache: true
    AgentCache-->>V3: false (skip local replay)

    Note over V3,API: Per-call model with explicit options

    Client->>V3: observe(instruction, { model: { modelName: "auto", temperature: 0.5 } })
    V3->>APIClient: observe({ options: { model } })
    APIClient->>APIClient: prepareModelConfig({ modelName: "auto", temperature: 0.5 })
    APIClient->>APIClient: return { modelName: "auto", temperature: 0.5 } (no apiKey)
    APIClient->>API: POST /observe (modelName: "auto", temperature: 0.5)
    API-->>APIClient: result
    APIClient-->>V3: result
    V3-->>Client: result

    Note over Client,API: Logging with undefined llmClient

    Client->>V3: act/extract/observe (any)
    V3->>V3: log model
    V3->>V3: this.llmClient?.modelName ?? this.modelName
    V3-->>V3: logs "auto" instead of crashing

Loading

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>