feat(integrations): Eve (native defineTool) - #2666
Conversation
🦋 Changeset detectedLatest commit: 1f5ef15 The changes in this PR will be included in the next version bump. This PR includes changesets to release 5 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
There was a problem hiding this comment.
All reported issues were addressed across 12 files
Architecture diagram
sequenceDiagram
participant Dev as Developer CLI
participant Bridge as Local Bridge (HTTP + Auth)
participant Proxy as Bridge MCP Proxy
participant Facade as Facade Stdio Server
participant Tools as Stagehand Facade Tools
participant BB as Browserbase
participant Eve as Eve Agent
Note over Dev,Bridge: Bridge startup (CLI run `pnpm bridge`)
Dev->>Bridge: startFacadeBridge()
Bridge->>Facade: Spawn stdio child process
Note right of Facade: Env allowlist: STAGEHAND_* / BROWSERBASE_* only + PATH,<br/>STAGEHAND_BROWSER defaults to "browserbase"
Facade-->>Bridge: stdout: "Ready" marker
Bridge->>Bridge: Allocate ephemeral port + 32-byte bearer token
Bridge-->>Dev: Print STAGEHAND_FACADE_MCP_URL / STAGEHAND_FACADE_MCP_TOKEN
Note over Eve,Bridge: Eve connects to Stagehand facade
Eve->>Bridge: MCP POST /mcp (Authorization: Bearer <token>)
Bridge->>Bridge: timingSafeEqual token comparison
alt Invalid or missing credentials
Bridge-->>Eve: 401 Unauthorized
else Token valid
Bridge->>Proxy: Forward JSON-RPC request
Proxy->>Facade: tools/list via stdio
Facade-->>Proxy: Advertised facade tools
Proxy->>Proxy: Filter to allowlist (run, snapshot, screenshot)
Proxy-->>Eve: Discovered tools (stagehand__run, __snapshot, __screenshot)
end
Note over Eve,BB: Tool call (happy path)
Eve->>Bridge: callTool("run", {code})
Bridge->>Proxy: Forward callTool
Proxy->>Facade: callTool via stdio (600s timeout, progress relayed)
Facade->>Tools: Execute run/snapshot/screenshot
Tools->>BB: Browser automation (Browserbase backend)
BB-->>Tools: Browser result
Tools-->>Facade: Tool result
Facade-->>Proxy: CallToolResult
Proxy-->>Bridge: Result (progress notifications forwarded)
Bridge-->>Eve: Tool result
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
|
Merged the facade-core |
There was a problem hiding this comment.
5 issues found across 12 files (changes from recent commits).
Confidence score: 4/5
- The biggest risk is in
packages/integrations/examples/eve/agent/tools/run.ts: the newrunexecution path lacks focused mocked-tool tests for both dispatch branches and rejectedruncalls, so regressions in action routing or failure handling could break tool execution without being caught — add targeted branch + rejection coverage aroundrunTool.execute/runbehavior. - In
packages/integrations/examples/eve/tests/tools.test.ts, the “before opening a browser” assertion is currently indirect (it only checks rejection), so a future reorder could launch the browser before validation and still leave this test misleading — assert the no-browser side effect explicitly in the test. packages/integrations/examples/eve/agent/tools/snapshot.tsandpackages/integrations/examples/eve/agent/tools/screenshot.tsboth have behavioral gaps, which leaves option forwarding, session discard-on-error, and screenshot serialization (file-part shape/base64/MIME) exposed to silent regressions — add focused success and rejection tests for each tool path.packages/integrations/examples/eve/agent/tools/run.tsduplicates result serialization logic fromfacade/stdio-server.ts, creating drift risk where the same run output differs across facade surfaces after future fixes — extract and reuse a shared serialization utility.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/integrations/examples/eve/agent/tools/snapshot.ts">
<violation number="1" location="packages/integrations/examples/eve/agent/tools/snapshot.ts:13">
P3: Snapshot execution has no behavioral coverage: an option could stop reaching `tools.snapshot`, or a rejected snapshot could stop discarding its session, without failing current tests. Add focused success and rejection-path tests for `snapshotTool.execute` with mocked session helpers.
(Based on your team's feedback about unit tests for new behavior.)</violation>
</file>
<file name="packages/integrations/examples/eve/agent/tools/screenshot.ts">
<violation number="1" location="packages/integrations/examples/eve/agent/tools/screenshot.ts:23">
P3: Screenshot result serialization has no behavioral coverage, so malformed file-part shape or a regression in base64/MIME forwarding reaches the agent unnoticed. Add focused tests for successful screenshot output and its Eve model-content conversion.
(Based on your team's feedback about unit tests for new behavior.)</violation>
</file>
<file name="packages/integrations/examples/eve/tests/tools.test.ts">
<violation number="1" location="packages/integrations/examples/eve/tests/tools.test.ts:33">
P3: The test title claims validation happens 'before opening a browser', but the test only asserts that runTool.execute rejects. It works today only because parse() in run.ts runs before getFacadeTools() (which launches a Browserbase/local browser in session.ts). If that ordering regresses, this network-free unit test would start launching a real browser session and hang or fail, breaking the 'no network, browser, or API keys' property the README promises. Consider stubbing src/session.js's getFacadeTools (or asserting rejection via the schema directly for the invalid inputs) so the test no longer depends on rejection happening before browser acquisition.</violation>
</file>
<file name="packages/integrations/examples/eve/agent/tools/run.ts">
<violation number="1" location="packages/integrations/examples/eve/agent/tools/run.ts:13">
P3: The new execution path has no focused mocked-tool coverage, leaving code/actions dispatch and failure handling unprotected from regressions. Add tests for both branches and a rejected `run` call.
(Based on your team's feedback about unit tests for new behavior.)</violation>
<violation number="2" location="packages/integrations/examples/eve/agent/tools/run.ts:29">
P3: Result serialization is duplicated from `facade/stdio-server.ts`, so future fixes can produce different `run` output across facade surfaces. A shared facade utility would keep this fallback behavior aligned.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
There was a problem hiding this comment.
All reported issues were addressed across 8 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
|
Smoke bugs fixed: session creation now goes through |
Adds a readonly `sessionId` to `StagehandBrowser` for Browserbase-backed handles (`browserbase.launch` / `browserbase.connect`); undefined for local browsers. The id was already threaded internally through the worker init metadata — this only surfaces it. **Why:** integration examples that persist sessions for reconnect-after-restart (Eve native tools in #2666, managed deep agents in #2653) currently have to recover the id out-of-band — stamping a `userMetadata` marker at launch and querying `sessions.list` — because the handle doesn't expose it. With this, that workaround collapses to `browser.sessionId`. **Scope:** two files (`browser/index.ts`, `browser/factories.ts`) + changeset. No behavior change; purely additive surface. Gate: build, typecheck, 186/186 unit tests. **Port parity:** TS is the contract — Python/Go should mirror (`session_id` on the Python browser handle) in follow-ups; the Python managed-deepagents example has the same workaround to delete. <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Added a readonly `sessionId` to `StagehandBrowser` handles for Browserbase-backed browsers (`browserbase.launch` / `browserbase.connect`). It’s undefined for local browsers and makes reconnect-after-restart flows simpler. - **New Features** - Access the Browserbase session id via `browser.sessionId`; avoids metadata markers + `sessions.list`. - Purely additive API; no behavior changes. <sup>Written for commit 33bb054. Summary will update on new commits.</sup> <a href="https://cubic.dev/pr/browserbase/stagehand/pull/2672?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
…2673) Python port of #2672 (TS is the contract; this mirrors it exactly): readonly `session_id` property on `StagehandBrowser`, populated from the worker init metadata for Browserbase launch/connect, `None` for local browsers. Kills the out-of-band session-id recovery in the managed deep-agents example (#2653), same as #2672 does for the Eve example (#2666). Gate: ruff format/check, ty check, 457 passed / 1 skipped. <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Add a read-only session_id property to StagehandBrowser in `sdk-python`, populated from worker init metadata for Browserbase and None for local. This lets clients persist the Browserbase session ID for reconnects without out-of-band recovery. <sup>Written for commit dad895e. Summary will update on new commits.</sup> <a href="https://cubic.dev/pr/browserbase/stagehand/pull/2673?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
There was a problem hiding this comment.
1 issue found across 3 files (changes from recent commits).
Confidence score: 5/5
- In
packages/integrations/examples/eve/src/session.ts, the new persistence/reconnect flow aroundbrowser.sessionIdis untested, so regressions in restoring sessions across restarts could slip through and make the example intermittently fail or reconnect to the wrong state — add focused unit coverage for launch/persist/reconnect paths (including stale or missing session IDs).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/integrations/examples/eve/src/session.ts">
<violation number="1" location="packages/integrations/examples/eve/src/session.ts:103">
P3: The new session-persistence behavior introduced here (capturing `browser.sessionId` from `browserbase.launch`, persisting it, and reconnecting by that ID across restarts) has no unit coverage, even though the example already has a vitest suite. The session lifecycle is the most fragile part of this example (the PR notes init timeouts and stranded sessions during smoke tests), so encoding the expected capture/reconnect behavior in a couple of focused tests would help prevent regressions and document the intended contract. Consider extracting `createResources`' session-creation decisioning into an injectable/testable helper (or exporting it) and asserting that a launched handle's `sessionId` is persisted and that a failed reconnect clears the persisted ID before launching a fresh session.</violation>
</file>
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
|
All alerts resolved. Learn more about Socket for GitHub. This PR previously contained dependency changes with security issues that have been resolved, removed, or ignored. |
Bundlers that inline the SDK (nitro, eve dev) re-anchor import.meta.url, so the derived archive/extension-dir paths point at files the bundle does not carry. STAGEHAND_EXTENSION_ARCHIVE_PATH and STAGEHAND_EXTENSION_DIRECTORY_PATH let hosts supply the real paths.
…screenshot) with stdio MCP host Promotes the playwrightCompatRuntime and the run/snapshot/screenshot tool contract from the facade experiment into packages/integrations as product code: contract.ts carries the reference tool definitions verbatim (schemas, descriptions, error strings), tools.ts adapts StagehandFacadeTools to a live Stagehand handle with per-page snapshot state and a serialized call queue, and a new stagehand-facade stdio bin exposes the three tools over MCP with screenshots as image content blocks.
AI-SDK-based MCP clients (Eve, Vercel AI SDK) reject tool input schemas with a top-level oneOf, failing every run call client-side before it reaches the server. The code/actions exclusivity stays in the tool description and is enforced at runtime by CodeModeRunInputSchema.
Native consumers (Eve defineTool, Vercel AI SDK tools) need direct access to StagehandFacadeTools and the pinned contract, not just the stdio bin.
- widen credential redaction (Browserbase, Google, bearer tokens) - bound shutdown against an in-flight launch (5s race) - surface unsupported model names as StagehandFacadeConfigError - only forward integral jpeg quality to page.screenshot - fail waitForOutput fast when the host exits before ready
One system prompt shared by every host example instead of three hand-authored variants.
…mples packages/integrations/ becomes a grouping directory: core/ is the @browserbasehq/stagehand-integrations package, and integration examples (deepagents, eve, vercel-ai) sit beside it. Fixes eve dev's snapshot EINVAL: eve copies a workspace dependency's package root, which previously contained the eve example itself.
9010314 to
6ab64c8
Compare
…ol surface Rebased onto main post-v4-release (squashed; original review + smoke history on PR #2666).
83031ff to
88abbcc
Compare
The protocol commits on this branch updated the schema without running the Python generator; CI's generate.py --check caught the drift.
The Gemini 3.6 Flash protocol change on this branch made the Go-embedded extension stale (extensionpack --check and the wheel-smoke build both fail on it). The docs sdk-reference parity test times out at its 5s default on cold PR runners (observed 5090ms) — give the multi-language surface extraction a real budget.
There was a problem hiding this comment.
All reported issues were addressed across 1 file (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
- reconnect only discards the persisted session id on session-gone errors; transient failures rethrow instead of stranding a keep-alive session - unhealthy discard clears the session id so recovery can't loop on a wedged session - session file scoped by API-key hash instead of one machine-global path - instructions.md regenerated from FACADE_AGENT_INSTRUCTIONS and pinned by a drift test - dotenv declared; eve wrapper gets an error handler and Windows shell
There was a problem hiding this comment.
All reported issues were addressed across 8 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
There was a problem hiding this comment.
3 issues found across 1 file (changes from recent commits).
Confidence score: 2/5
- In
packages/integrations/eve/src/session.ts, clearingsuspectSessionIdbefore a replacement launch succeeds can let the same wedged session be reattached on the next recovery attempt, causing repeated recovery failure loops — keepsuspectSessionIdset until launch success is confirmed. - In
packages/integrations/eve/src/session.ts, a hangingreleaseSessionHTTP call can block the entire recovery flow before a fresh browser launches, turning a best-effort cleanup step into an outage amplifier — add a short abort timeout so recovery remains bounded. - In
packages/integrations/eve/src/session.ts, the new recovery lifecycle (suspectSessionId, bounded reconnect retry inconnectToSession, and best-effortreleaseSession) lacks behavioral coverage, so regressions in edge-case recovery paths may ship unnoticed — add targeted tests for failed launch, stalled release, and retry-limit scenarios.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/integrations/eve/src/session.ts">
<violation number="1" location="packages/integrations/eve/src/session.ts:105">
P2: The newly added recovery lifecycle — the `suspectSessionId` wedge flag, the bounded reconnect retry in `connectToSession`, and the best-effort `releaseSession` on the `no brick`/`no strand` recovery path — has no behavioral test coverage. The only session-related test (`tests/tools.test.ts`) asserts that `discardFacadeToolsIfUnhealthy` is a function. This is exactly the edge-case-heavy logic (wedge flag, retry-then-release, release-then-fresh-launch) that is easy to regress, and it governs whether a keep-alive Browserbase session gets released or stranded on billing. Adding a couple of unit tests covering the typical recovery path (suspect session → release + fresh launch) and the transient-failure retry path would encode the contract.</violation>
<violation number="2" location="packages/integrations/eve/src/session.ts:106">
P1: A failed replacement launch can reattach the same wedged session on the next recovery attempt. Retain `suspectSessionId` until a replacement launch succeeds, so failed release/launch attempts cannot re-enable the broken reconnect path.</violation>
<violation number="3" location="packages/integrations/eve/src/session.ts:162">
P2: A stalled release HTTP request blocks all tool recovery before the fresh browser can launch. Add a short abort timeout to this best-effort request so the recovery path remains bounded.</violation>
</file>
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
| // fresh. The persisted id is never cleared on failure paths — the fresh | ||
| // launch overwrites it via persistSessionId only after it succeeds. | ||
| await releaseSession(apiKey, browserbaseSessionId, baseUrl); | ||
| suspectSessionId = undefined; |
There was a problem hiding this comment.
P1: A failed replacement launch can reattach the same wedged session on the next recovery attempt. Retain suspectSessionId until a replacement launch succeeds, so failed release/launch attempts cannot re-enable the broken reconnect path.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/integrations/eve/src/session.ts, line 106:
<comment>A failed replacement launch can reattach the same wedged session on the next recovery attempt. Retain `suspectSessionId` until a replacement launch succeeds, so failed release/launch attempts cannot re-enable the broken reconnect path.</comment>
<file context>
@@ -86,11 +91,19 @@ async function createResources(): Promise<FacadeResources> {
+ // fresh. The persisted id is never cleared on failure paths — the fresh
+ // launch overwrites it via persistSessionId only after it succeeds.
+ await releaseSession(apiKey, browserbaseSessionId, baseUrl);
+ suspectSessionId = undefined;
}
</file context>
| await fetch(`${baseUrl ?? "https://api.browserbase.com"}/v1/sessions/${sessionId}`, { | ||
| method: "POST", | ||
| headers: { | ||
| "x-bb-api-key": apiKey, | ||
| "Content-Type": "application/json", | ||
| }, | ||
| body: JSON.stringify({ status: "REQUEST_RELEASE" }), | ||
| }); |
There was a problem hiding this comment.
P2: A stalled release HTTP request blocks all tool recovery before the fresh browser can launch. Add a short abort timeout to this best-effort request so the recovery path remains bounded.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/integrations/eve/src/session.ts, line 162:
<comment>A stalled release HTTP request blocks all tool recovery before the fresh browser can launch. Add a short abort timeout to this best-effort request so the recovery path remains bounded.</comment>
<file context>
@@ -115,24 +128,48 @@ async function connectToSession(
+ // billing after we abandon it. Failures are swallowed: the session may
+ // already be gone, and release must never block the fresh launch.
+ try {
+ await fetch(`${baseUrl ?? "https://api.browserbase.com"}/v1/sessions/${sessionId}`, {
+ method: "POST",
+ headers: {
</file context>
| await fetch(`${baseUrl ?? "https://api.browserbase.com"}/v1/sessions/${sessionId}`, { | |
| method: "POST", | |
| headers: { | |
| "x-bb-api-key": apiKey, | |
| "Content-Type": "application/json", | |
| }, | |
| body: JSON.stringify({ status: "REQUEST_RELEASE" }), | |
| }); | |
| await fetch(`${baseUrl ?? "https://api.browserbase.com"}/v1/sessions/${sessionId}`, { | |
| method: "POST", | |
| headers: { | |
| "x-bb-api-key": apiKey, | |
| "Content-Type": "application/json", | |
| }, | |
| body: JSON.stringify({ status: "REQUEST_RELEASE" }), | |
| signal: AbortSignal.timeout(5_000), | |
| }); |
| // it doesn't strand on billing (the "no strand" invariant), then launch | ||
| // fresh. The persisted id is never cleared on failure paths — the fresh | ||
| // launch overwrites it via persistSessionId only after it succeeds. | ||
| await releaseSession(apiKey, browserbaseSessionId, baseUrl); |
There was a problem hiding this comment.
P2: The newly added recovery lifecycle — the suspectSessionId wedge flag, the bounded reconnect retry in connectToSession, and the best-effort releaseSession on the no brick/no strand recovery path — has no behavioral test coverage. The only session-related test (tests/tools.test.ts) asserts that discardFacadeToolsIfUnhealthy is a function. This is exactly the edge-case-heavy logic (wedge flag, retry-then-release, release-then-fresh-launch) that is easy to regress, and it governs whether a keep-alive Browserbase session gets released or stranded on billing. Adding a couple of unit tests covering the typical recovery path (suspect session → release + fresh launch) and the transient-failure retry path would encode the contract.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/integrations/eve/src/session.ts, line 105:
<comment>The newly added recovery lifecycle — the `suspectSessionId` wedge flag, the bounded reconnect retry in `connectToSession`, and the best-effort `releaseSession` on the `no brick`/`no strand` recovery path — has no behavioral test coverage. The only session-related test (`tests/tools.test.ts`) asserts that `discardFacadeToolsIfUnhealthy` is a function. This is exactly the edge-case-heavy logic (wedge flag, retry-then-release, release-then-fresh-launch) that is easy to regress, and it governs whether a keep-alive Browserbase session gets released or stranded on billing. Adding a couple of unit tests covering the typical recovery path (suspect session → release + fresh launch) and the transient-failure retry path would encode the contract.</comment>
<file context>
@@ -86,11 +91,19 @@ async function createResources(): Promise<FacadeResources> {
+ // it doesn't strand on billing (the "no strand" invariant), then launch
+ // fresh. The persisted id is never cleared on failure paths — the fresh
+ // launch overwrites it via persistSessionId only after it succeeds.
+ await releaseSession(apiKey, browserbaseSessionId, baseUrl);
+ suspectSessionId = undefined;
}
</file context>
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
Eve example for the Stagehand facade tool surface, using Eve-native authored tools (
defineTool) instead of an MCP connection.run/snapshot/screenshot— the shared facade contract; descriptions, schemas, and system prompt imported from@browserbasehq/stagehand-integrations/facade, never restated.browser.sessionId).packages/integrations/eve/README.md.pnpm dev/pnpm startroute through a wrapper that forwards the SDK extension asset paths (needs feat(sdk-ts): env overrides for extension asset paths #2676's env overrides, merged into this branch).Verified: contract test in CI; Browserbase smoke passed (agent cited snapshot ID
[0-19], heading "Example Domain"); localeve devbrowser flow verified end-to-end.Stacked on #2665.