feat(vercel): add Stagehand code-mode MCP example - #2626
Conversation
|
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
5 issues found and verified against the latest diff
Confidence score: 2/5
- In
packages/integrations/examples/vercel/src/agent.ts(runStagehandAgent), passing the full parentprocess.envinto model-executed JS exposes unrelated secrets to accidental or malicious exfiltration, which is the highest-impact risk here — restrict child env to a minimal allowlist of required variables. - In
.github/workflows/codemode-framework-examples.yml, running browser smoke jobs for external fork PRs without a maintainer opt-in can consume hosted-runner capacity and invite avoidable CI abuse — add the samesafe-to-test/same-repo gate used in other workflows. - In
packages/integrations/examples/vercel/src/e2e.tsandpackages/integrations/examples/vercel/src/smoke.ts, weak assertions can report PASS when behavior is actually wrong (page-count substring match and missing second-call title check), reducing signal from automated checks and allowing regressions through — tighten assertions to page/count context or structured output and verify title restoration on the second call. - In
packages/integrations/examples/vercel/src/agent.ts, failures before thetrypath can leave the stdio child process running when MCP setup/tool registration errors occur, causing process leaks and flaky local/CI runs over time — ensure child cleanup is guaranteed on all early-error paths.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/integrations/examples/vercel/src/e2e.ts">
<violation number="1" location="packages/integrations/examples/vercel/src/e2e.ts:20">
P2: The e2e can report PASS when the model returns the wrong page-count value because the assertion only searches for the substring `"2"`. Checking for `2` in a page/count context (or structured output) would make this validation meaningful.</violation>
</file>
<file name=".github/workflows/codemode-framework-examples.yml">
<violation number="1" location=".github/workflows/codemode-framework-examples.yml:36">
P2: External fork PRs will run this hosted-runner browser smoke job immediately, which can spend CI resources before a maintainer opt-in. Adding the same `safe-to-test`/same-repo gate used elsewhere would keep this workflow aligned with your existing fork-safety pattern.
(Based on your team's feedback about maintainer approval gates for external PR workflows.)</violation>
</file>
<file name="packages/integrations/examples/vercel/src/smoke.ts">
<violation number="1" location="packages/integrations/examples/vercel/src/smoke.ts:43">
P2: This smoke assertion can pass even when the active page is not restored across calls, because it validates second-call page count but not second-call title. Including the second-call title in the condition would better catch state-restoration regressions.</violation>
</file>
<file name="packages/integrations/examples/vercel/src/agent.ts">
<violation number="1" location="packages/integrations/examples/vercel/src/agent.ts:28">
P1: This passes the full parent environment into a process running model-generated JavaScript, so unrelated secrets become available for accidental or malicious exfiltration. A small allowlist of required variables (browser mode/credentials, PATH, etc.) would reduce that exposure.</violation>
<violation number="2" location="packages/integrations/examples/vercel/src/agent.ts:42">
P3: If MCP connection or tool registration fails, the spawned stdio child process isn't cleaned up. `createStagehandMcpBinding()` runs before the try in `runStagehandAgent`, so when `client.tools()` rejects the just-created client is never closed, leaking the child MCP process and any browser it launched. The official `@ai-sdk/mcp` usage guards this exact path by closing the client even on early failure. Consider closing the client inside `createStagehandMcpBinding` if `tools()` throws, and/or creating the binding within the try/finally so the error path also releases the client.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| transport: new Experimental_StdioMCPTransport({ | ||
| command: process.execPath, | ||
| args: [stdioServerPath], | ||
| env: definedEnvironment(), |
There was a problem hiding this comment.
P1: This passes the full parent environment into a process running model-generated JavaScript, so unrelated secrets become available for accidental or malicious exfiltration. A small allowlist of required variables (browser mode/credentials, PATH, etc.) would reduce that exposure.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/integrations/examples/vercel/src/agent.ts, line 28:
<comment>This passes the full parent environment into a process running model-generated JavaScript, so unrelated secrets become available for accidental or malicious exfiltration. A small allowlist of required variables (browser mode/credentials, PATH, etc.) would reduce that exposure.</comment>
<file context>
@@ -0,0 +1,66 @@
+ transport: new Experimental_StdioMCPTransport({
+ command: process.execPath,
+ args: [stdioServerPath],
+ env: definedEnvironment(),
+ }),
+ });
</file context>
| result.toolNames.length !== 2 || | ||
| result.toolNames.some((name) => name !== "code_execute") || | ||
| !result.text.includes("Example Domain") || | ||
| !result.text.includes("2") |
There was a problem hiding this comment.
P2: The e2e can report PASS when the model returns the wrong page-count value because the assertion only searches for the substring "2". Checking for 2 in a page/count context (or structured output) would make this validation meaningful.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/integrations/examples/vercel/src/e2e.ts, line 20:
<comment>The e2e can report PASS when the model returns the wrong page-count value because the assertion only searches for the substring `"2"`. Checking for `2` in a page/count context (or structured output) would make this validation meaningful.</comment>
<file context>
@@ -0,0 +1,25 @@
+ result.toolNames.length !== 2 ||
+ result.toolNames.some((name) => name !== "code_execute") ||
+ !result.text.includes("Example Domain") ||
+ !result.text.includes("2")
+) {
+ throw new Error(`Unexpected agent result: ${JSON.stringify(result)}`);
</file context>
| jobs: | ||
| vercel: | ||
| name: Vercel AI SDK | ||
| runs-on: ubuntu-latest |
There was a problem hiding this comment.
P2: External fork PRs will run this hosted-runner browser smoke job immediately, which can spend CI resources before a maintainer opt-in. Adding the same safe-to-test/same-repo gate used elsewhere would keep this workflow aligned with your existing fork-safety pattern.
(Based on your team's feedback about maintainer approval gates for external PR workflows.)
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At .github/workflows/codemode-framework-examples.yml, line 36:
<comment>External fork PRs will run this hosted-runner browser smoke job immediately, which can spend CI resources before a maintainer opt-in. Adding the same `safe-to-test`/same-repo gate used elsewhere would keep this workflow aligned with your existing fork-safety pattern.
(Based on your team's feedback about maintainer approval gates for external PR workflows.) </comment>
<file context>
@@ -0,0 +1,53 @@
+jobs:
+ vercel:
+ name: Vercel AI SDK
+ runs-on: ubuntu-latest
+ timeout-minutes: 20
+ steps:
</file context>
|
|
||
| const firstText = JSON.stringify(first); | ||
| const secondText = JSON.stringify(second); | ||
| if (!firstText.includes("Example Domain") || !secondText.includes('"pages":2')) { |
There was a problem hiding this comment.
P2: This smoke assertion can pass even when the active page is not restored across calls, because it validates second-call page count but not second-call title. Including the second-call title in the condition would better catch state-restoration regressions.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/integrations/examples/vercel/src/smoke.ts, line 43:
<comment>This smoke assertion can pass even when the active page is not restored across calls, because it validates second-call page count but not second-call title. Including the second-call title in the condition would better catch state-restoration regressions.</comment>
<file context>
@@ -0,0 +1,52 @@
+
+ const firstText = JSON.stringify(first);
+ const secondText = JSON.stringify(second);
+ if (!firstText.includes("Example Domain") || !secondText.includes('"pages":2')) {
+ throw new Error(`Expected browser state to persist across calls: ${firstText} ${secondText}`);
+ }
</file context>
| model: LanguageModel, | ||
| prompt: string, | ||
| ): Promise<StagehandAgentResult> { | ||
| const { client, tools } = await createStagehandMcpBinding(); |
There was a problem hiding this comment.
P3: If MCP connection or tool registration fails, the spawned stdio child process isn't cleaned up. createStagehandMcpBinding() runs before the try in runStagehandAgent, so when client.tools() rejects the just-created client is never closed, leaking the child MCP process and any browser it launched. The official @ai-sdk/mcp usage guards this exact path by closing the client even on early failure. Consider closing the client inside createStagehandMcpBinding if tools() throws, and/or creating the binding within the try/finally so the error path also releases the client.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/integrations/examples/vercel/src/agent.ts, line 42:
<comment>If MCP connection or tool registration fails, the spawned stdio child process isn't cleaned up. `createStagehandMcpBinding()` runs before the try in `runStagehandAgent`, so when `client.tools()` rejects the just-created client is never closed, leaking the child MCP process and any browser it launched. The official `@ai-sdk/mcp` usage guards this exact path by closing the client even on early failure. Consider closing the client inside `createStagehandMcpBinding` if `tools()` throws, and/or creating the binding within the try/finally so the error path also releases the client.</comment>
<file context>
@@ -0,0 +1,66 @@
+ model: LanguageModel,
+ prompt: string,
+): Promise<StagehandAgentResult> {
+ const { client, tools } = await createStagehandMcpBinding();
+ try {
+ const result = await generateText({
</file context>
1 participant
Why
The code-mode MCP needs to be exercised through real agent frameworks before its package is published. This first consumer layer keeps the Vercel AI SDK example beside the private workspace package so it can test the exact MCP build under review.
Stack
code_execute, executor, local/Browserbase configuration, and runtime testsWhat changed
STAGEHAND_BROWSER=localandSTAGEHAND_BROWSER=browserbasethrough the inherited MCP environmentThe AI SDK stdio transport is currently marked experimental upstream. Keeping execution in the MCP child preserves the process boundary needed for owner-enforced recovery from non-cooperative generated JavaScript.
E2E Test Matrix
pnpm --filter @browserbasehq/stagehand-integrations-example-vercel typecheckpnpm --filter @browserbasehq/stagehand-integrations testSTAGEHAND_BROWSER=local{"status":"PASS","tools":["code_execute"],"statePersisted":true}code_executecalls and returnedExample Domainwith two open pages.code_executecalls and returnedExample Domainwith two open pages.Changeset
None. This adds a private example and test surface without changing a published package.