Recover server-v4 drizzle stack onto main

[monadoid]

Why

The original server-v4 stacked PRs merged into each other after STG-1614 had already merged into main, so the env/test-harness work, Drizzle foundation, and Drizzle schema/controller flow never actually reached main.

This PR recovers that full stack onto current main in one branch.

What Changed

• cherry-picked the three stacked squash-merge commits from #1896, #1897, and #1898 onto fresh main
• preserved the full server-v4 env/test harness cleanup, Drizzle foundation, DB smoke tests, schema/migration work, and /v4/llms controller/service/repo flow
• added one small follow-up commit to normalize pnpm-lock.yaml against current main after applying the recovered stack

Test Plan

• pnpm install
• pnpm --filter @browserbasehq/stagehand-server-v4 run lint
• pnpm --filter @browserbasehq/stagehand-server-v4 run test:unit
• pnpm --filter @browserbasehq/stagehand-server-v4 run build
• pnpm --filter @browserbasehq/stagehand-server-v4 run gen:openapi

---

Summary by cubic

Recovers the server‑v4 Drizzle stack onto main, restoring DB‑backed /v4/llms and switching the public LLMId to UUID. Adds a typed env, clean app bootstrap, and a local pglite/Postgres runtime with migrations, tests, and a simpler source‑based test harness.

• New Features

  • Drizzle foundation with pglite default and Postgres option; new db:* scripts via drizzle-kit.
  • Fastify databasePlugin and unified DB client; app.ts bootstrap with health/readiness and Swagger/OpenAPI; env via @t3-oss/env-core.
  • Tables/relations for LLMs and Stagehand; DB‑derived Zod schemas; first migration; OpenAPI updated for UUID LLMId with strict format and pattern.
  • LLM module (repo/service/controller) backing /v4/llms; routes now delegate to the controller; browser session create uses the service and auto‑creates a system default LLM when llmId is missing.
  • Tests run from source via tsx; test server script simplified; turbo tasks cleaned up; schema cleanup (z.url()).

• Migration

  • Run: pnpm --filter @browserbasehq/stagehand-server-v4 db:migrate.
  • Local dev uses pglite at ~/.stagehand/db/stagehand-v4. For Postgres set STAGEHAND_DB_MODE=postgres and DATABASE_URL.
  • Update clients to treat LLMId as a UUID and regenerate from openapi.v4.yaml if needed.

^{Written for commit 1fb861e. Summary will update on new commits. Review in cubic}

[changeset-bot]

⚠️ No Changeset found

Latest commit: 1fb861e

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[cubic-dev-ai]

6 issues found across 44 files

Confidence score: 3/5

• There is moderate merge risk because multiple medium-severity (6/10) issues have concrete runtime impact: packages/server-v4/src/db/plugin.ts can leak an open DB client on startup-check failure, and packages/server-v4/src/routes/v4/browsersession/index.ts can create orphan system-default rows when secondary LLM validation fails.
• turbo.json introduces regression risk in CI/dev workflows since test:server no longer guarantees dist/tests is built for server-v3 scripts; adding the build:esm-tests dependency should stabilize test execution.
• Lower-severity items in packages/server-v4/test/unit/db/databaseClient.test.ts and the 404 schema mismatch in packages/server-v4/src/routes/v4/browsersession/index.ts are cleanup/contract issues, but they still add noise and can mask failures if left unresolved.
• Pay close attention to packages/server-v4/src/db/plugin.ts, packages/server-v4/src/routes/v4/browsersession/index.ts, and turbo.json - they carry the main resource-leak, data-integrity, and test-pipeline regression risks.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/server-v4/src/db/plugin.ts">

<violation number="1" location="packages/server-v4/src/db/plugin.ts:18">
P2: Close the DB connection when startup checks fail. Right now a migrate/ping error can leave an opened DB client because cleanup is only registered after those calls succeed.</violation>
</file>

<file name="packages/server-v4/test/unit/db/databaseClient.test.ts">

<violation number="1" location="packages/server-v4/test/unit/db/databaseClient.test.ts:19">
P3: Prefer forward-slash path construction instead of `path.join` here so generated paths stay normalized across platforms.

(Based on your team's feedback about using forward slashes instead of `path.join` for path construction.) [FEEDBACK_USED]</violation>

<violation number="2" location="packages/server-v4/test/unit/db/databaseClient.test.ts:23">
P2: Close each database connection in a `finally` block so cleanup runs even when setup/query/assertion fails.</violation>
</file>

<file name="packages/server-v4/src/routes/v4/browsersession/index.ts">

<violation number="1" location="packages/server-v4/src/routes/v4/browsersession/index.ts:19">
P2: Default LLM creation happens before secondary LLM ID validation, so failed requests can still insert orphan `system-default` rows.</violation>

<violation number="2" location="packages/server-v4/src/routes/v4/browsersession/index.ts:21">
P3: `getLlm` adds a reachable 404 path, but the route schema does not declare a 404 response.</violation>
</file>

<file name="turbo.json">

<violation number="1" location="turbo.json:502">
P2: `test:server` no longer builds the compiled `dist/tests` that server-v3 test scripts rely on. Add the `build:esm-tests` dependency so the tests exist before running.</violation>
</file>

Architecture diagram

sequenceDiagram
    participant Client
    participant App as Fastify Server
    participant DBPlugin as Database Plugin
    participant Svc as LLM Service
    participant Repo as LLM Repository
    participant Drz as Drizzle ORM
    participant DB as Postgres / PGlite

    Note over App, DB: NEW: Server-v4 Startup & Dependency Injection
    App->>DBPlugin: Register Plugin
    DBPlugin->>DBPlugin: NEW: Parse STAGEHAND_DB_MODE
    alt mode == 'pglite'
        DBPlugin->>DB: Open local storage (~/.stagehand/db)
        DBPlugin->>Drz: NEW: Run Migrations (migrateOnStartup)
    else mode == 'postgres'
        DBPlugin->>DB: Connect to DATABASE_URL
    end
    DBPlugin-->>App: Decorate app.db & app.dbClient

    Note over Client, DB: NEW: DB-Backed LLM Configuration Flow
    Client->>App: POST /v4/llms (LLMCreateRequest)
    App->>Svc: createLlm(input)
    Svc->>Svc: Validate via Zod (llmConfigInsertSchema)
    Svc->>Repo: create(values)
    Repo->>Drz: insert(llmConfigs)
    Drz->>DB: SQL INSERT
    DB-->>Repo: New LLM Record
    Repo-->>Svc: LLMConfigSelect
    Svc-->>App: LLM Result
    App-->>Client: 200 OK + LLM JSON

    Note over Client, DB: CHANGED: Browser Session Creation with LLM Reference
    Client->>App: POST /v4/browsersession (env, llmId?)
    alt NEW: llmId provided
        App->>Svc: getLlm(id)
        Svc->>Repo: getById(id)
        Repo-->>App: LLM Config
    else NEW: llmId missing
        App->>Svc: createSystemDefaultLlm()
        Svc->>Repo: create(source: 'system-default')
        Repo-->>App: New Default LLM
    end
    App->>App: CHANGED: Create session in stubState with UUID LLMId
    App-->>Client: 200 OK + BrowserSession (UUID id)

    Note over Client, DB: Error Handling & Validation
    Client->>App: GET /v4/llms/:id (Invalid UUID)
    App->>App: NEW: Zod Validation (LLMIdParamsSchema)
    App-->>Client: 400 Bad Request (Validation failed)
    
    Client->>App: GET /v4/llms/:id (Valid UUID, missing record)
    App->>Svc: getLlm(id)
    Svc->>Repo: getById(id)
    Repo-->>Svc: null
    Svc-->>App: Throw 404 (notFoundError)
    App-->>Client: 404 Not Found (LLM not found)

Loading

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

P2: Close the DB connection when startup checks fail. Right now a migrate/ping error can leave an opened DB client because cleanup is only registered after those calls succeed.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/server-v4/src/db/plugin.ts, line 18:

<comment>Close the DB connection when startup checks fail. Right now a migrate/ping error can leave an opened DB client because cleanup is only registered after those calls succeed.</comment>

<file context>
@@ -0,0 +1,36 @@
+    app.decorate("dbClient", null);
+    app.decorate("hasDatabase", false);
+
+    const connection = await createDatabaseConnection(database);
+
+    if (migrateOnStartup) {
</file context>

[cubic-dev-ai]

P2: Default LLM creation happens before secondary LLM ID validation, so failed requests can still insert orphan system-default rows.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/server-v4/src/routes/v4/browsersession/index.ts, line 19:

<comment>Default LLM creation happens before secondary LLM ID validation, so failed requests can still insert orphan `system-default` rows.</comment>

<file context>
@@ -16,7 +16,34 @@ const createBrowserSessionHandler: RouteHandlerMethod = async (
 ) => {
   const body = request.body as BrowserSessionCreateRequest;
-  const browserSession = createBrowserSession(body);
+  const llm =
+    body.llmId !== undefined
+      ? await request.server.llmService.getLlm(body.llmId)
</file context>

Suggested change

	const llm =
	body.llmId !== undefined
	? await request.server.llmService.getLlm(body.llmId)
	: await request.server.llmService.createSystemDefaultLlm();
	const [actLlmId, observeLlmId, extractLlmId] = await Promise.all([
	body.actLlmId
	? request.server.llmService
	.getLlm(body.actLlmId)
	.then((value) => value.id)
	: Promise.resolve(null),
	body.observeLlmId
	? request.server.llmService
	.getLlm(body.observeLlmId)
	.then((value) => value.id)
	: Promise.resolve(null),
	body.extractLlmId
	? request.server.llmService
	.getLlm(body.extractLlmId)
	.then((value) => value.id)
	: Promise.resolve(null),
	]);
	const [actLlmId, observeLlmId, extractLlmId] = await Promise.all([
	body.actLlmId
	? request.server.llmService
	.getLlm(body.actLlmId)
	.then((value) => value.id)
	: Promise.resolve(null),
	body.observeLlmId
	? request.server.llmService
	.getLlm(body.observeLlmId)
	.then((value) => value.id)
	: Promise.resolve(null),
	body.extractLlmId
	? request.server.llmService
	.getLlm(body.extractLlmId)
	.then((value) => value.id)
	: Promise.resolve(null),
	]);
	const llm =
	body.llmId !== undefined
	? await request.server.llmService.getLlm(body.llmId)
	: await request.server.llmService.createSystemDefaultLlm();

[cubic-dev-ai]

2 issues found across 8 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="turbo.json">

<violation number="1" location="turbo.json:73">
P1: `build:esm-tests` outputs are under-declared. The task emits `dist/src/**` as well, so caching/restores can drop required files and break dist-based test execution.</violation>

<violation number="2" location="turbo.json:75">
P1: `build:esm-tests` cache inputs are incomplete: it compiles `src/**/*.ts` but only tracks `tests/**/*.ts`. Add source files to inputs so Turbo invalidates this task when server-v3 source changes.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}