Skip to content

Route browser fs and logs endpoints directly to the VM - #178

Open
tnsardesai wants to merge 6 commits into
mainfrom
hypeship/direct-vm-fs-logs
Open

Route browser fs and logs endpoints directly to the VM#178
tnsardesai wants to merge 6 commits into
mainfrom
hypeship/direct-vm-fs-logs

Conversation

@tnsardesai

@tnsardesai tnsardesai commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds fs and logs/stream to the default KERNEL_BROWSER_ROUTING_SUBRESOURCES prefixes, so every /browsers/{id}/fs/* operation (JSON, binary read/write, multipart upload, watch SSE) and /browsers/{id}/logs/stream is rewritten by the routing fetch to the browser VM's cached base_url with ?jwt= and no API-key Authorization.

Browser lifecycle/metadata, extensions, replays, telemetry/events, and anything else under logs/ stay on the control plane. KERNEL_BROWSER_ROUTING_SUBRESOURCES still overrides the list, and an empty value still disables routing.

Fixes one latent bug the new routing surfaced: createRoutingFetch builds a probe Request from the incoming body for cache sniffing, and that construction derives a multipart/form-data content-type carrying its own boundary. Copying that header onto the routed request while re-encoding the FormData body produced a boundary mismatch, and the VM rejected direct fs.upload calls with 400 failed to read form part. The routed init now drops an inherited content-type when the body derives its own (FormData, URLSearchParams, Blob) and the caller did not set one explicitly. fs.upload is the first routed multipart endpoint, so nothing previously routed was affected.

Query preservation, header stripping, signal propagation, stale-JWT eviction and the fallback body construction are otherwise unchanged; the added tests pin the fallback's existing body replay behavior for the body shapes the SDK produces.

Tests

tests/lib/browser-routing.test.ts:

  • default prefix list and segment-boundary matching include fs and logs/stream, and still exclude telemetry/events, fsx/..., logs, logs/history, logstream, extensions, replays
  • a routing-fetch level test proves logs/stream routes to the VM while logs, logs/history and logstream stay on the API origin
  • fs JSON (listFiles), binary readFile, binary writeFile, multipart upload (indexed files[0][…] field names), fs/watch/{id}/events SSE and logs/stream SSE route to the VM with the query preserved, ?jwt= appended and no authorization header
  • a routed multipart request parses with request.formData(), which fails on the boundary mismatch described above (verified: the test fails without the fix)
  • logs.stream aborts stay wired to the routed request's signal
  • stale-JWT fallback replays the writeFile binary body byte for byte and the upload multipart body on the control plane with bearer auth, and evicts the route
  • a real undici request with a consumed stream never reaches the control plane after the stale-JWT response; it surfaces the existing connection error and leaves the route evicted
  • env override keeps fs/logs on the control plane; empty env disables routing
  • control-plane subresources (telemetry/events, replays, extensions) keep the API origin and bearer auth

Ran locally: jest (422 passed, 230 skipped), prettier --check ., eslint ., tsc, ./scripts/build.

Live validation

Ran against staging with real headless browsers: fs.writeFile, fs.readFile, fs.listFiles, fs.upload (two entries), fs.watch.start/events/stop and logs.stream all hit https://<browser-host>/browser/kernel/...?jwt=... and returned the expected data, with uploaded files reading back with the correct per-entry contents. telemetry/events and the browser delete stayed on the control plane. The fs.upload failure above was found this way and re-verified as fixed; re-validated after switching the default to logs/stream.


Note

Medium Risk
Changes which browser API calls hit the VM vs control plane and how multipart bodies are forwarded; behavior is covered by broad tests but affects file I/O and streaming paths.

Overview
Default direct-to-VM routing now includes fs and logs/stream, so /browsers/{id}/fs/* (list/read/write/upload/watch SSE) and live logs/stream SSE are rewritten to the cached browser base_url with ?jwt= and without API Authorization. Other paths (telemetry/events, replays, extensions, non-stream logs/*) stay on the control plane; KERNEL_BROWSER_ROUTING_SUBRESOURCES still overrides or disables routing.

Multipart fix in buildRoutedInit: when the routed body is FormData, URLSearchParams, or Blob and the caller did not set content-type, the inherited header from the probe Request is dropped so the outbound fetch can derive a boundary that matches the re-encoded body (fixes VM 400 on fs.upload).

Tests cover VM routing for fs/logs streams, segment-boundary allowlisting, stale-JWT fallback with body replay, stream abort wiring, env overrides, and parseable multipart on the routed request.

Reviewed by Cursor Bugbot for commit 32fd506. Bugbot is set up for automated code reviews on this repo. Configure here.

Add fs and logs to the default direct-to-VM subresource prefixes so
filesystem operations and log streaming use the cached browser base_url
and JWT instead of the control plane. Extensions, replays and
telemetry/events stay on the control plane.
The probe Request built for cache sniffing derives a multipart content-type
that carries its own boundary. Copying that header onto the routed request
while re-encoding the body produced a boundary mismatch, so the browser VM
rejected direct fs.upload calls with "failed to read form part". Let the
routed fetch derive the header when the body derives its own content-type.
Narrows the default prefix so a future logs read served by the control
plane is not swept onto the browser VM.
@tnsardesai
tnsardesai marked this pull request as ready for review September 2, 2026 20:57
@tnsardesai
tnsardesai requested review from Sayan- and rgarcia September 2, 2026 21:19
rgarcia
rgarcia previously approved these changes Sep 3, 2026

@rgarcia rgarcia left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reviewed alongside the go (#174) and python (#164) siblings, including a control-plane vs kernel-images surface comparison. all 16 fs/* and logs/stream endpoints match on params, body fields, 2xx codes and content types, and metro-api's /browser/kernel/* handler already wakes standby VMs and records session activity for direct requests, so the routing change itself looks safe.

the multipart content-type fix is correct and minimal: constructing the Request serialized the FormData once with boundary A, and the routed fetch re-serialized with boundary B while still sending header A. no previously routed endpoint had a multipart body, so this only surfaces with fs/upload / fs/upload_zip. the request.formData() test is the one that would have caught it. files[0][dest_path] naming is what the VM parser expects and matches what the control plane already forwarded.

Nits

  • src/lib/browser-routing.ts:314-319 — the headers.delete relies on routedInit.headers sharing the same Headers instance; consider deleting before building routedInit or a one-line note
  • tests/lib/browser-routing.test.ts — fallback mocks return 204 for fs/upload / fs/write_file; production returns 201

Sayan-
Sayan- previously approved these changes Sep 3, 2026

@Sayan- Sayan- left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. Verified against a mock control plane and VM over real HTTP: fs/* and logs/stream route to the VM with ?jwt= and no Authorization, telemetry/events and replays stay on the control plane with bearer auth, query strings and binary bodies survive intact, the env override pushes everything back, and a stale-JWT fallback replays a buffered body byte for byte and evicts the route.

  • p2: the stale-JWT replay tests inject fetch, which cannot distinguish a safe replay from an unsafe one. Constructing a Request from a drained stream inside an injected fetch yields an empty body and no error, so a body shape that fails at runtime still satisfies those assertions. Real undici throws instead. The routed-multipart boundary test does exercise the real encoder, so this applies only to the fallback-replay assertions.

  • p2: a non-replayable body on the stale-JWT path surfaces APIConnectionError: Connection error. against a healthy server. Python surfaces the original 401 and Go the failed request. Measured with fs.createReadStream and an async generator. The route is evicted either way, so the next call recovers.

  • p2, pre-existing: the writeFile JSDoc example passes fs.createReadStream('path/to/file'), which is not assignable to the declared contents type and is the shape that hits the above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants