Skip to content

Ship the Studio server in the workspace image (CS-003 reopened) - #175

Merged
Travis-Gilbert merged 1 commit into
mainfrom
feat/ide-substrate-wire
Aug 4, 2026
Merged

Ship the Studio server in the workspace image (CS-003 reopened)#175
Travis-Gilbert merged 1 commit into
mainfrom
feat/ide-substrate-wire

Conversation

@Travis-Gilbert

Copy link
Copy Markdown
Owner

Deploy 4e33d620 built green off 01143ad5 and /opt/commonplace/studio-server was absent from the running container. CS-003 was marked done by reading the Dockerfile; the container disagreed.

Two independent defects, either one fatal on its own.

No COPY --from=studio-server. The stage wrote /out and the final image never took it. BuildKit prunes a stage nothing copies from, so the fork was not merely uncopied, it was never compiled. That is why a build carrying the mangler patch finished without ever reaching the mangler.

The build gate shared a name with the runtime switch. Railway injects every service variable into whatever ARG a stage declares, so IDE_HOST=code-server, pinned on the service to hold the runtime host during cutover, would have switched the build off too. The safe-cutover shape the plan rests on, ship both binaries and flip a variable, was unreachable: pinning the variable deleted the binary from the next image.

Changes

  • studio-server stage gated on ARG BUILD_STUDIO_SERVER=1; IDE_HOST is runtime only, and the unused ARG IDE_HOST in the base stage is gone.
  • COPY --from=studio-server /out/ /opt/commonplace/studio-server/.
  • Rollback is --build-arg BUILD_STUDIO_SERVER=0, which still writes /out so the COPY stays valid and only the launcher is missing, which is the file entrypoint.sh probes with -x. A skipped build falls back to code-server by itself.
  • Corrections recorded in FOLLOW-UP-CODE-SERVE-WEB.md, SPEC-COMMONPLACE-OPENWORK-FORK-1.0.md, and .commonplace-canonical.

Blast radius

None at runtime. IDE_HOST=code-server stays pinned on the service, so this image ships both hosts and keeps running stock code-server. If the Studio stage fails, the image fails and Railway holds the previous deploy.

Validation

  • node scripts/check-register-manifest.mjs: ok, 8 registers, 37 registry ids.
  • .commonplace-canonical parses as JSON.
  • Dockerfile lint not-run: no Docker daemon on this machine. Railway is the only builder (the mac kernel SIGKILLs the VS Code build), so the Railway build is the oracle.
  • CS-002 smoke against the built container is the next step and has not run yet.

Copilot AI lite review requested due to automatic review settings August 4, 2026 14:07
@ecc-tools

ecc-tools Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

ECC bundle files are already tracked in this repository. Skipping generation of another bundle PR.

CS-003, reopened. Deploy 4e33d620 built green off 01143ad and
/opt/commonplace/studio-server was absent from the running container.
Two independent defects, either one fatal alone.

No COPY --from=studio-server existed anywhere. The stage wrote /out and
the final image never took it, and BuildKit prunes a stage nothing
copies from, so the fork was not merely uncopied, it was never
compiled. That is why a build carrying the mangler patch finished
without ever reaching the mangler.

The gate shared a name with the runtime switch. Railway injects every
service variable into whatever ARG a stage declares, so IDE_HOST set on
the service to keep the runtime host on code-server during cutover
would have switched the build off too. The safe-cutover shape this plan
rests on, ship both binaries and flip a variable, was unreachable:
pinning the variable deleted the binary from the next image.

The stage is now gated on BUILD_STUDIO_SERVER, IDE_HOST is runtime only
and the dead ARG in the base stage is gone, and the COPY exists. At
BUILD_STUDIO_SERVER=0 the stage still writes /out so the COPY stays
valid; what it omits is the launcher, which is the file entrypoint.sh
probes with -x, so a skipped build falls back to code-server by itself.

Proof is the container, not the Dockerfile. The previous done was read
off the source, which is how this survived a green build.
@Travis-Gilbert
Travis-Gilbert force-pushed the feat/ide-substrate-wire branch from 95a87a6 to cd43c3e Compare August 4, 2026 14:08
@Travis-Gilbert
Travis-Gilbert merged commit 2ab32f1 into main Aug 4, 2026
1 of 2 checks passed
@ecc-tools

ecc-tools Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

ECC bundle files are already tracked in this repository. Skipping generation of another bundle PR.

@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@Travis-Gilbert, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 50 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 7bde05f4-1261-4ddf-8f5b-0a540367ebad

📥 Commits

Reviewing files that changed from the base of the PR and between 01143ad and cd43c3e.

📒 Files selected for processing (4)
  • .commonplace-canonical
  • docs/plans/console/SPEC-COMMONPLACE-OPENWORK-FORK-1.0.md
  • docs/plans/ide-substrate-wire/FOLLOW-UP-CODE-SERVE-WEB.md
  • packaging/workspace/Dockerfile
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/ide-substrate-wire

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a production gap where the workspace image did not actually ship the Commonplace Studio server and strengthens the /IDE substrate wiring (pack + co-located editor API), including new doctor probes and an edge proxy to support WebSocket upgrades.

Changes:

  • Ensure the workspace Docker image builds and copies the Studio reh-web server output into the runtime image, with a build-time gate that doesn’t conflict with runtime host selection.
  • Add a co-located commonplace-api “editor substrate” process + bootstrap flow to generate THEOREM_EDITOR_* env for the theorem-vscode pack, plus doctor probes for health/readiness.
  • Add a console-side edge proxy for /IDE (WebSocket-capable) and update console views/tests/manifests to recognize the IDE surface.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

Show a summary per file
File Description
scripts/doctor.mjs CLI doctor: route + substrate assertions
pnpm-lock.yaml Workspace dependency graph updates for theorem-vscode + new packages
packaging/workspace/railway.toml Railway deploy docs + watch patterns for IDE substrate inputs
packaging/workspace/patch-code-server-product.mjs Patch code-server product.json for proposal grants
packaging/workspace/entrypoint.sh Start chat + IDE + optional co-located substrate; host selection logic
packaging/workspace/Dockerfile Build/copy Studio server + pack + editor substrate into workspace image
packaging/workspace/bootstrap-editor-substrate.mjs Wait/health-check substrate; createProject; write env stamp
packaging/commonplace-studio/scripts/smoke-server.sh Shell smoke for Studio server artifact + identity/telemetry/pack
packaging/commonplace-studio/scripts/ledger-gate.sh Make built-tree checks honor STUDIO_BUILD_DIR
packaging/commonplace-studio/scripts/build.sh Add server target; disk/path/node guards; pack staging options
packaging/commonplace-studio/RUNBOOK.md Update web validation to use server smoke script + reh-web target
packaging/commonplace-studio/patches/0001-mangler-keep-session-changes-overrides-protected.patch Build-break patch for upstream minify mangler visibility
packaging/commonplace-studio/LEDGER.md Record new patch debt + reproduction details
packages/theorem-acp/src/hosted-client.ts Allow explicit ws URL + token plumbing for hosted ACP
packages/block-view-contracts/package.json Add @noble/hashes dependency
docs/plans/ide-substrate-wire/PLAN.md New plan doc for IDE substrate wiring
docs/plans/ide-substrate-wire/FOLLOW-UP-CODE-SERVE-WEB.md Follow-up doc for Studio serve-web cutover and findings
docs/plans/ide-substrate-wire/EXECUTE-REPORT.md Execution report for IDE substrate wiring
docs/plans/console/SPEC-COMMONPLACE-OPENWORK-FORK-1.0.md Spec amendment A14 documenting selectable IDE host
docs/learnings/2026-08-03-zero-collapse-panel-clips-its-own-toggle.md New learning note (console UI gotcha)
docs/learnings/2026-08-03-stacked-ci-failures-mask-each-other.md New learning note (CI diagnosis)
docs/learnings/2026-08-03-resolve-dep-versions-from-the-consuming-package.md New learning note (pnpm resolution)
apps/theorem-vscode/test/substrate.test.ts Tests for project_id + x-api-key behavior
apps/theorem-vscode/src/substrate/client.ts Auth headers + SSE via fetch streaming fallback + project_id
apps/theorem-vscode/src/extension.ts Resolve config via env overrides; refresh on save; EventSource gating
apps/theorem-vscode/src/config.ts Central env/settings resolution for hosted pack
apps/theorem-vscode/src/agent/session-opener.ts Switch to HostedAcpClient.connect and explicit ws/token
apps/theorem-vscode/package.json Enable proposed APIs; update defaults + config descriptions; deps
apps/console/src/views/registry.tsx Add IDE surface descriptor and register
apps/console/src/views/OpenworkChatRegister.tsx Force full reload link for chat proxying
apps/console/src/views/IdeRegister.tsx New IDE register fallback component
apps/console/src/views/CodeFileView.tsx Clarify CM6 vs /IDE responsibilities in header comment
apps/console/src/lib/view-routing-retirement.test.ts Add /IDE routing expectations
apps/console/src/lib/register-impl.ts Map ide.surfacecode-server.ide
apps/console/src/lib/rail/rail-model.ts Add IDE to launch rail places
apps/console/src/lib/rail/rail-model.test.ts Update rail tests for 6th place
apps/console/src/lib/chat/last-console-view.ts Exclude /IDE from “last view” persistence
apps/console/src/editor-model/document-store.test.ts Fix stub typing by spreading modelHost()
apps/console/src/components/shell/Sidebar.tsx Add IDE icon mapping; update spec comment
apps/console/src/app/IDE/page.tsx App Router fallback for /IDE with auth-aware messaging
apps/console/src/app/chat/page.test.tsx Update chat route behavior assertions
apps/console/src/app/api/doctor/route.ts Add shared probe helper + substrate probes; env contract adjustments
apps/console/src/app/api/doctor/route.test.ts Unit tests for route probe redirect + auth behavior
apps/console/src/app/api/doctor/probe.ts New probe helper for canonical redirects + impl stamping
apps/console/scripts/start-railway.mjs Start Next on internal port and edge-proxy on public port
apps/console/scripts/edge-proxy.test.mjs Unit tests for /IDE path stripping + upstream derivation
apps/console/scripts/edge-proxy.mjs New WebSocket-capable edge proxy for /IDE with cookie auth
apps/console/package.json Include edge-proxy unit test in npm test
apps/console/e2e/global-setup.ts Add /IDE route to e2e route list
apps/console/e2e/console-sidebar.spec.ts Assert IDE nav + update keyboard shortcut coverage to 1..6
apps/console/e2e/cards.spec.ts Update rail count assertion from 5 → 6
.harness/checklists/ide-substrate-wire--plan-ide-substrate-wire-20260803a.json New checklist projection for plan execution
.commonplace-canonical Register/env contract updates to include IDE + workspace/substrate keys
Suppressed comments (1)

packaging/workspace/Dockerfile:50

  • EDITOR_API_IMAGE is advertised as a supported fast-path, but it is never used anywhere in this Dockerfile, so the docs here are currently incorrect and the arg is dead. That can lead to false confidence that a prebuilt image override is taking effect when it isn’t.
# Prefer a digest-pinned prebuilt image (fast):
#   --build-arg EDITOR_API_IMAGE=ghcr.io/.../commonplace-api@sha256:...
# Otherwise clone Theorem and build (slow, default when INCLUDE_EDITOR_SUBSTRATE=1).
ARG INCLUDE_EDITOR_SUBSTRATE=1
ARG EDITOR_API_IMAGE=

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@chatgpt-codex-connector

Copy link
Copy Markdown

💡 Codex Review

./scripts/build.sh server \

P1 Badge Use a Linux-compatible disk-space check for Studio

This newly invokes build.sh server inside node:24-bookworm, but free_gib() in that script runs df -g; on Debian's GNU coreutils that command exits with df: invalid option -- 'g'. Because the script enables set -euo pipefail, every default BUILD_STUDIO_SERVER=1 image build stops in require_disk() before cloning or compiling Studio. Use a portable byte/block-size option or handle GNU and BSD df separately.


if (!decodeActiveWorkspaceCookie(raw ?? '', secret)) {

P1 Badge Revalidate the active workspace before proxying the IDE

Any unexpired signed cp_active_workspace cookie is accepted here without checking the current Auth.js subject, live membership, or whether claims.workspaceId and tenant correspond to the single configured ideUrl. Unlike resolveHarnessPrincipal(), this means a cookie retained after sign-out or membership revocation remains sufficient for up to eight hours, and a user selecting another workspace can still reach this workspace's unauthenticated code-server and terminal. Revalidate the principal and membership and bind the claims to the upstream before forwarding.

AGENTS.md reference: apps/console/AGENTS.md:L24-L31


const html = stampHtml(Buffer.concat(chunks).toString('utf8'));
const body = Buffer.from(html, 'utf8');
outHeaders['content-length'] = String(body.length);
res.writeHead(up.statusCode || 502, outHeaders);

P1 Badge Remove stale encodings after rewriting IDE HTML

When the IDE upstream returns HTML with transfer-encoding: chunked or content-encoding: gzip/br, outHeaders retains those headers while the proxy buffers the bytes, converts them directly to UTF-8, and adds content-length. The chunked case sends both framing headers and clients reject it with Content-Length can't be present with Transfer-Encoding; the compressed case also corrupts the encoded body. Strip transfer framing and either decompress before stamping or avoid transforming encoded HTML.


const token =
env('THEOREM_EDITOR_API_KEY')
?? env('THEOREM_ACP_TOKEN')
?? (config.get<string>('token') || undefined);

P2 Badge Keep editor and ACP credentials separate

When both documented credentials are configured and differ, THEOREM_EDITOR_API_KEY wins this single token field, which is then used both by SubstrateClient and by openIdeSession() as the ACP bearer. The editor GraphQL calls work, but every ACP WebSocket session authenticates with the wrong credential and fails. Resolve separate editor and ACP token fields, using THEOREM_ACP_TOKEN for the hosted agent connection.


ENV COMMONPLACE_WORKSPACE_TENANT_ALLOWED_ROOTS={"Travis-Gilbert":["/workspace/repo"]}

P1 Badge Escape the tenant-map JSON in ENV

Dockerfile ENV parsing treats these unescaped double quotes as word-grouping syntax, so the container receives {Travis-Gilbert:[/workspace/repo]} rather than valid JSON. The co-located API requires this JSON tenant map to admit createProject, so its default startup/bootstrap path rejects the configured roots and leaves the editor substrate unavailable. Escape the embedded quotes so the runtime value remains valid JSON.

AGENTS.md reference: AGENTS.md:L24-L31


await consumeSse(response.body, (data) => this.onInvalidationFrame(data), abort.signal);
} catch (error) {
if (!abort.signal.aborted) {
this.options.onChangefeedStatus?.('stale');
this.log(`invalidations: ${describe(error)}`);
}
} finally {
if (this.fetchAbort === abort) this.fetchAbort = null;
}

P2 Badge Reconnect the authenticated invalidation stream

When the fetch-backed SSE response ends or encounters a transient network error, finally only clears fetchAbort; it never calls ensureChangefeed() or opens another stream. Existing subscriptions remain registered, so no later operation reconnects them and externally generated diagnostics, history, and search invalidations stay stale for the rest of the IDE session. Reopen the authenticated stream after non-abort termination with bounded retry behavior.


node - <<'NODE' "${SETTINGS_PATH}"
const fs = require('node:fs');
const path = process.argv[1];

P2 Badge Pass the settings path at the correct argv index

For node - <path>, process.argv[1] is the literal - and the supplied settings path is process.argv[2]. Every container start therefore reads and writes a file named - in the working directory while leaving the actual code-server settings.json untouched, so none of the environment-derived Theorem settings or the opt-in persisted token are installed. Pass the path before the heredoc in the expected position or read argv[2].


# Prefer a digest-pinned prebuilt image (fast):
# --build-arg EDITOR_API_IMAGE=ghcr.io/.../commonplace-api@sha256:...
# Otherwise clone Theorem and build (slow, default when INCLUDE_EDITOR_SUBSTRATE=1).
ARG INCLUDE_EDITOR_SUBSTRATE=1
ARG EDITOR_API_IMAGE=

P2 Badge Wire the advertised editor API image into the build

A repository-wide search finds EDITOR_API_IMAGE only in this declaration and its comments; no FROM or COPY consumes it. Supplying the documented digest-pinned prebuilt image therefore still runs editor-api-build, requires THEOREM_GIT_TOKEN, clones the private repository, and compiles Rust. This makes the advertised fast/prebuilt path unusable and causes builds configured with only EDITOR_API_IMAGE to fail for a missing Git token.


{
tier: 'place',
id: 'place-ide',
kind: 'ide',
label: 'IDE',
path: '/IDE',

P1 Badge Hard-navigate to the reverse-proxied IDE route

PLACE_ENTRIES is consumed by both shell rails through router.prefetch() and softNavigate()/router.push(). On a configured production deployment, those Next client requests to /IDE are intercepted by the edge and return a VS Code HTML document rather than an App Router flight payload, so clicking the new rail entry or using its shortcut cannot complete the client transition; after the timeout the code merely activates the fallback surface. Exclude this route from prefetching and use a full document navigation.


// Workspace door vars — required for live /chat and /IDE, but remain

P1 Badge Replace the newly added em dashes

The new comments here and at line 182 introduce em dashes, but the scoped Console constitution explicitly forbids em or en dashes even in code comments. Replace both characters with permitted punctuation so the change complies with the app's merge-blocking writing rules.

AGENTS.md reference: apps/console/AGENTS.md:L169-L172

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants