Skip to content

feat: add projectSetupLayer config and sandbox build command#110

Merged
murrayju merged 37 commits intomainfrom
project-setup-layer
Mar 17, 2026
Merged

feat: add projectSetupLayer config and sandbox build command#110
murrayju merged 37 commits intomainfrom
project-setup-layer

Conversation

@murrayju
Copy link
Member

Summary

  • Add projectSetupLayer config option: a bash script baked into a cached image layer between base and agent overlay, so projects can cache expensive system-level setup (apt packages, language runtimes, etc.)
  • Add ox sandbox build command for explicitly building sandbox layers with --no-cache support
  • Add --project flag to ox sandbox hash for inspecting setup layer hashes

Image Stack

base → projectSetupLayer (optional) → agent overlay → session

The setup script runs as root (Docker: docker exec --user root, Cloud: sudo), and its content is hashed with the base image hash for cache invalidation. When either the base image or the script changes, the layer rebuilds automatically.

New Commands

# Build all layers through agent
ox sandbox build --agent claude

# Build just through project setup layer  
ox sandbox build --project

# Force rebuild everything
ox sandbox build --agent claude --no-cache

# Inspect hashes
ox sandbox hash --project
ox sandbox hash --project --cloud

Key Changes

  • Config: New projectSetupLayer string field in OxConfig
  • Docker: ensureProjectSetupLayer() builds via run/exec/commit pattern, streams output to terminal
  • Cloud: ensureProjectSetupCloudSnapshot() builds via volume/sandbox/snapshot pattern
  • Resource cleanup: Classifies oxl-/oxlb- prefixed cloud resources and ox-sandbox:md5-*-l-* Docker images
  • CLI: sandbox build with --agent, --project, --cloud, --no-cache flags; sandbox hash --project
  • Force rebuild: force parameter threaded through all ensure functions and SandboxProvider.ensureImage()

Files Changed

File Changes
src/services/config.ts Add projectSetupLayer field
src/services/docker.ts Hash/tag computation, ensureProjectSetupLayer, force param, GHCR pull skip for setup layers
src/services/sandbox/cloudSnapshot.ts Cloud snapshot build, force param, setup hash in agent slugs
src/services/sandbox/cloudProvider.ts Chain setup layer into ensureImage()
src/services/sandbox/resources.ts Classify setup layer resources in cleanup
src/services/sandbox/sandboxExec.ts Add stream option for real-time output
src/services/sandbox/types.ts Add force to SandboxProvider.ensureImage()
src/services/sandbox/dockerProvider.ts Thread force through
src/commands/sandbox.ts build subcommand, --project flag on hash, flag validation
Tests New tests for hash, slug, and resource classification

murrayju added 23 commits March 16, 2026 15:03
…pLayer when configured

- Renamed --setup/-s flag to --project/-p for clarity
- --agent now incorporates the project setup hash in both Docker
  and Cloud output when projectSetupLayer is configured, matching
  the actual image/slug that gets built
- --cloud --agent now passes setupHash to getAgentSnapshotSlug
- Docker --agent tag includes -l-{setupHash} infix when setup layer exists
- --project + --agent: conflicting layer selection
- --project + --image: project layers aren't published to GHCR
- --image + --cloud: cloud uses slugs, not GHCR image refs
- Remove dead code from --project branch that handled the now-rejected --image combo
Add a new `ox sandbox build` command that explicitly builds sandbox
image layers (base, project setup, agent overlay) and all parent layers.

- Supports --agent, --project, --cloud flags (same validation as hash)
- --no-cache forces rebuild by threading a `force` param through all
  ensure functions, skipping existence checks
- Docker: skips imageExists checks when force=true
- Cloud: deletes existing snapshot before rebuilding when force=true
- Updates SandboxProvider interface with force option on ensureImage
- Progress output written to stderr for scriptability
…p layer

The agent overlay build tried to pull from GHCR before building locally.
When the base image included a project setup layer, the GHCR image (which
has no project setup) was pulled and re-tagged with a name implying the
setup was applied. This resulted in identical image IDs across different
project setup hashes.

Fix: only attempt GHCR pull when the base image is the standard base
(not a project setup layer). When a setup layer is present, go straight
to local build via docker run + exec + commit.
… cleanup

Project setup layer images use the GHCR repository prefix (since the base
image from ensureDockerImage is the GHCR tag). The resource classifier
computed current setup tags only for the local prefix (ox-sandbox:md5-...)
so GHCR-prefixed setup images fell through as 'old'.

Fix: compute current setup/overlay tags for both the local and GHCR base
image prefixes, and add them to currentGhcrTags so the GHCR classification
branch recognizes them as current.
…t GHCR

getProjectSetupTag now always produces an ox-sandbox:md5-... tag regardless
of the base image prefix, since project setup layers are built locally and
never published to GHCR. This also simplifies resource classification —
no need to compute tags for both prefixes.

Added tests verifying GHCR input is normalized to local prefix.
Wrap the build action in try/catch so errors from ensureDockerImage,
ensureProjectSetupLayer, ensureAgentOverlay, and cloud equivalents
are displayed cleanly instead of surfacing as unhandled promise rejections.
Extract stderr from ShellError and display it inline so users see
the actual failure reason (e.g., 'sudo: command not found') instead
of just a generic 'Failed with exit code 127'.
…stdout

- Rewrite progress handling with a runStep helper that ensures each
  line is properly terminated (even when ensure functions return early
  on cache hits without emitting progress events)
- Print the final image name/slug to stdout for scriptability
- All progress messages go to stderr via process.stderr.write
- Error messages use printErr for consistent stderr output
Docker: use `docker exec --user root` so the script can run apt-get
and other privileged operations without needing sudo installed.
The committed image still defaults to the ox user.

Cloud: pass `sudo: true` to sandboxExec so the script runs via
`sudo bash -c '...'`.
…dbox build

Docker: use Bun.spawn with stdout/stderr 'inherit' when stream option
is set, so the user sees apt-get progress, compilation output, etc.

Cloud: add stream option to sandboxExec that tees piped stdout/stderr
to process.stderr in real-time.

Also fix progress output formatting so labels and streamed content
are on separate lines.
Copilot AI review requested due to automatic review settings March 16, 2026 20:53
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new cached “project setup layer” between the sandbox base image and the agent overlay, plus new CLI utilities to build/inspect these layers (Docker and Cloud) with --no-cache/force support.

Changes:

  • Introduces projectSetupLayer in OxConfig, hashed with the base to form deterministic setup layer tags/slugs.
  • Implements setup-layer build flows for Docker (run/exec/commit) and Cloud (volume/sandbox/snapshot), and threads force through providers.
  • Expands resource classification/cleanup and CLI (ox sandbox build, ox sandbox hash --project) with new tests and docs.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
src/services/sandbox/types.ts Adds force?: boolean to SandboxProvider.ensureImage() options.
src/services/sandbox/sandboxExec.ts Adds stream?: boolean for real-time stdout/stderr streaming during sandbox exec.
src/services/sandbox/resources.ts Classifies project setup snapshots/volumes (oxl-*, oxlb-*) and setup-layer Docker images.
src/services/sandbox/resources.test.ts Adds/updates tests for setup-layer resource classification.
src/services/sandbox/dockerProvider.ts Threads force into Docker provider ensureImage().
src/services/sandbox/cloudSnapshot.ts Adds setup snapshot slug + build flow; updates agent slug to incorporate setup hash; adds force.
src/services/sandbox/cloudSnapshot.test.ts Tests for setup snapshot slugs and setupHash-aware agent slugs.
src/services/sandbox/cloudProvider.ts Chains setup snapshot into ensureImage() and threads force.
src/services/docker.ts Adds setup-layer hashing/tagging + Docker build flow; threads force; adjusts GHCR pull behavior.
src/services/docker.test.ts Tests for setup-layer hash/tag helpers.
src/services/config.ts Adds projectSetupLayer?: string config and CONFIG_KEYS entry.
src/commands/sandbox.ts Adds sandbox build and sandbox hash --project, plus flag validation and --no-cache.
docs/superpowers/specs/2026-03-16-project-setup-layer-design.md Design spec for the setup layer feature.
docs/superpowers/plans/2026-03-16-project-setup-layer.md Implementation plan documentation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Copy link
Member Author

@murrayju murrayju left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two significant correctness issues stood out:

  1. sandbox hash now computes Docker project/setup hashes from the embedded base.Dockerfile, not from the resolved base image. That makes ox sandbox hash --project / --agent report the wrong tag whenever a repo uses buildSandboxFromDockerfile or sandboxBaseImage.
  2. Docker resource discovery hard-codes the embedded base hash for its “current” tag set. Under those same non-default Docker base configs, the cleanup UI can classify the live setup-layer / agent-overlay images as old and offer to delete them.

After those are fixed, there is also a good DRY-up opportunity in cloudSnapshot.ts: ensureCloudSnapshot, ensureProjectSetupCloudSnapshot, and ensureAgentCloudSnapshot now repeat the same force/delete/check/boot/snapshot/cleanup scaffolding. Pulling that lifecycle into one helper would reduce the chance of the rebuild semantics drifting again.

murrayju added 13 commits March 16, 2026 17:24
- Use resolveSandboxImage() in sandbox hash command and resource
  discovery instead of hardcoding the embedded Dockerfile hash.
  Custom base images (buildSandboxFromDockerfile, sandboxBaseImage)
  now produce correct hashes and cleanup classifications.

- Fix sandboxExec stream path: separate early-return branch that
  collects output while streaming, avoiding double-consumption of
  stdout/stderr streams.

- Fix GHCR pull gate in ensureAgentOverlay: check for -l- setup
  layer infix instead of exact image name match, so locally-built
  standard bases still benefit from GHCR agent pulls.

- Fix force dedupe: always coalesce concurrent ensureDockerImageForAgent
  calls, even when force is set, preventing parallel rebuild races.

- Fix DockerSandboxProvider.ensureImage(): chain through project setup
  layer when no agent is specified, matching cloud provider behavior.
…ng agent image'

The readiness store now captures the `message` from building progress
events (e.g. 'Running project setup layer', 'Installing codex agent')
and the TUI component displays it. Falls back to 'Building {agent}
agent image' when no specific message is available.
… builds

Surface the latest stdout/stderr line from the project setup script
in both the readiness status bar and the session-starting screen.

- Add detail field to ImageBuildProgress and SandboxBuildProgress types
- Docker: pipe setup script output line-by-line, fire onProgress with
  detail for each non-empty line (only when not streaming to terminal)
- Cloud: add onLine callback to sandboxExec that fires for each line,
  wired into ensureProjectSetupCloudSnapshot's onProgress
- Readiness store: track agentBuildDetail from progress events
- ReadinessStatus: show detail line below build message (dimmed)
- Loading/StartingScreen: add subDetail prop for build output lines
- Router: add detail field to starting view type
- Session workflow store: forward progress.detail to starting view
…l line

The outer box was hardcoded to height=1, causing the detail line to
render on top of the status message. Now dynamically sets height=2
when agentBuildDetail is present.
@murrayju murrayju merged commit e98cbb0 into main Mar 17, 2026
2 checks passed
@murrayju murrayju deleted the project-setup-layer branch March 17, 2026 20:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants