Skip to content

M1.2: Snapshot Hub MVP — public registry of pre-built parent snapshots #29

@WaylandYang

Description

@WaylandYang

Tracked in ROADMAP.md M1.2. Filing this as the trackable issue so the work is visible alongside the rest of M1.

Problem

Right now the README quickstart requires a ~10-minute first-time setup: `bash scripts/setup-host.sh` + recipe `build.sh` (pulls a Docker image, mkfs.ext4, copies layers, warms up). That's a high bounce rate for users who land on the repo from a HN post or social link.

Proposal

A public registry of pre-built parent snapshots (memory.bin + rootfs.ext4 + vmstate.json + manifest, packed and compressed) that users `forkd pull` in ~30 s on a fresh host.

Pack format (v1)

```
.forkd-snapshot.tar.zstd
├── manifest.toml # name, version, base image, sizes, sha256 of each artifact
├── memory.bin # mmap-able parent memory
├── rootfs.ext4 # block device for child
└── vmstate.json # Firecracker vCPU + device state
```

CLI surface

  • `forkd pull deeplethe/python-numpy` — fetches + verifies + extracts into local snapshot directory.
  • `forkd push ` — uploads (auth via bucket-side signed URL or token).
  • `forkd images list` — what's cached locally.

Hosting (v1)

R2 or S3 bucket controlled by the project maintainer. CDN-fronted public read. Signed write for now (only the project pushes); user-submitted snapshots are future work.

Done criteria

  • All 6 existing recipes pushed to the hub.
  • On a clean Ubuntu 24.04 host with forkd installed but no recipes built locally, `forkd pull deeplethe/python-numpy && forkd fork --tag python-numpy -n 100` works end-to-end. Total wall-clock < 60 s.
  • README quickstart rewritten to lead with `forkd pull` instead of `bash recipes//build.sh`.

Risk

Base-image redistribution licensing. Need a 1-day pass over each recipe's base image license — all our current bases (`python:slim`, `node:slim`, `mcr.microsoft.com/playwright`, `quay.io/jupyter/scipy-notebook`, `e2bdev/code-interpreter`, `agent-infra/sandbox`) are Apache / BSD / MIT / similar, so this should be clean, but worth verifying before publishing.

Out of scope for this issue

  • User-uploadable hub (signed-in users push their own snapshots). Tracked as a separate future ticket once the basic MVP has traction.
  • Web UI / index page.
  • Diff-aware push/pull (will land naturally on top of #TBD diff-snapshots).
  • OCI registry compatibility — appealing for tooling reuse but adds Bun-of-complexity to v1. v1 is plain HTTPS object storage.

Dependencies

  • None for v1.
  • M2.1 (#TBD diff snapshots) will extend the pack format to express chains. Forward-compatible — v1 manifest reserves a `parent_tag` field.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions