Add Cloud Snapshots: Named Save/Load/List of Emulator State (#335 P1)

[thzgajendra]

Objective / Issue

First P1 item of the "minikube for cloud resources" roadmap (#335). Persistence (#337) auto-saves a single state on stop; this adds named, restorable snapshots on a live server — a local, free equivalent of LocalStack's paid Cloud Pods, built on the same persist engine.

What we found

• The engine to capture/restore state already exists (persist.Export/Restore), and the admin control plane already does atomic whole-emulator rebuilds for /_cloudemu/reset. So snapshots are mostly a thin layer: an admin endpoint to move state in/out of the running daemon, plus a CLI to name and store it.
• Blast radius: additive. New admin endpoint + new CLI subcommand; the persist-on-stop path is refactored to reuse a shared core but behaves identically.

How we fixed it / How it works

• persist: optional Meta header on Snapshot + whole-emulator ExportAll/RestoreAll helpers (serve's persist-on-stop path now reuses them).
• admin: GET /_cloudemu/snapshot (export whole-emulator state as JSON) and POST /_cloudemu/snapshot (rebuild-empty + restore). Both act on every provider like reset; wired via NewControl (nil = disabled, e.g. --admin=false).
• CLI: cloudemu snapshot save|load|list|delete <name>. save/load talk to the running daemon; list/delete are file ops. Snapshots are single JSON files under ~/.cloudemu/snapshots/.

sequenceDiagram
    actor U as You
    participant CLI as cloudemu snapshot
    participant S as serve (daemon)
    participant P as persist
    participant F as snapshot file (disk)

    U->>CLI: snapshot save baseline
    CLI->>S: GET /_cloudemu/snapshot
    S->>P: ExportAll(all providers)
    P-->>S: Snapshot JSON
    S-->>CLI: state
    CLI->>F: atomic write (+meta header)

    Note over U,S: … a destructive test wipes/changes state …

    U->>CLI: snapshot load baseline
    CLI->>F: read
    CLI->>S: POST /_cloudemu/snapshot
    S->>S: rebuild to empty (reset)
    S->>P: RestoreAll(snapshot → providers)
    S-->>U: state restored (no restart)

Loading

Alternatives not taken

• Stop-edit-file-restart — clunky and loses the "live" benefit; the admin endpoint restores in place.
• A separate snapshot format — reused persist.Snapshot so snapshots auto-cover every service persistence gains, with zero snapshot-code change.

Docs / Tests / Playground

• Docs: docs/standalone-server.md — named-snapshots section (uses, file location, --admin/provider requirement, name rules).
• Unit: persist multi-provider ExportAll/RestoreAll; admin snapshot GET/POST/disabled(501); CLI name validation, flag parsing, and save→list→load→delete against a test server + daemon-down error.

Test plan

• go test -race ./persist/... ./server/admin/... ./cmd/cloudemu/...
• Full local CI: gofmt / build / vet / go test ./... / go mod tidy / golangci-lint = clean (CodeQL: only pre-existing findings, none in changed files).
• Live E2E via cloudemu start + real aws CLI on the default ~/.cloudemu: create bucket/object + DynamoDB table/item → snapshot save good-state → POST /_cloudemu/reset + create junk → snapshot load good-state → junk gone, all resources restored (bodies + items intact, aws+azure+gcp captured). Error paths: load-missing → not found; save ../evil → name rejected (traversal blocked); save while stopped → clear "not running".

Risk & Rollback

• Low: additive. load wipes-then-restores (same semantics as reset) — intentional. Snapshot endpoint is gated on --admin; name sanitization blocks path traversal; writes are atomic.

Conclusion

cloudemu snapshot save/load/list/delete gives a free, local, multi-cloud, diffable "Cloud Pods": save a baseline, run destructive tests, restore in seconds, or hand a snapshot file to a teammate.

Follow-ups (#335): coverage grows automatically as more services join persistence; later — cross-machine sharing/registry, snapshot diff.

[NitinKumar004]

Deep review — named cloud snapshots

Clean, correct, and well-tested. Verified in an isolated worktree at the PR head (#337 merged since, so the diff is the 10-file set). Gates green: build (darwin) + GOOS=windows go build ./cmd/cloudemu (the non-unix runSnapshot stub compiles) / vet / go test -race ./persist/... ./server/admin/... ./cmd/cloudemu/ (ok all) / gofmt clean.

What's right:

• Whole-emulator via one port done properly — snapshotFn/restoreFn close over all targets (like reset) and are wired through NewControl; nil closures → 501, so --admin=false disables it.
• Restore semantics correct — validates SchemaVersion, rebuild()s to empty, then RestoreAll.
• Secure by construction — name regex ^[A-Za-z0-9._-]{1,64}$ + explicit ./.. exclusion blocks path traversal (tested with ../etc, a/b, …); snapshot writes reuse #337's atomic temp+rename; POST body capped at 512 MiB.
• Lock convention followed — the "read targets under rebuildMu, run Export/Restore outside it" pattern matches the existing seedFor, so no new concurrency regression.
• Tests actually exercise the flows: name validation, save→list→load→delete against a test server, daemon-down, admin-off 501, multi-provider ExportAll/RestoreAll.

These are all Low / hardening / nit — nothing blocking. Two are inline; the third:

L1 (Low, hardening). The non-loopback --admin warning in serve.go currently says only that POST /_cloudemu/reset wipes state. This PR adds GET /_cloudemu/snapshot, an unauthenticated dump of all emulated state including secret values — a new network-reachable read/exfiltration surface on --host 0.0.0.0. Emulated data, so Low, but the warning should be broadened to cover read/exfiltration, not just wipe.

[NitinKumar004]

L2 (Low) — destructive load with no rollback. rebuild() wipes all state, then RestoreAll repopulates. If RestoreAll errors partway (a driver returns an error mid-restore), the emulator is left wiped + partially restored with no recovery of the prior state — the POST returns 400 but the data's already gone. Low probability (restore into freshly-empty providers rarely fails) and consistent with the documented reset-semantics, but worth a doc note now and, later, a "restore into a staging build and swap on success" hardening so a failed load can't destroy the running state.

[thzgajendra]

Documented. Added a code comment on the rebuild() line and a note in docs/standalone-server.md making the destructive semantics explicit: load wipes then repopulates, and a mid-RestoreAll failure leaves the running state cleared. Called out your suggested hardening — restore into a staging build and swap in only on success — as the follow-up. 0f5c5a9

[NitinKumar004]

L3 (nit). The CREATED column prints the file's ModTime, not the Meta.CreatedAt (RFC3339) actually written into the snapshot — and list ignores the stored Meta entirely, so it never surfaces the captured Providers. Reading the Meta header here would make the column accurate and let list show providers too.

[thzgajendra]

Fixed. list now reads each snapshot's Meta header and shows the recorded createdAt plus a new PROVIDERS column, falling back to the file mtime / filename only when a file is unreadable or has no meta. Verified live: v1 2026-08-07T12:20:39Z aws,gcp 305. 0f5c5a9

[thzgajendra]

Thanks for the review — all three items addressed in 0f5c5a9 (all non-blocking, fixed anyway).

• L1 (admin warning): broadened the non-loopback --admin warning to state that GET /_cloudemu/snapshot dumps all emulated state — including secret values — to any caller, not just that reset wipes state.
• L2 (destructive load): documented the wipe-then-restore semantics (doc note + code comment); a mid-restore failure clears running state. Flagged your staging-build-and-swap-on-success idea as the follow-up hardening.
• L3 (list Meta): list now surfaces the stored Meta — accurate createdAt and a PROVIDERS column.

Gate green: build / gofmt / go test -race ./persist/... ./server/admin/... ./cmd/cloudemu/... / golangci-lint --new-from-rev = 0.

[NitinKumar004]

Approving — review comments addressed

Verified the hardening commit (0f5c5a9c) in an isolated worktree at the PR head; gate green (build darwin + GOOS=windows go build ./cmd/cloudemu / vet / go test -race ./persist/... ./server/admin/... ./cmd/cloudemu/ ok all / gofmt clean).

• L1 — the non-loopback --admin warning now names GET /_cloudemu/snapshot dumping all emulated state (including secret values), not just reset wiping.
• L2 — load's destructive wipe-then-restore is now documented in both the restoreFn comment and the docs, with the staging-and-swap future hardening called out.
• L3 — snapshot list reads each file's Meta header (recorded createdAt + a providers column), with a clean filename/mtime fallback when Meta is absent.

One non-blocking nit the L3 change introduced: readSnapshotMeta json.Unmarshals each file into a full persist.Snapshot, so list allocates every object and base64-decodes all bodies (snapshots can be up to 512 MiB) just to read the small Meta header. Decoding into a minimal struct{ Meta *persist.Meta } would let json skip the big providers field — a cheap win whenever you touch this next. Not a blocker.

Clean, responsive work across the whole #335 snapshot line. LGTM.