You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Full redesign into "refactor" and "migrate" steps (#2)
Closes #1.
## Summary
Replaces the single flag-staged `split` command with two command
families, split exactly at the tool's most important operational line —
splitting code (pure, offline, safe to iterate) versus migrating state
(backup-worthy, engine-dependent, one-shot) — connected by a reviewable
map:
```
demonolith refactor # map → run → verify (the code split)
refactor map # analyze → write the map (the review artifact)
refactor run # execute the map: write the new module directories
refactor verify # gate: emitted output ≡ source
demonolith migrate # map → prove → run → verify (the state migration)
migrate map # pull read-only, back up, split into local state copies
migrate prove # prove the split changes nothing (plans over the local copies)
migrate run # push each module's state to its new backend (guarded, never forced)
migrate verify # judge the result against the real backends
```
Every verb has one meaning: `map` produces the reviewable artifact
(deliberately not `plan` — the engines' own lifecycle verbs
init/plan/apply/refresh are avoided as command names), `run` executes
with guards, `verify` judges, `prove` rehearses. The bare family
commands run their steps in order, pausing for approval before the run
step (`-y` approves; a non-TTY pause refuses naming `-y`). The refactor
family never needs an engine or credentials; the migrate family names
its engine explicitly (`--engine {terraform|tofu}`, no default).
Everything the old `split` did survives under the new surface; `--state`
is gone because migrate *is* the state step, and `--revert` is gone
because the output is ordinary committed files, so git is the undo.
## Version
+semver: minor
## Receipts
Every command writes a receipt; "map" and "proof" are receipt types, not
separate concepts. The **map receipt** (`demonolith-refactor-map.yaml`,
"the map") is the durable contract: placement, state moves, cross-module
wiring edges, ordering edges, duplicated data sources, and each module's
derived state location — computed by `refactor map` before anything is
emitted, executed verbatim by `refactor run` (which refuses a drifted
source and finalizes the `emit_checksum` tying every later step to this
exact generation). One map per root, overwritten each run; history lives
in version control. The schema is a public, versioned API: reviewers
read it in PRs, pipelines parse it, and a control plane can ingest it —
its `cross_edges` and `ordering_edges` are exactly the wiring needed to
adopt the split-out modules. Changes within a major version are additive
only; consumers refuse a map whose major version they don't know.
The migrate steps write theirs — `demonolith-migrate-map.yaml`,
`demonolith-migrate-run.yaml`, `demonolith-migrate-prove.yaml`,
`demonolith-migrate-verify.yaml` — fixed filenames, overwritten per
execution, each carrying its `created` datetime and the generation's
`map_checksum` inside the document (`map:`/`map_checksum:` fields), so
an external system reads what ran, when, and for which map without
parsing filenames. Prove/verify receipts carry a `mode` and per-module
change counts with `ok:` — a receipt can be negative — plus the external
input *names* (never values). Every command closes its terminal output
with the same block: a `Receipt:` heading over the file it wrote.
## Guards and retries
- **Staleness**: every migrate step recomputes the checksum over the
emitted roots and refuses a map that no longer describes them; the
checksum excludes later-stage artifacts (lock files, state files,
`demono.*` files), so running the pipeline never invalidates its own
map.
- **Drift**: `refactor run` refuses if the source changed since the map
was written; the migrate steps refuse if the source analysis no longer
matches the map.
- **Target-dir safety**: a module dir that exists and is not
demonolith's own previous output fails `refactor map` before anything is
emitted.
- **Prove-receipt precondition**: `migrate run` requires a passing prove
receipt no older than the map receipt (`--unproven` is the explicit
override; run never proves on its own).
- **Never forced, retry by re-running**: `migrate run` pushes only into
empty targets or targets already holding this module's state — matched
by lineage or by identical content modulo the identity fields a fresh
split regenerates — so a crashed run is retried by just re-running:
already-pushed modules skip, the rest push, and a lost workdir is
re-split automatically at the next `migrate map`. A failed run writes a
partial (non-complete) receipt recording how far it got, never demoting
a complete receipt of the generation. Genuinely non-matching existing
state refuses with instructions, unless `--overwrite` explicitly
replaces it (`state push -force`, with an unconditional stderr warning
naming every replaced target). The monolith's own state is never
written; retiring it is a deliberate human cutover.
- **Resume**: after a partial split, `migrate map` reuses the working
monolith state (never re-pulls, which would corrupt a partial split),
classifies each move as pending or already-applied by inspecting state
addresses, and lineage-guards against a leftover workdir from a
different backend or source.
## The proof
`migrate prove` is the offline rehearsal of the migrated world: it walks
the modules in dependency order, feeds each producer's extracted planned
outputs into its consumers' inputs (the role a control plane plays at
runtime), plans each module against its local state copy, and asserts
**zero changes** — creates, destroys, and in-place updates all fail,
because a wrong input value that forces no replacement is still a wrong
value. `migrate verify` is the same proof against the **real backends**,
refresh on: a full init that must find the pushed state, then a plan
that must show zero changes. Both print one line per module as they
work, so a crash shows exactly where it stopped. The offline proof
strips the backend block out of each root's `root.tf` for the duration
(`required_providers` stays, so provider resolution keeps working) and
preserves any pre-existing local state.
Cross-module inputs are never user-suppliable — the proof supplies them
itself, so wrong wiring cannot be papered over.
## Values and credentials
Variable values resolve in the engine's own precedence — `TF_VAR_*` env,
the root's `terraform.tfvars`/`*.auto.tfvars`, `--var-file` files,
`--var` flags. `migrate prove` writes each module's resolved root
variable values into `demono.root.tfvars`; `migrate run` writes the
cross-module input values into `demono.graph.tfvars`, resolving them
from the applied state and filling what state cannot yield (child-module
outputs, expressions included) from the producer values the proof
computed, recorded in a gitignored workdir sidecar — so after map →
prove → run a module plans standalone on the two files alone. The files
are deliberately not `.auto.tfvars`: the proofs load them explicitly
with `-var-file`, and so does anyone planning a module on its own.
`--no-tfvars` (for tests) writes nothing and passes everything in
memory. A value that only ever existed as a `-var` flag on the original
apply is unrecoverable from state and must be re-supplied.
Backend credentials are materialized by `migrate run`/`verify` (refactor
deals with code only) as gitignored per-module `demono.env` files (0600,
shell-sourceable `export` lines) in the engines' official environment
variables, sourced automatically around each module's backend
operations; `demono.env` is backend-credentials-only. Provider
environment (credentials, mirrors) is deliberately not captured and not
guessed at: provider env vars are provider-internal conventions with no
machine-readable mapping, and credentials also arrive via files and
metadata services no scan can see — the documented prerequisite is that
the monolith root inits and plans cleanly, with every demonolith command
run in that same shell session.
## Emitted roots
Each module directory gets a `root.tf` holding the `terraform{}` block
per convention — the propagated `required_providers` plus the derived
backend — with resources, providers, and locals in `main.tf` and
generated boundary `variable`/`output` blocks alongside. Backends are
derived, not hand-written: the monolith's backend block is carried into
every root with its location attribute postfixed per module
(`prod/terraform.tfstate` → `prod/terraform-networking.tfstate`),
covering every built-in backend type — local, s3, azurerm, gcs, consul,
http (the whole address trio), cos, oss, kubernetes, pg, and remote (the
nested workspace name). Workspace-driven configurations (a `cloud`
block, remote in prefix mode) are explicit refusals with `--no-backend`
as the way out; a backend configured partly or wholly via
`-backend-config` flags works through the init-time resolved config.
Every module directory gets its own `.gitignore` for the local artifacts
the steps leave behind — part of the checksummed output, so `refactor`
owns it.
Data placement is not decorator-driven: a data source is a stateless
read and follows its consumers automatically — copied into every module
that references it, directly or transitively. A decorator on a `data`
block is a hard error; a data source with no placed consumer falls to
the remainder (default module name `legacy`), reported. `refactor
--monorepo` keeps local child-module calls pointing at their original
in-repo directories instead of copying them; the mode is recorded in the
map so verify re-emits identically, with relative source paths computed
against the real out dir even though verify emits into a scratch dir.
The default out dir is `modules`, resolved against `--root-dir` and
required to be inside it (the map records it root-relative, so an
outside dir would be non-portable); the migrate workdir lives beneath it
as `modules/.demono/`, self-gitignored.
## The Snap CD bootstrap
`refactor` emits `<out>/snapcd` by default (`--no-bootstrap` skips): a
Terraform root of `snapcd_*` resources generated from the map alone — a
namespace, one `snapcd_module` per module, every cross edge as
`snapcd_module_input_from_output`, every ordering edge as
`snapcd_depends_on_module`, external inputs as literals bound to the
bootstrap's variables. A `--monorepo` split additionally sets
`default_trigger_path_filter_enabled = true` on the generated namespace,
so modules only redeploy when a commit touches their own directory.
Applying it against a Snap CD server is the adoption step. The module is
covered by the emit checksum but is not a placement module — never
state-moved, never proven. Generating it from the map alone is the proof
that the public contract carries everything a control plane needs.
## Interactive mode
Interactive mode is a front-end, not a parallel channel: every choice
resolves to a flag or a source decorator a non-interactive run
reproduces, and the migrate wizard ends by printing the equivalent
non-interactive command.
`refactor map -i` prompts for the run's parameters (root, output dir,
remainder name, monorepo, bootstrap), then walks the unannotated blocks
— assignments are written back into the source as `@demono:move`
decorators, reviewable in git — and confirms before writing the map.
`migrate -i` fronts the pipeline with a full inputs wizard: engine and
state source; every variable the migration consumes — the root variables
the split-out modules declare (parsed with their defaults) plus the
boundary's external inputs — checked against the engine's precedence
with **only the gaps shown** (required variables with no value anywhere,
attributed to the modules declaring them, with a `name=value` / `@file`
loop and a hard gate before continuing with gaps); the derived backend
with the credential attributes headed for `demono.env` and collection of
extra `-backend-config` values; and the ambient-credentials contract
stated plainly with one confirmation that this is the working session.
Flags passed alongside `-i` pre-fill the answers.
## Terminal output
Output styling is role-based, mapped onto the terminal's own 16-color
palette (theme-safe, no RGB, no backgrounds): headings bold, prompts
bold cyan (visually distinct from information), step banners bold
magenta (`── migrate map ──`, on both families' substeps), statuses
green/yellow/red, secondary detail dimmed. Gated on TTY, `NO_COLOR`,
`TERM=dumb`, and a persistent `--no-color` flag. House rules: any line
introducing an indented list is a bold heading; outcome words carry
status colors; sentences stay plain, never hard-wrapped; a blank line
separates every result block from the next prompt or heading.
Reports favor plain language: the analysis ends with a topologically
sorted deploy-order dependency graph ("app (depends on: cluster,
database)") instead of a raw ordering-edge dump; `refactor verify`
speaks about "the output on disk" rather than assuming git, and shows a
first-difference preview per changed file; per-module progress prints
live (`app: proving ... zero changes`), so failures are located at a
glance. Confirmation defaults favor the safe-but-forward path: the
state-split and pipeline pauses default yes, catchall triage defaults
no.
## Machine interface
Exit codes are uniform: `0` success, `1` operational error, `2` negative
verdict — the run worked but the answer is "no" (a difference, a failed
proof, a stale map) — so a pipeline can distinguish "the split is wrong"
from "the job broke". `--output json` replaces the human report with one
JSON document (`map`, `map_path`, `receipt_path`, per-module verdicts,
tfvars files, unresolved graph inputs). Without a TTY nothing prompts;
`--interactive` errors instead of silently falling back.
## Correctness fixes surfaced by realistic monoliths
- **Ordering-only `depends_on` entries survived the carve**: the emit
filter dropped a cross-module `depends_on` entry only when the producer
also carried a value edge, leaving emitted roots referencing blocks they
do not contain. The filter now tests placement ownership; the ordering
is carried by the map's ordering edge.
- **One input per producer, however many attributes**: a consumer
referencing several attributes of one producer collapsed into a single
input carrying one value. Cross edges are per (producer, attribute);
multi-attribute producers expose one attr-scoped output per attribute,
single-attribute producers keep plain names.
- **Monorepo roots could never verify**: verify re-emits into a scratch
dir, and the relative module-source rewrite used the physical emit dir,
so the paths never matched the committed ones. Relative sources are
computed against the real out dir (emitter `PathBase`).
A `LIMITATIONS.md` documents the carve's inherent limits and the manual
handling for each — path-relative references, sensitive values crossing
a boundary, whole-block placement, dropped `moved`/`import`/`check`
blocks, workspace dependence, provider-config-only data consumers,
string-typed threading, multiplied data re-reads, occupied-target
semantics, and the prove-to-run drift window.
## Testing
End-to-end journeys against a real engine cover the full surface — the
map/run split and its refusal guards, the bare `migrate` pipeline to
zero-changes verification (idempotent on re-run), crash-and-retry
(partial receipts, `--overwrite`, re-carve after a lost workdir), the
proof and its threaded fill, the tfvars split, backend derivation and
refusals per type, and emit/verify semantics. The suite runs in about a
minute, fully offline (local mock modules, no timers, per-package
`TF_PLUGIN_CACHE_DIR`).
Beyond the suite, the companion walkthrough in
[`snapcd-samples/sample-deployment-demonolith`](https://github.com/snapcd-samples/sample-deployment-demonolith)
(updated separately) has been run end to end against a live server and
works reliably.
## Release tooling
Releases now take their notes from the PR, the same mechanism the main
snapcd repo uses: `scripts/release-notes.sh` reads the `<!--
release-notes -->` block out of the merged PR's description (falling
back to the commit subject, so a release never fails for want of one),
the release workflow feeds it to GoReleaser, and a PR template carries
the block and the `+semver:` directive GitVersion already understands.
# Version
+semver: minor
# Release Notes
<!-- release-notes -->
Replaces the single flag-staged `split` command with two command
families, split exactly at the tool's most important operational line —
splitting code (pure, offline, safe to iterate) versus migrating state
(backup-worthy, engine-dependent, one-shot) — connected by a reviewable
map:
```
demonolith refactor # map → run → verify (the code split)
refactor map # analyze → write the map (the review artifact)
refactor run # execute the map: write the new module directories
refactor verify # gate: emitted output ≡ source
demonolith migrate # map → prove → run → verify (the state migration)
migrate map # pull read-only, back up, split into local state copies
migrate prove # prove the split changes nothing (plans over the local copies)
migrate run # push each module's state to its new backend (guarded, never forced)
migrate verify # judge the result against the real backends
```
Every verb has one meaning: `map` produces the reviewable artifact
(deliberately not `plan` — the engines' own lifecycle verbs
init/plan/apply/refresh are avoided as command names), `run` executes
with guards, `verify` judges, `prove` rehearses. The bare family
commands run their steps in order, pausing for approval before the run
step (`-y` approves; a non-TTY pause refuses naming `-y`).
The refactor family never needs an engine or credentials; the migrate
family names its engine explicitly (`--engine {terraform|tofu}`, no
default).
Everything the old `split` did survives under the new surface.
An interactive mode that guides the user through the entire process can
be invoked as `demonolith refactor -i` or `demonolith migrate -i
<!-- /release-notes -->