Skip to content

Add declarative JSON-manifest mode: session apply#6

Merged
dkijania merged 3 commits into
mainfrom
session-apply-json-manifest
May 19, 2026
Merged

Add declarative JSON-manifest mode: session apply#6
dkijania merged 3 commits into
mainfrom
session-apply-json-manifest

Conversation

@dkijania
Copy link
Copy Markdown
Member

Summary

Adds deb-toolkit session apply <session-dir> <plan.json>, a declarative twin of the existing CLI verb chain. Every CLI verb has an equivalent manifest step, and both modes dispatch into the same Session methods — anything achievable in one is achievable in the other.

Why

The existing verbs are great for interactive / one-off use, but when the transformation itself is the artifact (CI pipelines, hardfork release engineering, anything reviewed in a PR) a declarative manifest is a much better fit:

  • One process, one outcome — the plan either completes or aborts at the failing step.
  • Self-documenting — the manifest IS the change list. No wrapper bash script to grep through.
  • Path-relative — local source paths are resolved against the manifest's own directory, so a folder containing plan.json + its referenced data files is a portable, hashable bundle.
Situation Mode
Interactive poking, one-off ops CLI verbs
Wrapper bash scripts on dev boxes CLI verbs
CI-driven hardfork generation JSON manifest
Anything reviewed in a PR JSON manifest
Reproducible release engineering JSON manifest

Schema (informal)

```json
{
"description": "Generate mina-hardfork-mainnet from a 3.0.0devnet base",
"steps": [
{ "op": "remove", "pattern": "/var/lib/coda/genesis_ledger_devnet.tar.gz" },
{ "op": "insert", "dest": "/var/lib/coda/genesis_ledger.tar.gz",
"sources": ["./genesis_ledger.tar.gz"] },
{ "op": "rename-package", "new_name": "mina-hardfork-mainnet" },
{ "op": "reversion", "new_version": "4.0.0", "update_deps": true },
{ "op": "replace-suite", "new_suite": "umt" },
{ "op": "read-field", "field": "Version", "expected": "4.0.0" }
]
}
```

Full reference: docs/session-manifest.md.

Notable design choices

  • read-field doubles as an assertion primitive — when expected is set, the step fails unless the actual value matches. Lets CI plans pin Version after a reversion step to catch regressions in the verb itself before they ship.
  • serde(deny_unknown_fields) everywhere — a typo in a manifest fails at parse time rather than silently changing behaviour.
  • No automatic rollback on failure — session mutations are cheap (temp dir) and the original .deb is never touched until session save, so a half-applied session is discardable. Persistent step-state tracking would add complexity for a rarely-exercised "resume" path. Documented in docs/session-manifest.md#failure-semantics.
  • Local-source-paths resolve against the manifest dir, not cwd — makes a plan.json + data-files directory a self-contained portable bundle (terraform / docker-compose convention). Package paths (dest, pattern) are unaffected and remain anchored at /.

Documentation added (the user explicitly asked for this to be widely documented)

Tests

  • 7 new unit tests in apply.rs covering schema validation (deny_unknown_fields, default values, unknown ops, path resolution).
  • tests/session_apply.rs — three integration tests:
    • Full hardfork manifest applied end-to-end, including verification that bundle-relative source paths resolve correctly when the manifest lives in its own directory.
    • read-field assertion failure aborts the plan at the failing step (earlier steps remain applied, later steps don't run).
    • Unknown op produces a useful error rather than silently succeeding.

Total: 36 unit + 6 integration tests, all green; clippy and fmt clean.

Test plan

  • CI: full cargo test passes.
  • CI: cargo clippy --no-deps --all-targets -- -D warnings clean.
  • CI: cargo fmt --check clean.
  • Manual: run the example hardfork bundle against a real `mina-mainnet` .deb and verify the produced `mina-hardfork-mainnet` package installs cleanly.

🤖 Generated with Claude Code

# Feature: `session apply <session-dir> <plan.json>`

A declarative twin of the existing CLI verb chain. Every CLI verb has
an equivalent manifest step, and both modes dispatch into the same
`Session` methods — anything achievable in one is achievable in the
other. Manifests win whenever the *transformation itself* is the
artifact you want to check in, code-review, and run reproducibly:

  * One process, one outcome — the plan either completes or aborts at
    the failing step. The original .deb is never touched until
    `session save`, so a half-applied session is discardable.
  * Self-documenting — the manifest IS the change list.
  * Path-relative — local source paths in the manifest are resolved
    against the manifest file's directory, so a folder containing
    `plan.json` plus its referenced data files is a portable bundle.

Schema (full reference in docs/session-manifest.md):

    { "description": "...",
      "steps": [
        { "op": "rename-package", "new_name": "..." },
        { "op": "replace-suite",  "new_suite": "..." },
        { "op": "reversion", "new_version": "...", "update_deps": true },
        { "op": "insert", "dest": "...", "sources": [...], "directory": false },
        { "op": "remove", "pattern": "..." },
        { "op": "move",   "source": "...", "destination": "..." },
        { "op": "replace", "pattern": "...", "replacement": "..." },
        { "op": "read-field", "field": "...", "expected": "..." }
      ] }

`read-field` doubles as an assertion primitive — when `expected` is
set, the step fails unless the actual value matches. Lets CI plans pin
Version after a reversion step.

Strict parsing: unknown ops, unknown fields, and missing required
fields all fail at parse time (`serde(deny_unknown_fields)`).

# Test-infrastructure refactor

The session integration tests had grown a lot of inline boilerplate
(`Command::new("dpkg-deb")`, inline `have()`/`run()`/`dpkg_info`/
`dpkg_contents` helpers duplicated across files, ~50-line fixture
construction per test). This commit extracts that into a shared
`tests/common/` module:

  * `Toolkit` — typed wrapper around the deb-toolkit binary with a
    method per session verb (`session_open`, `session_rename_package`,
    etc.). Returns `CmdOutput` for assertion chaining.
  * `DebFixture` — fluent builder that materializes a `.deb` on disk
    via `dpkg-deb --build`. Replaces the inline fs::write + dpkg-deb
    shell-out boilerplate.
  * `dpkg::info()` / `dpkg::contents()` — assertion helpers.
  * `skip_unless!("dpkg-deb")` macro — uniform skip-when-tool-missing.

`tests/session_roundtrip.rs` and `tests/session_apply.rs` are rewritten
on top of these helpers and read top-to-bottom as narratives now.

# Generic naming

The previous PR used mina-specific fixtures (`mina-mainnet`,
`/var/lib/coda/genesis_ledger.tar.gz`, `mina-hardfork-mainnet`) which
biased readers toward thinking the session subsystem was mina-only.
All tests, examples, and docs now use a generic `example-app` →
`example-app-variant` rebrand scenario. The same workflow applies to
any project that wants to fork a stable release into a sub-channel;
the bundle README explicitly calls out how to adapt the template.

  * `tests/session_hardfork_scenario.rs` → `tests/session_variant_scenario.rs`
  * `examples/manifests/hardfork-bundle/`  → `examples/manifests/variant-bundle/`
  * `docs/session-manifest.md` examples updated to match.

# Documentation

Per "document very widely":

  * `docs/session-manifest.md` — full schema reference, path-resolution
    rules, failure semantics, idempotency notes, when-to-use-which-mode
    table, design rationale.
  * `examples/manifests/README.md` — example index.
  * `examples/manifests/simple-rename.json` — minimal example.
  * `examples/manifests/variant-bundle/` — complete generic bundle
    (plan.json + README walking through each step + adaptation guide).
  * Module-level doc comment on `src/session/apply.rs` mirroring the
    public docs, with the same rationale.
  * Per-variant doc comments on every `Step` enum variant.
  * `README.md` gains a "Session subsystem" section pointing at all of
    the above.

# Tests
  * 7 new unit tests in `apply.rs` covering schema validation
    (deny_unknown_fields, default values, unknown ops, path resolution).
  * `tests/session_apply.rs` — three integration tests:
      - Full variant-rebrand manifest applied end-to-end, including
        verification that bundle-relative source paths resolve
        correctly when the manifest lives in its own directory.
      - `read-field` assertion failure aborts the plan at the failing
        step (earlier steps remain applied, later steps don't run).
      - Unknown op produces a useful error rather than silently
        succeeding.

Total: 36 unit + 6 integration tests, all green; clippy and fmt clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@dkijania dkijania force-pushed the session-apply-json-manifest branch from a7febad to 22efab9 Compare May 18, 2026 14:12
@dkijania
Copy link
Copy Markdown
Member Author

Force-pushed a refactor addressing the review feedback:

1. Extracted test utilities (tests/common/mod.rs) — every integration test in tests/ now shares:

  • Toolkit — typed wrapper around the deb-toolkit binary with one method per session verb (session_open, session_rename_package, …). Returns a chainable CmdOutput.
  • DebFixture — fluent builder for fixture .deb files. Replaces the inline fs::create_dir_all + fs::write(DEBIAN/control) + dpkg-deb --build boilerplate every test used to carry.
  • dpkg::info() / dpkg::contents() — assertion helpers.
  • skip_unless!("dpkg-deb") macro.

2. Generic naming throughout. Dropped all the mina-specific fixtures and replaced with a project-agnostic example-appexample-app-variant rebrand scenario. Same shape of workflow (rename, reversion, swap data/config, flip suite) but no terminology bias. Affects:

  • tests/session_hardfork_scenario.rstests/session_variant_scenario.rs
  • examples/manifests/hardfork-bundle/examples/manifests/variant-bundle/
  • docs/session-manifest.md examples
  • Module-level docs in src/session/apply.rs

3. Rewrote the existing tests on top of the helpers. Each test now reads top-to-bottom as a narrative — tk.session_rename_package(&session, "...") instead of Command::new(bin).args(["session", "rename-package", ...]).status().unwrap() everywhere. Includes refactoring the already-merged session_roundtrip.rs.

4. The variant-bundle example README explicitly walks through how to adapt the template to a real project — generic enough to be a starting point.

All 36 unit + 6 integration tests still green, clippy and fmt clean.

Validates the manifest against an opened session without committing
any change. Useful for CI gates that want to catch typos and broken
bundle paths before the real apply runs.

What gets checked:
  * Parse — schema is well-formed (deny_unknown_fields, missing
    required fields, wrong types — same as a real run).
  * Local source files exist — every `sources` entry in `insert` and
    every `replacement` in `replace` is resolved against the manifest
    directory and verified.
  * Step shape — `insert` with multiple sources but `directory: false`
    is rejected (since the real run would too).

What's intentionally left to the real run:
  * Glob match counts — `remove` / `replace` patterns are matched
    against the live data tree, so we can't tell at validate-time
    whether they'll match zero files.
  * `read-field` assertions — they depend on the cumulative effect
    of earlier (unapplied) steps.

The session directory is left bit-for-bit unchanged.

Tests:
  * 5 new unit tests in `check_step` covering missing-source,
    existing-source, multi-source-without-directory-flag,
    missing-replacement, and metadata-op no-ops.
  * 2 new integration tests: dry-run on a valid plan leaves every
    control field and data file untouched; dry-run on a plan with a
    missing source file surfaces the error.

docs/session-manifest.md gains a "Dry runs" section spelling out
exactly what is and isn't checked.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@dkijania dkijania mentioned this pull request May 18, 2026
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant