Skip to content

cli: day-1 bootstrap helpers — config check, agent cert, node pair, plaintext crypt#171

Merged
mbertschler merged 18 commits into
mainfrom
claude/issue-162-bootstrap-helpers
Jul 24, 2026
Merged

cli: day-1 bootstrap helpers — config check, agent cert, node pair, plaintext crypt#171
mbertschler merged 18 commits into
mainfrom
claude/issue-162-bootstrap-helpers

Conversation

@mbertschler

Copy link
Copy Markdown
Owner

Summary

Bootstrap-day helpers from #162 — the friction-densest stretch of the reference-setup walk (F1–F4, F34). Four of the five scoped items land here; listener-less agent mode (F35) is deferred to a small follow-up PR so this one stays reviewable.

Changes

  • squirrel config check (F4, F8, F34, F21) — parses and resolves the whole config (env vars included), stats every volume path and node byte-path, validates offload_requires against target capabilities, and prints the affirmative N volumes, M destinations, K nodes — all resolvable. Strictly read-only (a question per ux-principle 2): it never opens the store or creates a path. Missing/empty volume paths, missing node byte-paths, and structurally-unsatisfiable offload targets are surfaced; problems exit non-zero, advisory warnings do not.
  • squirrel agent cert (F1) — generates an ECDSA self-signed cert+key at the configured [agent.tls] paths and prints the sha256: pin ready to paste into peers' [nodes.X.tls]. Refuses to clobber an existing cert without --force (regenerating changes the fingerprint and breaks every peer that pinned it). The agent now also logs its own fingerprint on the startup line.
  • squirrel node pair <peer> (F3) — see decision below.
  • Plaintext crypt passwords (F2) — see breaking-change note below.

⚠️ Decision to confirm: node pair shape

The issue left "exact shape to be designed in the PR". I implemented the simplest defensible shape: squirrel node pair <peer> emits the matching config halves for both sides of the relationship. It generates the two bearer tokens a bidirectional pair needs and places each in the two slots that must agree ([nodes.X].auth.bearer on one side ≡ [agent.auth.peers.X].bearer on the other), so the four-binding token matrix is correct by construction — killing the F3 error class. This node's endpoint/fingerprint are filled where known; peer-side facts it cannot know (peer endpoint/path/fingerprint, and where the peer mounts this node) are clearly-marked <...> placeholders, fillable via --peer-endpoint / --peer-path / --peer-fingerprint / --local-endpoint. It prints only — it never edits either config, so the operator reviews and pastes.

Alternatives not taken (heavier): writing directly into config files, or a stateful interactive handshake between two running agents. Please confirm this print-both-halves shape before it sets precedent.

⚠️ Breaking change + migration: crypt secrets

Crypt password/password2 are now plaintext by default (a literal or { env = "VAR" }); squirrel obscures them into rclone's on-disk form at load time, retiring the manual rclone obscure step. A config that already holds obscured values will double-obscure and fail to decrypt. Migration, documented in docs/.../encrypted.md and the README:

  • Recommended: put the plaintext in the env var / literal and drop nothing else — squirrel handles the rest.
  • Keep existing obscured values: add obscured = true to the crypt block (a marker key) to render them verbatim.

The obscure helper matches rclone's format (AES-CTR under rclone's public key, base64url) but derives the IV deterministically from the plaintext, so the rendered rclone.conf stays byte-stable and the "unexpected rewrite is a signal" invariant in WriteRcloneConfig holds. Determinism costs nothing: obscure is obfuscation, not encryption (the key is public). Cross-checked against real rclone reveal in a (rclone-gated) test. This reconciles with the parallel #153 obscure work — expect a merge reconciliation if both land.

Testing

  • go vet ./..., go test ./..., golangci-lint run (v2.12.2) — all clean.
  • New tests: obscure round-trip/determinism/rclone-reveal cross-check; plaintext-vs-obscured crypt resolution; cert generation + fingerprint identity with the peer-sync verifier; config check affirmative/empty/missing/offload-unsatisfiable/read-only; agent cert generate/refuse-overwrite/force/no-tls; node pair both-halves cross-match + placeholders + fingerprint embedding.

Scope

Addresses #162 (F1–F4, F34). Listener-less agent mode (F35) follows in a separate PR referencing #162, so this PR intentionally does not close the issue.

🤖 Generated with Claude Code

https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7


Generated by Claude Code

claude added 3 commits July 24, 2026 11:14
Crypt `password`/`password2` are now plaintext by default (a literal or
{ env = "VAR" }); squirrel obscures them into rclone's on-disk form at
load time, retiring the manual `rclone obscure` step. The obscure helper
matches rclone's format (AES-CTR under rclone's public key, base64url)
but derives the IV deterministically from the plaintext so the rendered
rclone.conf stays byte-stable — the "unexpected rewrite is a signal"
invariant WriteRcloneConfig relies on. Determinism costs nothing here:
obscure is obfuscation, not encryption (the key is public).

Breaking change: a config whose values are already obscured must add
`obscured = true` to the crypt block to keep them verbatim, or reveal
them back to plaintext. Docs and the existing crypt tests carry the new
marker.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
`squirrel agent cert` generates an ECDSA self-signed cert+key at the
configured [agent.tls] paths and prints the sha256: pin ready to paste
into peers' [nodes.X.tls] cert_fingerprint. It refuses to clobber an
existing cert without --force, since regenerating changes the
fingerprint and breaks every peer that pinned it.

The agent also logs its own cert fingerprint on the startup line, so an
operator can read the pin straight from the log. The pin is computed the
same way the peer-sync client verifies it (SHA-256 over the leaf DER),
via a shared agent.FingerprintCertFile helper.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
`squirrel config check` parses and resolves the whole config (env vars
included), stats every volume path and node byte-path, validates
offload_requires against target capabilities, and prints the affirmative
"N volumes, M destinations, K nodes — all resolvable" summary. It is
strictly read-only (a question, per ux-principle 2): it never opens the
store or creates a path. Missing/empty volume paths (F4/F8), missing node
byte-paths (F34), and structurally-unsatisfiable offload targets — a
mirror-layout crypt destination (F21) — are surfaced; problems exit
non-zero, advisory warnings do not.

`squirrel node pair <peer>` generates the two bearer tokens a
bidirectional peer relationship needs and emits the matching config
halves for both machines, with each token already placed in the two
slots that must agree — killing the F3 four-binding token-matrix error
class. It fills this node's endpoint/fingerprint where known and leaves
clearly-marked placeholders for the peer-side facts it cannot know. It
prints only; it never edits either config.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
Copilot AI review requested due to automatic review settings July 24, 2026 11:15
Follow the friction log's own convention of striking closed entries with
a PR reference: F1 (agent cert + startup fingerprint), F2 (plaintext
crypt), F3 (node pair), F4 (config check). F34 is only partially closed
— config check surfaces a missing node byte-path, but load-time
validation is still absent — so it is annotated, not struck.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds “day-1 bootstrap” CLI helpers and config ergonomics to reduce setup friction around config validation, agent TLS pinning, node pairing, and crypt secret handling.

Changes:

  • Add squirrel config check for read-only, full config resolution + path/policy validation with an affirmative summary.
  • Add squirrel agent cert and log the agent TLS fingerprint at startup for easy peer pinning.
  • Add squirrel node pair <peer> to emit both sides of a bidirectional peer relationship (tokens/endpoints/fingerprints) and switch crypt password/password2 to plaintext-by-default with an obscured = true migration marker.

Reviewed changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
sync/rclone_test.go Updates test config fixture to include crypt.obscured = true.
README.md Documents plaintext-by-default crypt secrets and the obscured migration marker.
docs/src/content/docs/reference/configuration.md Updates reference docs for crypt secrets + obscured flag.
docs/src/content/docs/layouts/encrypted.md Updates encrypted layout docs and adds migration note for already-obscured configs.
config/obscure.go Implements rclone-compatible “obscure” encoding with deterministic IV derivation.
config/obscure_test.go Adds round-trip/determinism tests and an optional cross-check against rclone reveal.
config/destinations.go Resolves crypt secrets (plaintext→obscured) and supports crypt.obscured passthrough.
config/crypt_plaintext_test.go Tests plaintext→obscured behavior and obscured = true migration behavior.
config/config.go Updates Crypt struct docs to reflect plaintext-by-default + obscured behavior.
config/config_test.go Updates config tests/fixtures to include obscured = true where needed.
cmd/squirrel/root.go Registers new top-level config and node command groups.
cmd/squirrel/node.go Adds squirrel node parent command.
cmd/squirrel/node_pair.go Implements squirrel node pair <peer> which prints both config halves with tokens/placeholders.
cmd/squirrel/node_pair_test.go Tests node pairing output shape, token cross-match, and fingerprint embedding.
cmd/squirrel/config.go Adds squirrel config parent command.
cmd/squirrel/config_check.go Implements squirrel config check validation/statting + summarized outcome.
cmd/squirrel/config_check_test.go Tests config check success/warn/fail paths and “read-only” invariant.
cmd/squirrel/agent.go Adds agent cert subcommand and logs cert fingerprint on startup when TLS is enabled.
cmd/squirrel/agent_cert.go Implements squirrel agent cert to generate cert/key and print pasteable pin snippet.
cmd/squirrel/agent_cert_test.go Tests cert generation, refusal to overwrite, --force, and missing TLS config behavior.
agent/cert.go Adds self-signed cert generation + fingerprinting helpers used by CLI and server logging.
agent/cert_test.go Tests cert/key validity, fingerprint correctness/shape, uniqueness, and file fingerprinting.
agent/agent.go Adds Server.CertFingerprint() accessor used for startup logging.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread config/destinations.go
Comment thread config/obscure_test.go Outdated
Comment thread agent/cert_test.go
claude added 2 commits July 24, 2026 11:24
- config/obscure_test.go: the IV-uniqueness check now base64-decodes the
  obscured values and compares the actual leading IV bytes (via a new
  obscuredIV helper) instead of a base64 string prefix, and checks the
  obscureRclone errors so a regression can't hide behind zero values.
- agent/cert_test.go: TestGenerateSelfSignedCertUnique now checks the
  errors from GenerateSelfSignedCert rather than comparing zero-value
  fingerprints on a silent failure.
- config/destinations.go: cryptObscuredFlag's error is wrapped with the
  "crypt:" prefix at the call site, matching the other resolveCrypt
  errors (unknown field, secret resolution); the test asserts the
  prefixed message.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
Make the HTTP listener optional. When `[agent] listen` is absent/empty the
agent runs its background schedulers (index/sync/audit cadences) WITHOUT
binding an HTTP server — the mode for cadence-only machines (a roaming
laptop that pushes to a hub and never receives peer syncs). When `listen`
is set, behaviour is unchanged: scheduler + HTTP server, token required.

- config: `listen` no longer required; a bearer token is required only
  when a listener is configured (nothing to authenticate otherwise).
- agent: validateConfig relaxed accordingly; new Server.RunSchedulers runs
  the scan + cadence loops with the same shutdown discipline as Serve but
  no listener.
- cmd: runAgent dispatches to RunSchedulers when listen is empty, and
  fails fast if a listener-less agent has no cadence and no scan (silent
  idling is degradation, not a valid config); the marker/init boundary is
  untouched — the agent still never escalates.
- docs: reference-setup laptop drops the dummy listen + token; agent guide
  and configuration reference describe listener-less operation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
claude added 5 commits July 24, 2026 11:36
Strike the listener-less friction entry now that the cadence-only agent
mode lands, per the friction log's PR-reference convention.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
Add ResetDestination: an operator-driven verb that forgets a destination's
recorded remote state (remote_objects, remote_packs, the durability vector,
and push-freshness) so the next sync treats it as fresh and re-uploads. It
is the supported recovery for a wrecked or repointed destination, replacing
hand SQL or a cross-machine rename.

The reset is audit-preserving: the runs table and the append-only
destination_run_ids_history advance log are untouched, and the content/files
rows are never modified. The reset itself is recorded as a kind='audit' run
(reusing the destination-scoped audit shape, so no schema change) with a new
free-text 'reset-destination' runs_audit transition carrying the cleared
counts. Everything runs in one transaction — a reset is all-or-nothing.

CountDestinationRecordedState backs the CLI's dry-run preview.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
The packed and content-addressed layout guards keyed on run history by
destination name: a recorded success whose manifest segment / pack map is
absent was refused outright, so following the guard's own advice (a fresh
root) still refused. Now, when the marker is missing, the guard probes the
configured root; an empty root — a wiped or repointed destination, or one
cleared by `squirrel destination reset` — is treated as a fresh start and
re-uploaded. A non-empty root, or any error probing it, keeps the refusal
(fail-closed). The refusal message now also points at `destination reset`.

Adds Rclone.remoteRootEmpty (recursive lsf) and a shared freshStartOnEmptyRoot
helper. The fake-rclone test shim learns `lsf`; the mirror-era guard tests now
seed a remote file so they exercise a genuine layout conflict, and new tests
cover the empty-root fresh start on both layouts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
`squirrel destination reset <name>` is a weighty change command: it prints
what it will clear, refuses without --yes, and offers --dry-run — mirroring
ux-principle 2. Parent `destination` command and the `reset` subcommand live
in separate files. A configured-but-unrecorded destination reports "nothing
to reset" rather than minting an empty run.

restore's node dead-end ("restore from node destinations is not supported")
now points at the machine-replacement runbook — rebuilding an edge machine
from its hub is a reverse peer push driven from the hub, not a restore.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
New "Recovery & disaster runbooks" guide: resetting a wrecked destination,
rebuilding a dead machine from its hub via a reverse peer push (re-pairing
with `node pair`), and the manual DR that already works and stays — mirror
restore, ride-along index-snapshot recovery, and packed/content-addressed
recovery. Adds `squirrel destination reset` to the CLI reference and the
guide to the sidebar.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
claude and others added 7 commits July 24, 2026 11:40
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
- config/agent.go: the "token required" error now states the requirement
  is conditional on a listener — "auth.token is required when [agent]
  listen is set (... omit listen for a listener-less, scheduler-only
  agent)" — instead of the unconditional "no agent without
  authentication", which read as wrong now that listener-less mode exists.
- cmd/squirrel/agent.go: the listener-less "nothing to run" fail-fast now
  lists hook.interval alongside sync_every/index_every/scan_interval, so
  the remediation matches the real scheduling rules (agentHasWork and the
  scheduler both count an interval hook as work).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
FIX 1 (fail-closed): remoteRootEmpty no longer reads any lsf error with
empty stdout as an empty root — runPlain carries stderr only in the error,
so an auth/network failure was being mistaken for a wiped destination and
letting the layout guard proceed. Now only rclone's canonical
"directory not found" (or "object not found") counts as a fresh root; every
other error surfaces so the guard fails closed. New test injects an
auth-style lsf error (guard refuses) vs a not-found (fresh root).

FIX 2: the restore node-restore signpost pointed at a repo source path
(docs/guides/recovery) that doesn't exist for an installed binary; it now
names the published docs route (guides/recovery).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
F35 made Listen optional and Token conditional on Listen, but three
doc comments on the public Config/Agent structs still declared them
unconditionally required, contradicting the relaxed validateConfig /
resolveAgentAuth. Correct the comments so the documented contract
matches the validated one. Doc-only; no behaviour change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Skm2Qm6wWUB7dD4CpPTz7
Textual conflicts:
- cmd/squirrel/root.go: keep this branch's config/node/destination
  commands alongside main's `squirrel version`.
- cmd/squirrel/agent.go: keep the startup cert-fingerprint attributes (F1)
  on top of main's `version` var, which replaced this branch's local
  `agentVersion` const.
- sync/{content_addressed,packed}_test.go: keep both sides' tests — this
  branch's F20 fresh-start cases and main's dry-run previews (which
  replaced the dry-run-refused cases), plus both sets of fake-rclone env
  resets.

Semantic conflicts the merge exposed:
- Duplicate rclone-obscure helpers: main's config/rclone_obscure.go
  (#153) and this branch's config/obscure.go declared the same symbol.
  Consolidated on main's rcloneObscure — it is the merged, golden-pinned
  implementation — and repointed resolveCryptSecret at it (it can no
  longer fail, so the error path went away). Ported this branch's unique
  coverage, the cross-check against a real `rclone reveal`, into main's
  test file; the IV-distinctness assertion could not come along, as main
  deliberately fixes the IV at zero and documents that trade-off.
- The remote-marker gate (#150/#170) writes .squirrel-volume under the
  destination root, which made this branch's "empty root means fresh
  start" probe (F20) unreachable: a wiped root is re-bootstrapped with a
  marker before any layout guard runs, so it never looks empty again.
  remoteRootEmpty now skips volume markers — they are squirrel's own
  bookkeeping, not content. Anything else still reads as non-empty and
  keeps the guard's refusal. Main's two dry-run layout-flip tests now
  seed mirror-era bytes, the same adaptation this branch already applied
  to their non-dry-run siblings, so they still describe a genuine layout
  conflict rather than a wiped destination.
- The fake rclone shim's `lsf` returned underlying names through a crypt
  overlay; real rclone reports decrypted ones. It now strips the data
  suffix, so a name-matching caller sees what rclone would.
- F21 is implemented on both sides: main rejects an unsatisfiable
  offload_requires at config load, ahead of `config check`'s own
  classification. Its test now pins the surviving contract — name the
  target, explain it can never gate, exit non-zero.

go vet, go test ./..., and gofmt are clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQUzRmQHqbP2kV9kFrbBHL
@mbertschler
mbertschler merged commit 85431d7 into main Jul 24, 2026
3 checks passed
@mbertschler
mbertschler deleted the claude/issue-162-bootstrap-helpers branch July 24, 2026 21:55
mbertschler pushed a commit that referenced this pull request Jul 24, 2026
main now carries #171, so its bootstrap-helper and fresh-start work meets
this branch's refusal/audit work.

- sync/content_addressed.go, sync/packed.go: the layout guards had both
  changed — main added the fresh-start-on-empty-root recognition and the
  `destination reset` hint, this branch wrapped the refusal in ErrRefused
  so it records a terminal 'refused' run. Composed: an empty root is still
  a fresh start, and a genuine layout conflict refuses with both the reset
  hint and the sentinel.
- store/runs_audit.go, cmd/squirrel/agent.go: both sides appended at the
  same spot — keep this branch's abort/alarm/contested transitions and the
  startup reaper alongside main's reset-destination transition and its
  listener-less helpers.

The reaper still runs in runAgent ahead of serveAgent, so it covers both
the HTTP and the new listener-less path.

go vet, go test ./..., and gofmt are clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQUzRmQHqbP2kV9kFrbBHL
mbertschler pushed a commit that referenced this pull request Jul 24, 2026
main now carries #171 and the listener-less agent mode (#175), which is
where this branch's two new cadences had to be reconciled.

Textual conflicts:
- store/runs_audit.go, agent/agent.go: both sides appended at the same
  spot — keep this branch's pull-durability transition and its
  VerifyRunner/DurabilityPuller types alongside main's reset-destination
  transition; main rewrote the Config doc comment for listener-less mode,
  so its version of that comment stands.
- docs reference: both sides added to the [agent] section. Kept main's
  listener-less paragraph and this branch's key tables, and corrected the
  table to match — `listen` is no longer required, and `auth.token` is
  required only alongside it. Left as-is the two would have contradicted
  each other in adjacent paragraphs.

Semantic conflict: main's listener-less start gate (cmd's agentHasWork)
duplicated the scheduler's volume-cadence logic and predated this branch's
verify_every / pull_durability_every, so it would have refused to start a
listener-less agent whose only work is one of those cadences — exactly the
receive-only htpc in design/reference-setup.md that F33 exists to serve.
Replaced the duplicate with an exported agent.ScheduledWorkInConfig, built
on the same resolvers the scheduler's own anyScheduledWork gate uses, so
the two cannot drift again; the refusal message now names every cadence.
Covered by a table test including the receive-only pull-only config.

go vet, go test ./..., and gofmt are clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQUzRmQHqbP2kV9kFrbBHL
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants