Skip to content

fix(dispatch): make SSH CLI install work and return snapshot results - #1893

Merged
bobleer merged 1 commit into
GCWing:mainfrom
bobleer:bob/dispatch-optimization-333fe1
Jul 30, 2026
Merged

fix(dispatch): make SSH CLI install work and return snapshot results#1893
bobleer merged 1 commit into
GCWing:mainfrom
bobleer:bob/dispatch-optimization-333fe1

Conversation

@bobleer

@bobleer bobleer commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Preparing an SSH dispatch target failed for every host, and a finished snapshot job had no way to give its work back.

Every fresh install failed, with an unreadable error

bitfun-cli is a shim that resolves the real binary as its own sibling via current_exe(). The installer staged the pair under dotfile temp names in ~/.local/bin, then smoke-tested the shim before committing — at which point no file named bitfun existed yet. The shim exited 1 and the install rolled back with staged bitfun-cli companion did not run, and the check discarded stderr, so the one actionable line never surfaced.

Staging now happens in a private subdirectory under the real filenames: sibling resolution holds, the commit stays a same-filesystem atomic rename, and both companion checks report the loader's own message.

This also closes a silent hole. On an upgrade the old check "passed" by exec'ing the previous binary, so it never validated the new one at all.

A build without dispatch installed as if it had worked

This installer exists only to serve dispatch, so dispatch --help is verified before anything is replaced. Without that gate, installing a release predating the feature succeeds, the target is then reported incompatible, and the user reinstalls forever.

For the same reason, a target already running the release version is no longer offered that install. The message carries the probe's own error, so a genuinely feature-less release is distinguishable from a transient failure.

Note: no published release currently contains dispatch — v0.2.14 predates it (git ls-tree v0.2.14 has zero files under src/apps/cli/src/dispatch/). This gate makes that state report honestly instead of looping; a release carrying dispatch is still required before the prebuilt path can produce a working target.

The controller's uplink was the only path for the archive

The target now fetches the release itself when it can, falling back to the SFTP push on any failure.

A target has no minisign and no trust root, so it only ever checks a plain SHA-256. That is sound only because this machine verifies the signature over the tiny .sha256 sidecar first and hands down a proven digest — the pattern already used for relay deploys. When that sidecar is unsigned the digest is unauthenticated and the archive's own signature becomes the sole protection, so the bytes must flow through the controller: that case is refused, not downgraded.

Unsupported hosts got a flat error

Probe now reports libc family, glibc version, toolchain and free space, and names the actual reason (unsupported platform, musl, glibc below the 2.35 release floor).

A source build is offered whenever no prebuilt install can proceed, listing every missing prerequisite at once. It reuses the same staging, smoke-test and commit path so rollback semantics cannot drift, and it never installs a toolchain on someone's server.

Results were stranded on the target

The design doc says the UI names the managed path and that applying results stays a confirmed local operation. Neither existed.

The delivered manifest is persisted as a baseline, a workspace-result verb diffs the terminal tree against it, and the changed files come back as a bundle. The diff is content-addressed from the manifest's per-file digests, so it works for workspaces that are not git repositories.

Nothing is written locally until the user reviews the list. When a path moved on both sides the apply aborts and reports the conflict rather than choosing a winner; taking the target's version is a separate explicit confirmation. workspace_result_bundle is advertised as an optional capability so older targets stay usable.

The dialog had no information hierarchy

Two type sizes, no heading elements, three approval cards crushed into a 560px modal, and the consent gate coloured identically to the install action. Rebuilt with real heading levels, sectioned cards, single-column options, warning colour reserved for consent, and a pinned footer over an independently scrolling body.

Verification

Against a real Ubuntu 24.04 / glibc 2.39 target:

Check Result
Fresh install on a bare host succeeds; both entrypoints run
Upgrade over an existing install atomic, both entrypoints run
Rollback on a broken archive previous binary byte-identical, exit recorded
Target-side download of the real 35 MB release ~3s, digest verified, .part cleaned
Digest mismatch rejected, nothing published
Dispatch-support gate vs. real v0.2.14 refused, existing install byte-identical

The snapshot → commit → result round trip was driven through the real CLI verbs, confirming the baseline manifest is persisted and the diff is correct (including that a rewrite with identical bytes is not reported as a change).

Automated: 119 services-integrations, 12 dispatch_workspace, 54 CLI, and 1318 web-ui tests. Clippy clean on all changed crates.

Known gaps

  • The source build pins to the release tag, so it only yields a dispatch-capable CLI once a release carrying dispatch exists.
  • The source build path itself was not executed end to end: the test host has no Rust toolchain, and installing one there was not mine to decide.

Preparing an SSH dispatch target failed for every host, and a finished
snapshot job had no way to give its work back.

**Every fresh install failed, with an unreadable error.** `bitfun-cli` is a
shim that resolves the real binary as its own sibling via `current_exe()`.
The installer staged the pair under dotfile temp names in `~/.local/bin`,
then smoke-tested the shim *before* committing — at which point no file named
`bitfun` existed yet, so the shim exited 1 and the install rolled back with
`staged bitfun-cli companion did not run`. The check discarded stderr, so the
one actionable line never surfaced. Stage into a private subdirectory under
their real filenames instead: sibling resolution then holds, the commit stays
a same-filesystem atomic rename, and both companion checks now report the
loader's own message. This also fixes a silent hole — on an upgrade the old
check "passed" by exec'ing the *previous* binary, so it never validated the
new one at all.

**A build without dispatch installed as if it had worked.** This installer
exists only to serve dispatch, so verify `dispatch --help` before replacing
anything. Without it, installing a release that predates the feature succeeds,
the target is then reported incompatible, and the user reinstalls forever. For
the same reason, a target already running the release version is no longer
offered that install; the message carries the probe's own error so a genuinely
feature-less release is distinguishable from a transient failure.

**The controller's uplink was the only path for the archive.** Let the target
fetch the release itself when it can, falling back to the SFTP push on any
failure. A target has no minisign and no trust root, so it only ever checks a
plain SHA-256 — sound only because this machine verifies the signature over
the tiny `.sha256` sidecar first and hands down a proven digest, the pattern
already used for relay deploys. When that sidecar is unsigned the digest is
unauthenticated and the archive's own signature becomes the sole protection,
so the bytes must flow through here: that case is refused, not downgraded.

**Unsupported hosts got a flat error.** Probe libc family, glibc version,
toolchain and free space, and name the actual reason (unsupported platform,
musl, glibc below the 2.35 release floor). Offer a source build whenever no
prebuilt install can proceed, listing every missing prerequisite at once. It
reuses the same staging, smoke-test and commit path, so rollback semantics
cannot drift; it never installs a toolchain on someone's server.

**Results were stranded on the target.** The design contract says the UI names
the managed path and that applying results stays a confirmed local operation;
neither existed. Persist the delivered manifest as the baseline, add a
`workspace-result` verb that diffs the terminal tree against it, and pull the
changed files as a bundle. The diff is content-addressed from the manifest's
per-file digests, so it works for workspaces that are not git repositories.
Nothing is written locally until the user reviews the list: when a path moved
on both sides the apply aborts and reports the conflict rather than choosing a
winner. `workspace_result_bundle` is advertised as an optional capability so
older targets stay usable.

**The dialog had no information hierarchy.** Two type sizes, no heading
elements, three approval cards crushed into a 560px modal, and the consent
gate coloured identically to the install action. Rebuild it with real heading
levels, sectioned cards, single-column options, warning colour reserved for
consent, and a pinned footer over an independently scrolling body.

Verified against a real Ubuntu 24.04 target: fresh install, upgrade, rollback
on a broken archive, target-side download of the real 35 MB release in ~3s,
digest-mismatch rejection, and the dispatch-support gate refusing the real
v0.2.14 release while leaving the existing install byte-identical. The
snapshot -> commit -> result round trip was driven through the real CLI verbs.
@bobleer
bobleer force-pushed the bob/dispatch-optimization-333fe1 branch from 616df09 to 22c1fe1 Compare July 30, 2026 10:56
@bobleer
bobleer merged commit 94b376b into GCWing:main Jul 30, 2026
7 checks passed
bobleer added a commit that referenced this pull request Jul 30, 2026
Four loose ends in the result-return path shipped in #1893, found by
re-reading that change rather than by a failure.

**An older target failed with clap's own error.** `workspace_result_bundle` is
advertised and deliberately kept out of `REQUIRED_DISPATCH_CAPABILITIES` so a
CLI that predates it stays fully usable for running jobs — but nothing checked
it, so pulling from such a target surfaced `unrecognized subcommand
'__workspace_result'`, which says nothing about what to do. Probe for the
capability first and fail with the actual remedy. Checked against the live
protocol rather than anything cached, so it cannot go stale.

**The staging directory did not follow its own neighbour's rule.** Result
bundles land beside `.workspace-uploads`, which hardens itself to 0700 even
though the outbound root is already owner-only. `.results` was created with a
bare `create_dir_all` and its bundle and summary written under the process
umask. The parent's 0700 does block traversal today, so this was defence in
depth rather than an open door — but the bundle carries the user's source,
including the ignored files the snapshot deliberately shipped, and the file
beside it records which paths changed. Harden the directory and create both
files 0600 before writing, so their contents are never briefly umask-governed.

**Pulled bundles were never collected.** `remove_workspace_snapshot` clears
only `.workspace-uploads`, so terminal jobs expired after 30 days while their
bundles accumulated forever. Add `remove_result_bundle` and call it from the
same retention sweep. Kept separate from the snapshot removal on purpose: that
one runs as soon as the target durably owns the job, long before the user has
had a chance to look at the results.

**Account devices could receive a snapshot but never return one.** Device
dispatch has a full chunked upload path, yet `pull_result` bailed with
"requires an SSH target", leaving device-dispatched snapshot jobs with no way
home. SSH pulls the bundle over SFTP; a device transport carries JSON only, so
add a bounded read-side verb and stream the same bytes back in base64 chunks —
the mirror of the upload. The verb never rebuilds the bundle, so the digest the
target reported stays the digest the controller verifies, and the reassembled
bytes are checked against it before anything is staged. Both transports record
the same durable summary, so the apply step stays transport-blind.

Verified through the real CLI: the chunked read reassembles a bundle across
five 64-byte chunks with a matching digest, and rejects a zero length, an
oversized length, and an offset past the end. Device streaming and its
digest-mismatch refusal are covered by tests over a stub RPC.

Note: `cargo clippy --all-targets` on services-integrations reports three
pre-existing `octal_escapes` errors in manager.rs:6037, unrelated to this
change and present on main; only `--all-targets` surfaces them.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant