fix(hub): keep the control box's agent credentials current - #274
Conversation
Boxes created through a remote hub came up signed out of Claude. Measured on the live control box: custody held a token that expired at 15:04, and a create at 19:32 overwrote a backup holding a valid one (expiry 01:04) with it. Claude's OAuth refresh ROTATES the refresh token, so an older copy is not merely expired — it is dead, and the box cannot recover from it. Three defects composed into the loop the user saw (push credentials, one box works, the next is broken again): 1. `seedHostBackupsFromCustody` overwrote the host backup unconditionally on every create. It now runs the same newest-wins gate the fanout already uses (`shouldAcceptCredentialUpdate`) and writes atomically via `writeCredentialBackup`. This alone would have prevented the observed clobber. 2. Nothing ever wrote custody back. `CredentialsFanout` already accepts a box's refreshed token into the host backup and fans it to other boxes; it now also records it in custody when the relay owns one, so custody self-heals from the boxes themselves instead of decaying until the next PC push. 3. The agent-command hub path never refreshed custody. `create --via-hub` has always called `syncAgentCredentialsIfChanged`; `agentbox <provider> claude` goes through `_cloud-agent-via-hub.ts`, which did not — so it enqueued against whatever a previous `create` had pushed. Also, since the same investigation turned them up: - A PC destroy left the control box's own `state.json` record and per-box SSH key dir behind, so the box kept rendering as `running` in `hub boxes list`, the dashboard and the tray (a cloud row shows its last known state, it does not probe). The reap endpoint now drops both, scoped to that one sandbox id — never a sweep, because `prune --all` matches box dirs by the docker-shaped `<id>-<n>-<mnemonic>` name and would take a LIVE VPS box's private key. - Hetzner names no longer double the prefix. A control-box box is named after the worker's clone dir, which already starts with `agentbox-`, so the server came out `agentbox-agentbox-hub-worker-<uuid>-…` at exactly 63 characters — flush against Hetzner's ceiling. Deduping returns 9 characters to the name. The credential guard is covered by a test that fails without it (verified by reverting the guard), plus custody write-back, local-reap and dedupe cases. Both new test files redirect HOME first — they read and write real credential backups and state.json. Claude-Session: https://claude.ai/code/session_01RVgJBBAsfaiZcWvZNUTCM5
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 6e0c238. Configure here.
| await this.custody.put( | ||
| `agents/${agent}/${spec.credential.boxRelPath}`, | ||
| Buffer.from(content, 'utf8'), | ||
| ); |
There was a problem hiding this comment.
Custody write-back skips newest-wins
Medium Severity
recordInCustody always puts an accepted box credential into custody, but acceptance is gated only against the control box host backup. A PC syncAgentCredentialsIfChanged can leave custody newer than that backup; a running box’s fanout update then overwrites the fresher custody blob with an older OAuth rotation, undoing the same newest-wins rule seedHostBackupsFromCustody now applies locally.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 6e0c238. Configure here.


Boxes created through a remote hub came up signed out of Claude. Measured on the live control box:
agents/claude/.credentials.json…D18SvwAA~/.agentbox/claude-credentials.json…D18SvwAAHalf an hour before that reading, the same backup held a valid token (
expiresAt 01:04). The 19:32 create overwrote it with custody's 7-hour-old copy. A Claude refresh rotates the refresh token, so the older blob is dead rather than merely expired — the box cannot recover from it.Three defects, one loop
seedHostBackupsFromCustodyoverwrote unconditionally on every create. It now runs the newest-wins gate the fanout already uses (shouldAcceptCredentialUpdate) and writes atomically viawriteCredentialBackup. This alone would have prevented the observed clobber — custody's15:04against a backup at01:04yields "not newer than backup".CredentialsFanoutalready accepts a box's refreshed token into the host backup and fans it to other boxes; it now also records it in custody when the relay owns one (one-line wiring —opts.custodyis already aRelayServerOptionsfield). Custody self-heals from the boxes instead of decaying until the next PC push.create --via-hubhas always calledsyncAgentCredentialsIfChanged;agentbox <provider> claudegoes through_cloud-agent-via-hub.ts, which did not.Daytona looked immune only because its credentials live on a volume the in-box agent keeps refreshing; e2b and vercel are affected identically.
Also, from the same investigation
state.jsonrecord and per-box SSH key dir — so the box kept renderingrunninginhub boxes list, the dashboard and the tray (a cloud row shows its last known state, it doesn't probe). The reap endpoint now drops both, scoped to that one sandbox id: never a sweep, becauseprune --allmatches box dirs by the docker-shaped<id>-<n>-<mnemonic>name and would take a live VPS box's private key.agentbox-, so the server came outagentbox-agentbox-hub-worker-<uuid>-…at exactly 63 characters — flush against Hetzner's ceiling. Deduping returns 9 characters.Tests
The credential guard has a test that fails without it (verified by reverting the guard and watching it go red), plus custody write-back, local-reap and dedupe cases. Both new test files redirect
HOMEbefore importing — they read and write real credential backups andstate.json, and the sibling relay tests have no isolation.pnpm build/typecheck(31) /lint(17) /test(30) green.Not verified end-to-end on the live control box yet — that needs this deployed there, then a create without any manual credential push.
https://claude.ai/code/session_01RVgJBBAsfaiZcWvZNUTCM5
Note
Medium Risk
Changes authentication credential flow on the control box and destroy/reap behavior; scope is focused and covered by new tests, but mistakes could still affect box login or leave stale hub state.
Overview
Fixes hub-created boxes coming up signed out of Claude and a few related control-box hygiene issues.
Agent credentials on the control box no longer decay or get clobbered. Before a hub-routed create, foreground/background
claude|codex|opencodepaths now callsyncAgentCredentialsIfChanged(same ascreate --via-hub). On the worker,seedHostBackupsFromCustodyapplies the same newest-wins gate as credential fan-out instead of always overwriting host backups from custody. When a box refreshes a token,CredentialsFanoutmirrors the accepted blob into custody (relay wired withopts.custody), so custody stays current without another PC push.Destroy/reap from a PC now also removes the control box’s own
state.jsonentry and the per-box SSH key directory, so destroyed boxes stop showing asrunningin hub list, dashboard, and tray.Hetzner resource names skip a doubled
agentbox-prefix when the box name already includes it (avoidsagentbox-agentbox-…at the 63-char limit).Docs/changelog updated; new tests cover seed-backup gating, custody write-back, local reap, and prefix dedupe.
Reviewed by Cursor Bugbot for commit 6e0c238. Configure here.