fix(sandbox): auto-retry transient claim failures before showing errored - #5210
Merged
Conversation
Builds on the terminal-claim-failure surfacing: instead of immediately showing the errored card on every terminal claim failure, auto-reprovision in place for the failures that are plausibly transient infra hiccups — scheduling-timeout (capacity), reconciler-error, claim-never-created, and the synthetic 'watcher ended' (unknown). Bounded to MAX_CLAIM_AUTO_RETRIES (2) per boot, then falls through to the errored card + Retry. image-pull-backoff and crash-loop-backoff are excluded: a fresh claim fails identically, so those surface immediately. The retry fires once per failed episode (re-armed when the phase leaves 'failed'), reuses useSandboxStart's inflight dedup, and honors the same userStopped / others-thread gates as auto-start and self-heal. A user-driven retry() resets the budget. While budget remains, deriveStartError keeps the booting overlay (claimRetryExhausted=false) instead of flashing errored. Pure helpers isRetryableClaimFailure + shouldAutoRetryClaim are unit-tested.
pedrofrxncx
enabled auto-merge (squash)
July 24, 2026 17:38
decocms Bot
pushed a commit
that referenced
this pull request
Jul 24, 2026
PR: #5210 fix(sandbox): auto-retry transient claim failures before showing errored Bump type: patch - decocms (apps/api/package.json): 4.123.8 -> 4.123.9 Deploy-Scope: web
pedrofrxncx
added a commit
that referenced
this pull request
Jul 24, 2026
…5221) * fix(sandbox): scope the claim auto-retry budget to the active branch SandboxLifecycleProvider lives above Chat.ActiveTaskProvider and is not remounted on a task switch, but its bounded claim-auto-retry budget (added in #5210) was tracked in plain refs shared across the whole provider lifetime. Switching to a different task/branch after one branch's boot exhausted its retry budget carried that exhausted count into the new branch's fresh boot, so a transient claim failure there skipped auto-retry and surfaced the errored card immediately instead of retrying. Replaced the two refs with a small piece of state keyed by branch, reconciled via the new pure `reconcileClaimRetryEpisode` (reset to a fresh budget whenever the branch differs from the tracked one), and added unit tests for it. * fix --------- Co-authored-by: Pedro França <pedrofrxncx@deco.cx>
pedrofrxncx
pushed a commit
that referenced
this pull request
Jul 27, 2026
#5240) Reverts #5112 (share agent sandboxes by branch), #5116 (always share agent sandboxes) and #5132 (use shared staging branch), restoring the pre-#5112 behaviour: hosted agent sandboxes are per-user again, and a new GitHub-backed thread gets a generated branch instead of `staging`. Reconstructed against current paths rather than reverted: #5120 split apps/mesh into apps/api + apps/web and hoisted branch-name.ts, runtime-defaults.ts and thread/schema.ts into packages/shared, and #5174 deleted vm-events.ts outright, so none of the three commits reverse-apply. Core change: SandboxId goes back to a flat `{ userId, projectRef }` and sandboxIdKey back to `userId:projectRef`, so every hosted claim handle changes. Live shared claims must be drained BEFORE this deploys. Also reverted, because they only exist inside the shared model: - #5225 withoutLegacyAgentSandboxEntries — left in place it would strip the sandboxMap entries the reverted server writes back, leaving shouldAutoStart permanently true (silent, compiles fine) - #5219 resolveSharedThreadVm and the shared-scope branch - #5183 stale-provisioning reset (lived inside the deleted storage) Deliberately preserved: - migrations 137/138 and their index.ts entries — Kysely's #ensureNoMissingMigrations runs unconditionally, so deleting an applied migration crash-loops every pod. The tables are dropped by a follow-up forward migration after the drain, not by this commit. - #5114/#5118 withClaimGitLock, #5126 org-context guard, #5143 analytics gate, #5171 owner-keyed thread graft, #5195/#5207/#5210/#5221 client auto-retry, the daemon status-code train and the settings-validation train - #5112's org-fs least-privilege narrowing and the start dedup key fix - #5116's squashed queue-tray pluralization (en + pt-br) - the remote-branch-name validations #5132 deleted, since this restores the dynamic origin/HEAD lookup that made them load-bearing Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Pedro França <pedrofrxncx@deco.cx>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
#5207 requires a manual Retry for every terminal claim failure, including ones that are just transient infra hiccups (a node had no capacity for a moment, a control-plane blip). Those should self-heal.
What
On a terminal claim
failedphase, auto-reprovision in place for the reasons a fresh claim can plausibly resolve, bounded toMAX_CLAIM_AUTO_RETRIES(2 → 3 total attempts), then fall through to the errored card.ClaimFailureReasonscheduling-timeoutreconciler-errorclaim-never-createdunknown(synthetic "watcher ended")image-pull-backoffcrash-loop-backoffWhile auto-retry budget remains,
deriveStartErrorreturns null (claimRetryExhausted=false) so the booting overlay holds instead of flashing errored between attempts.Correctness details
claimFailureHandledRefguard is set when a retry fires and re-armed only when the phase leavesfailed, so the persistentfailedphase can't re-fire every render once the mutation settles.claimRetryCountRef; a user-drivenretry()resets both refs for a fresh budget.userStopped,autoStartBlocked(others' thread), andisPendingall suppress it.useSandboxStart's module-level inflight dedup, so even a double-fire coalesces to one upstreamSANDBOX_START.Tests
Pure helpers
isRetryableClaimFailure+shouldAutoRetryClaimunit-tested (retryable vs non-retryable reasons; budget exhausted; already-handled; pending / stopped / gated). ExistingderiveStartErrortests updated for the newclaimRetryExhaustedarg + a new keep-booting case.bun test55 pass ·tscclean ·lint0 errors ·fmtclean.Summary by cubic
Automatically retries transient claim failures during sandbox boot before showing the errored card. This self-heals common infra hiccups and reduces user clicks and flicker.
userStoppedand others-thread gates, dedups inflight starts, and resets budget on user Retry; added unit tests for retry logic.Written for commit 418e4fe. Summary will update on new commits.