Skip to content

fix(daytona): release fixed leases with API keys and reconcile native expiry - #2111

Merged
steipete merged 3 commits into
mainfrom
codex/daytona-fixed-expiry
Sep 11, 2026
Merged

fix(daytona): release fixed leases with API keys and reconcile native expiry#2111
steipete merged 3 commits into
mainfrom
codex/daytona-fixed-expiry

Conversation

@steipete

@steipete steipete commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Fixed Daytona leases could be created with an API key but could not be released: cleanup called an organization endpoint that accepts browser OAuth credentials only. Native TTL expiry also left an acquired fixed lease permanently unresolved after Daytona removed its sandbox.

Use the authenticated organization returned by the current-key endpoint for API-key cleanup and empty-account acquisition. Preserve resource-backed acquisition for older Daytona servers that omit that field. Reconcile an acquired fixed lease through the existing release owner only after checking its original organization, exact native UUID, and complete failure-inclusive database inventory. Inspection records the released tombstone without issuing a provider deletion; ambiguous creates and incomplete observations retain their claims.

Validation:

  • Reproduced both failures with released Crabbox 0.56.0.
  • Real Daytona CLI proof with the repaired build: fixed acquisition with an API key, explicit and repeated stop, native one-minute TTL expiry, released inspection, and rejection of replaying the retired fixed ID.
  • go test -race -timeout=10m ./internal/providers/daytona -count=1 passed (153.825s); scoped vet and docs generation passed.
  • Independent Codex review found no actionable P0–P2 findings.
  • The OpenClaw registered-backend live scenario also passed against this build: shared lease creation, token refresh, native pause/resume, preserved remote edits, reopen, recreate, and full cleanup.
  • Extended existing HTTP-backed lifecycle coverage for JWT-only organization endpoints, old-server identity compatibility, changed or malformed identity, incomplete inventory, and expired-lease inspection.

This unblocks the lease-backed sandbox integration in openclaw/openclaw#144454. No new configuration, dependency version, or storage format is introduced. Older servers without current-key organization metadata still require an OAuth organization profile for fixed cleanup.

Inspectable live CLI evidence

Repeated against the final production source tree in ca25cc1c02cd15e066813524e893c24a2cd27ede, using an isolated HOME/workspace, the real Daytona API, and an API key. The projection below includes only lease identity, state, readiness, and command exit status; credential-bearing warmup output is omitted. No mocks were used. Both leases were released, and repeated stop also returned zero.

$ crabbox warmup --provider daytona --class small --lease-id cbx_08ef6fc01684 --slug lifecycle-proof --keep --ttl 5m
exit 0
$ crabbox stop cbx_08ef6fc01684
exit 0
$ crabbox inspect --id cbx_08ef6fc01684 --json
{"id": "cbx_08ef6fc01684", "state": "released", "ready": false}
$ crabbox warmup --provider daytona --class small --lease-id cbx_b788ad47f463 --slug lifecycle-proof --keep --ttl 1m
exit 0
Daytona native TTL elapsed; exact native sandbox GET returned HTTP 404.
$ crabbox inspect --id cbx_b788ad47f463 --json
{"id": "cbx_b788ad47f463", "state": "released", "ready": false}
$ crabbox warmup --provider daytona --class small --lease-id cbx_b788ad47f463 --slug lifecycle-proof --keep --ttl 1m
exit 4 (retired fixed ID rejected)
$ crabbox stop cbx_b788ad47f463
exit 0

@clawsweeper

clawsweeper Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@steipete
steipete marked this pull request as ready for review September 11, 2026 18:12
@clawsweeper clawsweeper Bot added P2 Normal priority bug or improvement with limited blast radius. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Sep 11, 2026
@clawsweeper

clawsweeper Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed September 11, 2026, 2:30 PM ET / 18:30 UTC (Revision 2).

ClawSweeper review

What this changes

The PR repairs Daytona API-key cleanup and records acquired fixed leases as released after verified native expiry or external deletion, with lifecycle tests and documentation.

Merge readiness

Blocked before merge - 1 item remains

This fix remains necessary on current main and v0.56.0. No actionable patch defect was found. GitHub confirms the author has repository admin access, so the previous external-contributor proof blocker does not apply.

Priority: P2
Reviewed head: ca25cc1c02cd15e066813524e893c24a2cd27ede

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) Recorded reviewer rating: A focused repair with meaningful lifecycle coverage and no actionable findings; verified admin authorship resolves the previous contributor-proof gate. Real behavior proof remains required by host policy.
Proof confidence 🌊 off-meta tidepool Recorded reviewer rating; Required by policy: the recorded not-applicable assessment does not satisfy the current PR proof policy. Put relevant after-change evidence in the main PR body, then request a fresh review with @clawsweeper re-review. Recorded reviewer context: GitHub confirms repository admin authorship, so ordinary contributor proof is exempt. The body reports repaired Daytona acquisition, stop, expiry inspection, and replay rejection without inspectable output; these claims were not promoted to verified runtime proof. No unresolved authority violation requires an exception to that exemption.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Required by policy Required by policy: the recorded not-applicable assessment does not satisfy the current PR proof policy. Put relevant after-change evidence in the main PR body, then request a fresh review with @clawsweeper re-review. Recorded reviewer context: GitHub confirms repository admin authorship, so ordinary contributor proof is exempt. The body reports repaired Daytona acquisition, stop, expiry inspection, and replay rejection without inspectable output; these claims were not promoted to verified runtime proof. No unresolved authority violation requires an exception to that exemption.
Evidence reviewed 10 items Applicable repository policy: Read the complete root AGENTS.md. The tracked policy inventory contains no applicable nested instructions, and no maintainer-notes directory exists. Provider isolation, credential handling, and changelog ownership informed this review.
Verified maintainer exemption: GitHub returned permission=admin for steipete. This independently establishes maintainer status despite the supplied CONTRIBUTOR association, resolving both the external-contributor proof gate and contributor-only changelog restriction.
Current main and release still need this work: Fetched main still authenticates cleanup through GetOrganization and returns missing-sandbox status errors without expiry reconciliation. Daytona files are unchanged between the pinned base and fetched main. The v0.56.0 status implementation also returns the resolution error directly; GitHub identifies v0.56.0 as the latest release.
Findings None None.
Security None None.

How this fits together

Crabbox’s Daytona adapter translates CLI lease operations into provider requests and durable local ownership records. Cleanup reconciles the exact sandbox before recording a terminal claim that prevents reuse of its fixed ID.

flowchart TD
  A[Stop or inspect request] --> B[Lock local lease claim]
  B --> C[Verify authenticated organization]
  C --> D[Check exact sandbox and database inventory]
  D --> E{Removal verified?}
  E -->|Yes| F[Persist released claim]
  E -->|No| G[Retain claim]
  F --> H[Reject fixed ID reuse]
Loading

Before merge

  • Add real behavior proof - Required by policy: the recorded not-applicable assessment does not satisfy the current PR proof policy. Put relevant after-change evidence in the main PR body, then request a fresh review with @clawsweeper re-review. Recorded reviewer context: GitHub confirms repository admin authorship, so ordinary contributor proof is exempt. The body reports repaired Daytona acquisition, stop, expiry inspection, and replay rejection without inspectable output; these claims were not promoted to verified runtime proof. No unresolved authority violation requires an exception to that exemption.
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Production and test delta Production +67/-14; tests +219/-133 Production growth implements authenticated cleanup and expiry reconciliation while expanding existing lifecycle coverage.

Technical review

Best possible solution:

Keep expiry reconciliation within the existing Daytona release owner, preserving exact-resource checks, legacy acquisition compatibility, and permanent fixed-ID retirement.

Do we have a high-confidence way to reproduce the issue?

Yes, from source: API-key cleanup calls Daytona’s JWT-only organization endpoint, and a missing acquired sandbox remains unresolved on main. The author reports both failures on v0.56.0; this review did not execute them.

Is this the best way to solve the issue?

Yes. The patch repairs authentication inside the provider adapter and reuses existing locked release and terminal-claim machinery without adding competing lifecycle state or changing the stored format.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against cfeb2faacc77.

Labels

Label changes:

  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit. This is the recorded reviewer rating; real behavior proof remains required by host policy.
  • remove rating: 🦪 silver shellfish: Current PR rating is rating: 🐚 platinum hermit, so this older rating label is no longer current.

Label justifications:

  • P2: This repairs cleanup and terminal-state reporting for fixed Daytona leases with a provider-specific blast radius.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit. This is the recorded reviewer rating; real behavior proof remains required by host policy.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Required by policy: the recorded not-applicable assessment does not satisfy the current PR proof policy. Put relevant after-change evidence in the main PR body, then request a fresh review with @clawsweeper re-review. Recorded reviewer context: GitHub confirms repository admin authorship, so ordinary contributor proof is exempt. The body reports repaired Daytona acquisition, stop, expiry inspection, and replay rejection without inspectable output; these claims were not promoted to verified runtime proof. No unresolved authority violation requires an exception to that exemption.

Evidence

What I checked:

  • Applicable repository policy: Read the complete root AGENTS.md. The tracked policy inventory contains no applicable nested instructions, and no maintainer-notes directory exists. Provider isolation, credential handling, and changelog ownership informed this review. (AGENTS.md:1, ca25cc1c02cd)
  • Verified maintainer exemption: GitHub returned permission=admin for steipete. This independently establishes maintainer status despite the supplied CONTRIBUTOR association, resolving both the external-contributor proof gate and contributor-only changelog restriction.
  • Current main and release still need this work: Fetched main still authenticates cleanup through GetOrganization and returns missing-sandbox status errors without expiry reconciliation. Daytona files are unchanged between the pinned base and fetched main. The v0.56.0 status implementation also returns the resolution error directly; GitHub identifies v0.56.0 as the latest release. (internal/providers/daytona/backend_run.go:280, 2c9e78e32f92)
  • Cleanup authority and upgrade compatibility: Release retains the durable claim lock, unchanged-claim check, checkpoint authorization, endpoint binding, and exact resource identity checks. The new absence path requires an acquired claim, matching persisted labels and immutable UUID, authenticated original organization, and complete inventory absence. Inspection cannot issue DELETE. Existing claim fields and terminal serialization are reused without a format migration. (internal/providers/daytona/fixed.go:352, ca25cc1c02cd)
  • Regression coverage: Inspected the expanded HTTP fixtures for empty accounts, legacy metadata, malformed or changed organization identity, native expiry through stop and inspect, retired-ID rejection, and incomplete or failure-containing inventory. Tests were not executed during this read-only review. (internal/providers/daytona/fixed_test.go:619, ca25cc1c02cd)
  • Review continuity and captured proof: The captured body identified by sourceRevision f3f29a4027760f8d39a35e8d596e135e2109de2da5869eb34f06ffad4d49cdb7 reports real Daytona runs but supplies no inspectable transcript. Live body and comment reads added no runtime evidence. The prior request at fix(daytona): release fixed leases with API keys and reconcile native expiry #2111 (comment) is exempted by the independently verified admin status, not treated as completed proof. The latest commit consolidates organization resolution; no prior correctness findings remained. (ca25cc1c02cd)

Likely related people:

  • unknown: The claimed source-line change could not be verified from bounded local history. (role: source history unknown; confidence: low)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (1 earlier review cycle)
  • reviewed 2026-09-11T18:21:27.817Z sha da2ab35 :: needs real behavior proof before merge. :: none

@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Sep 11, 2026
@steipete

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

The main PR body now includes sanitized, inspectable CLI output from a fresh real Daytona run against the final source tree: API-key acquisition and cleanup, native one-minute TTL expiry, released inspection, terminal-ID replay rejection, and repeated cleanup. The source head remains ca25cc1.

@clawsweeper

clawsweeper Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@clawsweeper

clawsweeper Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper status: review started.

I am starting a fresh review of this pull request: fix(daytona): release fixed leases with API keys and reconcile native expiry This is item 1/1 in the current shard. Shard 0/1.

This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking.

Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted.

@steipete
steipete merged commit 2ead1ee into main Sep 11, 2026
28 checks passed
@steipete
steipete deleted the codex/daytona-fixed-expiry branch September 11, 2026 18:47
steipete added a commit to openclaw/openclaw that referenced this pull request Sep 11, 2026
Route Crabbox execution and cleanup through its current repository claim
instead of retained SSH credentials. Replay shared reservations from their
original provider workspace while preserving each caller's workspace context.

Share remote workspace, skills, filesystem, and execution staging between
static SSH and provider command transports. Publish initial workspaces with
native no-replace rename and clean read-only staging directories safely.

Validation: 190 focused tests; selected types, lint, SDK and docs checks;
registered Daytona live flow including ownership transfer and cleanup;
non-root Linux/macOS bootstrap edge proof; independent P2 review clean.

Requires the Crabbox command-ownership contract in
openclaw/crabbox#2119, following the cleanup and TTL
repair in openclaw/crabbox#2111.
steipete added a commit to openclaw/openclaw that referenced this pull request Sep 11, 2026
…tion (#144454)

* feat(crabbox): add a lease-backed sandbox backend for tool-call isolation

Register `agents.defaults.sandbox.backend: "crabbox"` from the existing
Crabbox plugin when `plugins.entries.crabbox.config.sandbox` is present.
The Gateway, agent loop, channels, and model credentials stay on the host;
exec, file tools, and media reads run on a box that Crabbox leases for the
sandbox scope under a fixed, scope-derived lease ID, so restarts and
sibling sessions adopt the same lease instead of allocating another.

The endpoint comes from `crabbox ssh --show-secret`; token-based providers
such as Daytona rotate the SSH user, so the inner SSH backend handle is
rebuilt once the endpoint is older than ten minutes. Everything below the
endpoint reuses the built-in SSH backend and the shared remote-shell
filesystem bridge. Recreate and prune stop the lease.

Crabbox remains the only provider owner: no per-provider plugin, no new
dependency. Not supported: the sandboxed browser and sandbox.docker.binds.

* fix(crabbox): rotate sandbox lease identity and verify recorded host keys

Fixed Crabbox lease IDs are single-use: a stopped lease leaves a terminal
tombstone, so deriving the ID from the sandbox scope broke `openclaw
sandbox recreate`. Each runtime generation now mints its own ID and the
sandbox registry carries it, so restarts adopt the live lease and recreate
provisions under a fresh one.

Crabbox connects with its own SSH client, so its per-lease known_hosts may
not hold the OpenSSH entry yet. Record the host key on first contact and
run the SSH backend with strict checking against that file instead of
disabling verification, so a later impostor cannot receive the token
carried in the SSH user.

* fix(crabbox): treat terminal fixed leases as replaceable during adoption

Crabbox reports a stopped fixed lease as "has no active create attempt; it
cannot allocate a replacement". Recognize that wording so recreate mints a
fresh lease instead of failing; unknown inspection outcomes still propagate.

* fix(crabbox): make sandbox lease lifecycle durable

Reserve runtime generations before provisioning, preserve one SSH workspace owner across credential refreshes, and retain runtime authority through deferred execution and upload admission. Keep failed provisioning and cleanup recoverable under the original lease identity and workspace.

* fix(ci): clean sandbox exports and update merge fixtures

* fix(sandbox): retain provider ownership through remote operations

Route Crabbox execution and cleanup through its current repository claim
instead of retained SSH credentials. Replay shared reservations from their
original provider workspace while preserving each caller's workspace context.

Share remote workspace, skills, filesystem, and execution staging between
static SSH and provider command transports. Publish initial workspaces with
native no-replace rename and clean read-only staging directories safely.

Validation: 190 focused tests; selected types, lint, SDK and docs checks;
registered Daytona live flow including ownership transfer and cleanup;
non-root Linux/macOS bootstrap edge proof; independent P2 review clean.

Requires the Crabbox command-ownership contract in
openclaw/crabbox#2119, following the cleanup and TTL
repair in openclaw/crabbox#2111.

* fix(sandbox): retain ownership of prepared execution cleanup
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal priority bug or improvement with limited blast radius. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant