docs(design): plan warm and resumable Daytona sessions (F-020)#5214
docs(design): plan warm and resumable Daytona sessions (F-020)#5214mmabrouk wants to merge 2 commits into
Conversation
Plan-feature workspace for the two-tier warm-session design on top of PR #5197: Tier 1 parks the sandbox to stopped behind a default-off flag, Tier 2 defers a true running-warm pool. Includes the load-bearing finding that the vendored Daytona provider lacks pause/reconnect hooks, an ask-codex xhigh review round folded into the plan, and the open billing decisions. Claude-Session: https://claude.ai/code/session_018MaXPNpvzN22kngHno3VMj
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Feedback needed on three decisions before implementation starts: (1) the crash-orphan compute budget and the DAYTONA_AUTOSTOP value (5 min can stop a sandbox under a live long tool call; the old default was 15) — needs the billing owner; (2) where the sandbox-pointer fence lives (session_states generation column vs the existing turn counter vs the Redis owner claim) — see open-questions.md #3; (3) whether Tier 2 should ship at all before Tier 1's real second-turn latency is measured on E3. Everything else in plan.md is ready to implement once these are settled. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 62da98bc-f563-4773-a4f5-c26e6426585e
📒 Files selected for processing (6)
docs/design/agent-workflows/projects/warm-daytona-sessions/README.mddocs/design/agent-workflows/projects/warm-daytona-sessions/context.mddocs/design/agent-workflows/projects/warm-daytona-sessions/open-questions.mddocs/design/agent-workflows/projects/warm-daytona-sessions/plan.mddocs/design/agent-workflows/projects/warm-daytona-sessions/research.mddocs/design/agent-workflows/projects/warm-daytona-sessions/status.md
| 6. The reaper cascade is the sweeper, and the orphan budget is a decision. Once a sandbox | ||
| survives turn end, Daytona's own timers reap abandonment: autoStop (5 min) stops a warm | ||
| sandbox, autoArchive (15) colds it, autoDelete (30) deletes it. No runner cron is needed. | ||
| Two open points for a billing owner: | ||
| - Crash orphans. `ephemeral: true` auto-deleted on stop, so a crashed runner leaked little. | ||
| Now a SIGKILL'd runner leaves a running sandbox billing compute for up to 5 idle minutes, | ||
| then storage until autoDelete. Note the API-side `orphan_sweep.py` does NOT help here: it | ||
| only cleans Postgres rows and Redis locks and never contacts Daytona. If the 5-minute | ||
| compute budget is unacceptable, the options are a lower `DAYTONA_AUTOSTOP` or a new | ||
| provider-side sweeper; do not lean on the existing sweep task. | ||
| - Auto-stop can fire mid-turn. Daytona's inactivity clock resets on external API | ||
| interactions, not on processes running inside the sandbox. A long silent tool call or an | ||
| unanswered approval gate can outlast a 5-minute autoStop and the sandbox stops under a live | ||
| turn. The old default was 15. Compare the autoStop value against the runner's maximum | ||
| silent interval (the 300s run-limits guard is one bound) before keeping 5. |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift
Make the auto-stop decision a Tier 1 enablement blocker.
With a 5-minute auto-stop and a 300-second maximum silent interval, Daytona can stop a still-live tool call or approval wait. Before enabling Tier 1, either set auto-stop above the maximum silent interval with margin or add a reliable keepalive mechanism and test this boundary explicitly.
| 7. Stale stored-id hygiene. When autoDelete reaps a sandbox, `session_states.sandbox_id` goes | ||
| stale. The ladder degrades gracefully (failed get, fresh create), but the doomed reconnect | ||
| repeats every turn. Clear the stored id on a failed reconnect, conditionally (see item 3). |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
Do not clear the stored ID for every reconnect failure.
Only confirmed terminal cases such as not-found, deleted, or unrecoverable error states should clear the pointer. Transient API failures and transitional-state timeouts should retain it for retry; otherwise one temporary failure forces a cold create and can overwrite a still-recoverable sandbox.
| 4. Eviction wired to the lifecycle, with awaited teardown. On TTL expiry or LRU eviction, a | ||
| Daytona entry stops the sandbox (drop to Tier 1 warm) rather than deletes it, so the session | ||
| can still reconnect after the live window closes. And unlike the local pool's fire-and-forget | ||
| LRU destroy (fine for a soft RAM cache), the Daytona pool must await the stop before | ||
| admitting a replacement, or the cap does not actually bound running sandboxes: the old one | ||
| still bills while the new one starts, and a failed stop exceeds the cap indefinitely. A | ||
| failed stop escalates to delete. |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift
Bound and verify stop-before-replacement.
Awaiting stop() is insufficient if the provider call hangs or returns before the sandbox reaches stopped. Add a deadline, poll/reconcile the actual state, and admit a replacement only after the old sandbox is stopped or deleted; otherwise MAX_RUNNING cannot reliably cap billed sandboxes.
| | State | How reached | Disk | Billing | Resume cost | | ||
| |---|---|---|---|---| | ||
| | Running (hot) | create, or start a stopped one | live | full compute | none, already up | | ||
| | Stopped (warm) | `pauseSandbox()` / autoStop after 5 min idle | retained | storage only, no compute | fast start (about 1s) + remount + session load | | ||
| | Archived (cold) | autoArchive after 15 min idle | moved to cold storage | cheaper cold storage | slower restore + remount + load | | ||
| | Deleted (dead) | autoDelete after 30 min idle, or `destroySandbox()` | gone | none | full cold create + mount + replay | | ||
|
|
||
| Tier 1 parks to Stopped: storage cost only, and the autoArchive then autoDelete cascade bounds | ||
| even that. Tier 2 parks to Running for a short TTL: the parked cost is live compute, so the TTL | ||
| is a direct billing knob, and the pool cap bounds the concurrent spend. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Show the relevant section with line numbers.
sed -n '170,220p' docs/design/agent-workflows/projects/warm-daytona-sessions/research.md
printf '\n---\n'
# Search for measurement / latency claims in the document.
rg -n "measure|measured|latenc|resume cost|about 1s|session/load|mount" docs/design/agent-workflows/projects/warm-daytona-sessions/research.mdRepository: Agenta-AI/agenta
Length of output: 5295
🏁 Script executed:
#!/bin/bash
set -euo pipefail
sed -n '130,160p' docs/design/agent-workflows/projects/warm-daytona-sessions/research.mdRepository: Agenta-AI/agenta
Length of output: 2006
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Look for any explicit "Phase 3" measurement requirement or latency estimate caveats nearby.
rg -n "Phase 3|measure|measured|estimate|about 1s|resume|restart path|latency" docs/design/agent-workflows/projects/warm-daytona-sessions/research.mdRepository: Agenta-AI/agenta
Length of output: 1374
Qualify the “about 1s” resume estimate
The Stopped (warm) row presents a concrete resume latency, but the surrounding text doesn’t show an end-to-end measurement for stop/start + remount + session/load. Mark it as an estimate or remove the number until that path is measured.
| ## Open questions | ||
|
|
||
| See `open-questions.md`. The three that gate shipping: the orphan compute budget and autoStop | ||
| value (billing owner), the pointer-fencing design (reviewer), and the Tier 2 TTL and running | ||
| cap (billing owner). |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Separate Tier 1 blockers from Tier 2 prerequisites.
The status currently lists the Tier 2 TTL and running cap as a shipping gate, while plan.md explicitly defers Tier 2 and recommends enabling Tier 1 first. Split this into “Tier 1 enablement blockers” and “Tier 2 prerequisites” so approval of the first rollout is unambiguous.
…cabulary, navigation)
mmabrouk
left a comment
mmabrouk
left a comment
There was a problem hiding this comment.
Navigation aids for the rewrite. These inline notes mark where to start, what each key section answers, and the one or two decisions that need a human. The design and every technical claim are unchanged from the previous version; only the structure, vocabulary, and ordering moved.
| - **Park a sandbox:** stop it but keep its disk, so the next turn can restart the same one instead | ||
| of rebuilding it. This is the whole idea behind the project. | ||
|
|
||
| ## Read the files in this order |
There was a problem hiding this comment.
Start reading here. This lists the five files in order and says what each one answers, so you can stop at the first that gives you what you need.
| @@ -0,0 +1,111 @@ | |||
| # Context: warm and resumable Daytona sessions | |||
|
|
|||
| ## What a user sees today | |||
There was a problem hiding this comment.
The story starts here: what a user actually experiences today (every Daytona turn waits about twenty seconds), before any mechanism or jargon.
| The net effect is a full rebuild on every turn, exactly what F-020 reported. `research.md` | ||
| walks the code that proves each step. | ||
|
|
||
| ## The key finding |
There was a problem hiding this comment.
This section answers the 'so what is actually wrong' question in one paragraph: the warm-reuse code is already written, and only two missing Daytona functions plus a few gaps block it.
| reconnect that starts the old instance and then fails a later step (daemon, URL, or client | ||
| setup) leaves it running while the runner builds a second one. | ||
|
|
||
| ### The gap that defeats it all: the Daytona provider has no pause or reconnect |
There was a problem hiding this comment.
This is the code-level evidence for that key finding. It names the exact three consequences of the two missing provider functions.
| needs a compatibility check and a guard against racing writes. Those are the must-fix items under | ||
| park-to-stopped below. | ||
|
|
||
| ## The two levels of reuse |
There was a problem hiding this comment.
The proposal in one screen: park-to-stopped (cheaper, ships first) and park-to-running (pricier, deferred). Read just this if you only want the shape.
| `session_states.sandbox_id`, and the reconnect ladder at the start of a run. | ||
| - `patches/sandbox-agent@0.4.2.patch`: the native session reload and the local process-group kill. | ||
|
|
||
| ### Must-fix before it can be turned on |
There was a problem hiding this comment.
This section answers why the recommendation is 'flag off, then turn on after a live test' and not 'on by default'. These are the correctness gaps the design review found.
| - Fidelity: highest. It is the only level that can hold an open approval gate for byte-exact | ||
| resume, though holding a gate open is a separate concern (F-018) and is not needed for chat. | ||
|
|
||
| ## Recommendation |
There was a problem hiding this comment.
This is the decision to weigh in on: ship park-to-stopped behind a default-off flag now, and defer park-to-running until a billing owner sets its cost limits.
| owner owns the cost ones. Each names the phase it gates. The three that gate shipping at all are | ||
| the abandonment budget, the pointer write guard, and the park-to-running cost limits. | ||
|
|
||
| 1. **Abandonment compute budget** (Phase 2, billing owner). With `ephemeral: false` and a real |
There was a problem hiding this comment.
This is the decision that most needs you or a billing owner. With the sandbox no longer auto-deleted on stop, a crashed runner can bill compute for up to five minutes. Acceptable, or should the stop timer drop?
mmabrouk
changed the title
1 participant
What a user sees today
When you chat with an agent that runs on Daytona, every message waits about twenty seconds before the agent starts to answer. That wait is a fresh cloud sandbox being built from scratch, once per turn, even though the previous turn already built one. The answer itself is still correct, because a separate feature replays the conversation into the fresh sandbox, but every turn pays the full build time. This is QA finding F-020.
Why it happens
Recent, still-untested work already added the machinery to keep a sandbox warm: park it (stop it but keep its disk) at the end of a clean turn, then restart the same one and reload the conversation on the next turn. But the piece of code that talks to Daytona is missing the two functions that machinery calls, one to pause a sandbox and one to reconnect to a stopped one. So the park call falls through to a plain delete, and the next turn cannot find the sandbox and rebuilds it. The runner asks for the right behavior; the piece that carries it out is missing.
What this PR contains
A design-only planning workspace at
docs/design/agent-workflows/projects/warm-daytona-sessions/(README, context, research, plan, open-questions, status). No code changes. No live Daytona runs.This revision is a readability rewrite of that workspace. The design and every technical claim are unchanged. What moved is the structure, the vocabulary (labels like "Tier 1 / Tier 2" and "P0" are now self-describing names such as "park-to-stopped" and "must-fix"), the ordering (the story first, context before conclusions), and the navigation (the README states a reading order, and inline review comments on this PR mark where to start and which decisions need a human).
What we propose
Two levels of reuse, in order of cost:
What the design review changed
An adversarial design review found real correctness gaps that block turning the feature on: a failed pause loses its delete fallback and leaks a running sandbox; a failed reconnect can leak a second sandbox; two turns racing on one conversation can cross-write the stored sandbox id; and reuse applies none of the new request's settings, so it can resurrect stale credentials or a weaker network policy. The recommendation changed from "on by default" to "behind a flag that is off by default, turned on only after one live test".
status.mdrecords the full list.Decisions that still need an owner
How to read it
Start with
README.md(the reading order and what each file answers), thencontext.mdfor the story. The inline comments on this PR mark the start point and the decisions that need you.https://claude.ai/code/session_018MaXPNpvzN22kngHno3VMj