Antigravity: a long-lived stream-json stdin session cuts first-response latency ~35x — worth a PR?

[vecsop]

---

Note on venue: CONTRIBUTING.md and the issue templates both point exploratory "is this a good idea" conversations at Discussions 鈫?Ideas rather than a formal Issue, so that's probably the better home for this 鈥?filing as an Issue only if maintainers would rather have it there. Posting the same content either way.

Context

antigravity.ts's current comment on why it doesn't use agy's -c resume flag:

Tested both shapes; -c activates agy's internal agentic loop (multi-step model retries, tool calls, fallback-to-cached-response on tool errors) which can't be steered from OD's system-prompt OVERRIDE 鈥?even with the strongest wording we got an identical byte-for-byte form re-emission on turn 2 when turn 1's tool-call retry path returned the cached form response.

We hit the same wall independently and ended up prototyping a third shape that sidesteps it: instead of either (a) a fresh agy -p <prompt> cold-spawn per turn (today's behavior) or (b) -c resume (rejected for the reason above), keep one agy child alive per conversation across turns using its stream-json stdin mode, and route subsequent turns' prompts into that same process instead of re-spawning. This never touches -c or agy's own agentic-loop/resume machinery at all 鈥?it's a different session shape, not a workaround for the one already rejected.

What we measured (downstream prototype, not upstream code)

• First-turn response latency: 13959ms 鈫?394ms (cold-spawn overhead paid once per conversation instead of once per turn).
• Full turn latency: 18.2s 鈫?2.4s.
• Ran 42 turns on the same conversation against the same child PID with no observed degradation.
• The form-double-submission failure mode -c hit (byte-for-byte re-emission from the cached tool-call retry path) does not reproduce on this shape 鈥?confirmed with a dedicated test asserting zero duplicate form submissions across the turn boundary.

Rough shape

• Pool one agy child per (conversation, model) pair behind a stream-json stdin connection, keyed so a same-model follow-up turn reuses the live child instead of spawning a new one.
• Idle children are torn down after a timeout; a model change or a sufficiently long idle gap falls back to a fresh cold spawn (today's behavior), so this only changes the common case (same conversation, same model, back-to-back turns).
• Three correctness gaps we found and had to close while building this, flagging in case they're relevant to how a real implementation should look regardless of whether this specific approach lands:
  1. Failure teardown (the daemon's existing retry/finishWithRetryDecision path) has some branches that assumed a spawn-per-turn lifecycle and needed a single explicit choke point once a child can outlive a turn.
  2. The model-selection lock (acquireAntigravityModelLock/waitForAgyToReadModel) has a release path that depends on watching a specific spawn's --log-file output; a reused child that never re-emits that line needs an immediate release on the pool-hit path instead of falling through to that watcher.
  3. waitForAgyToReadModel's existing 15s default timeout is tighter than agy's typical cold-boot window (~12-25s in our testing) for the very first spawn in a pooled conversation.

Why we're asking first instead of just opening a PR

This is a genuinely different session-management shape from what's in the file today, not a small tweak 鈥?happy to build it properly as a PR if there's interest, but wanted to check the direction (and whether there's context we're missing about why this wasn't the chosen shape) before investing in an upstream-quality implementation. If there's a reason stream-json-stdin-pooling doesn't fit OD's model here, that'd be useful to know before we go further, too.