Meridian reliability: make concurrent RLM subagents production-safe over Claude Max

[rynfar]

Umbrella for making Meridian (the Claude Agent SDK proxy at rynfar/meridian) a production-safe provider for Prime Agent, including concurrent RLM subagents. Meridian's README currently rates Prime Agent "single-agent verified" with these failure modes: overload amplification, expensive cache churn after fresh-session replay, loss of child-task context during recovery, undelivered tool envelopes, incomplete parent-to-child cancellation.

Measured evidence (Meridian telemetry, real prime traffic 2026-08-17/18)

• prime adapter averaged 65.8% cache hit vs 95.1% for OpenCode; 212 successful requests ran below 20% cache hit and wrote ~12M cache-creation tokens.
• Repeated cache_miss: Cache hit rate 0% on resume (expected >50%) diagnostics on continuations.
• Resume-failure cascade observed: "You've hit your session limit" on resume, then the same request retried resume=false (full fresh replay) one second later; a dozen requests failed in ~15s.
• 3× ENVELOPE VIOLATION [undelivered_tool_use] on the ipython tool, two coinciding with upstream 529s.

Root causes on the Prime Agent side (verified in this repo)

1. Provider identity is unstable and collapses for inline subagents. The Meridian extension stamps metadata.user_id from ctx.sessionManager.getSessionId() in before_provider_request. Inline RLM children reuse the parent's extension runner (packages/coding-agent/src/core/agent-session.ts inline child construction copies onPayload: this.agent.onPayload; the root streamFn/onPayload in packages/coding-agent/src/core/sdk.ts closes over the root extensionRunnerRef), so every inline child request is stamped with the parent's session key. Interleaved parent/child histories on one proxy session key force a fresh-session replay nearly every round. SessionManager.sessionId also mutates on fork/branch/load, and compaction/branch-summarization/refinement completeSimple calls bypass onPayload entirely (unkeyed). → feat(agent): stable child-scoped provider identity for RLM subagents, side questions, and auxiliary requests #23
2. Retries multiply against an overloaded upstream. Session-level auto-retry (_isRetryableError treats every stopReason === "error" as retryable; fixed 2s/4s/8s backoff, no jitter, no Retry-After) stacks on the Anthropic SDK's own maxRetries, ≈ 4 × (1 + sdkRetries) requests per failed turn per agent, times N concurrent children. maxRetryDelayMs is declared and threaded but never read by any provider (dead code). → fix(agent): kind-aware retry with Retry-After and jitter; implement dead maxRetryDelayMs; stop SDK-times-session retry multiplication #24
3. User-facing cancel does not reach children. requestAbort() does not call _cancelActiveRlmChildRuns() (only abort() does), and every user-facing cancel path uses requestAbort(). Orphaned children keep consuming the Max subscription. → fix(agent): requestAbort() must cascade to active RLM child runs #25
4. The cached prompt prefix is volatile. Mutable continual-harness state and Current date: are serialized into the system prompt; cache_control rides the last tool definition so tool-set changes/reordering invalidate the tool cache. → perf(agent): keep the prompt-cache prefix stable across harness updates, date flips, and tool refreshes #26

Meridian-side work (tracked in rynfar/meridian, cross-linked)

• Emit Retry-After on 429/503 and scope [1m] rate-limit benching per session instead of profile-wide (today one child's rate limit downgrades and cold-caches every sibling).
• Parent→child session registry + cancellation, keyed off a parent linkage stamped in metadata.user_id.
• Fresh replay drops assistant tool_use blocks (Fresh replay drops every assistant tool_use block, so a thinking-plus-tool-call turn becomes an empty turn rynfar/meridian#888) — the mechanism behind lost child-task context after recovery.

Sequencing

• feat(agent): stable child-scoped provider identity for RLM subagents, side questions, and auxiliary requests #23 stable, child-scoped provider identity (P0 — biggest single lever on cache churn)
• fix(agent): kind-aware retry with Retry-After and jitter; implement dead maxRetryDelayMs; stop SDK-times-session retry multiplication #24 kind-aware retry/backoff, honor Retry-After, implement maxRetryDelayMs, stop retry multiplication (P0)
• fix(agent): requestAbort() must cascade to active RLM child runs #25 cascade cancellation from requestAbort() to RLM children (P1)
• perf(agent): keep the prompt-cache prefix stable across harness updates, date flips, and tool refreshes #26 cache-prefix stability for system prompt and tool ordering (P1)
• Meridian-side: Retry-After + per-session 1M benching; parent-tree cancellation; replay fidelity (Fresh replay drops every assistant tool_use block, so a thinking-plus-tool-call turn becomes an empty turn rynfar/meridian#888)
• Extension contract v2: stamp a stable per-agent identity plus parent linkage (depends on feat(agent): stable child-scoped provider identity for RLM subagents, side questions, and auxiliary requests #23)

Per PYLON.md, each item needs an upstream-overlap check and a .pylon/features.yaml decision before landing.

[rynfar]

Status: all three P0 implementation slices are up and reviewed.

• feat(agent): stable child-scoped provider identity for RLM subagents, side questions, and auxiliary requests #23 → PR fix(coding-agent): scope provider identity to the agent that owns each request #31 (child-scoped provider identity): CI green, reviewed, CLEAN. The live meridian extension needs no changes.
• fix(agent): kind-aware retry with Retry-After and jitter; implement dead maxRetryDelayMs; stop SDK-times-session retry multiplication #24 → PR fix(agent): kind-aware retry with Retry-After and a real delay cap #30 (kind-aware retry): reviewed, rebased onto current pylon, plus a reviewer follow-up commit — Meridian delivers mid-stream waits as error.retry_after (seconds) in the SSE frame since headers are already on the wire, and the anthropic provider now prefers that in-frame value; without it the Retry-After honoring was nearly inert for streaming clients.
• Rate-limit responses carry no Retry-After, and [1m] benching is profile-wide — one RLM child's 429 cold-caches every concurrent sibling rynfar/meridian#901 → feat: emit Retry-After and scope [1m] rate-limit benching per session rynfar/meridian#907 (emit Retry-After; per-session [1m] benching): CI green, reviewed. Bench scoping split by trigger: Extra Usage exhaustion stays profile-wide (subscription fact), plain rate limits are session-scoped.

Merge-order note for #30/#31: both touch agent-session.ts, side-question.ts, and the .pylon ledgers; whichever merges second needs a rebase.

Remaining, deliberately queued to avoid stacking more unmerged branches on agent-session.ts: #25 (requestAbort cascade), #26 (cache-prefix stability), and rynfar/meridian#902 (parent-tree cancellation, blocked on the parent_session_id wire contract deferred from #23).

[rynfar]

All sequenced work has landed. Merged:

• feat(agent): stable child-scoped provider identity for RLM subagents, side questions, and auxiliary requests #23 → PR fix(coding-agent): scope provider identity to the agent that owns each request #31 (child-scoped provider identity)
• fix(agent): kind-aware retry with Retry-After and jitter; implement dead maxRetryDelayMs; stop SDK-times-session retry multiplication #24 → PR fix(agent): kind-aware retry with Retry-After and a real delay cap #30 (kind-aware retry, Retry-After incl. in-frame SSE variant, dead maxRetryDelayMs implemented, retry demultiplication)
• fix(agent): requestAbort() must cascade to active RLM child runs #25 → PR fix(coding-agent): cancelling a turn now stops its subagents #35 (requestAbort cascades to RLM children; deliberate reversal of upstream PR make ctrl+c cancel active turns deterministically PrimeIntellect-ai/prime-agent#346 semantics, recorded in the ledger)
• perf(agent): keep the prompt-cache prefix stable across harness updates, date flips, and tool refreshes #26 → PR perf(coding-agent): keep the prompt-cache prefix stable across harness updates, date flips, and tool refreshes #36 (stable cache prefix; volatile content placement is a per-model contract via appendOnlyHistory for session-cached backends)
• feat(agent): expose parent session identity to child extension contexts for RLM trees #34 → PR feat(coding-agent): expose the spawning session's identity to child extension contexts #38 (parent session identity for extensions, immediate-parent semantics)
• Rate-limit responses carry no Retry-After, and [1m] benching is profile-wide — one RLM child's 429 cold-caches every concurrent sibling rynfar/meridian#901 → feat: emit Retry-After and scope [1m] rate-limit benching per session rynfar/meridian#907 (Retry-After emission; session-scoped [1m] benching)
• Parent-to-child cancellation: session-tree registry and abort propagation for RLM subagent trees rynfar/meridian#902 → feat: cancel RLM subagent trees when the parent request aborts rynfar/meridian#912 (live session-tree registry; parent abort propagates to descendant requests; POST /v1/sessions/:key/cancel)
• Ledger state flip in PR docs: mark turn-scoped subagent cancellation as shipped #40.

Extension contract v2 is deployed to the maintainer's ~/.prime/agent/extensions/meridian.ts: appendOnlyHistory: true on the model entry and guarded parent_session_id stamping.

Remaining deployment note: merged is not yet running — Meridian needs a release published and the global install updated, and the Prime Agent fork has no release pipeline by design (see PYLON.md release boundary), so running builds must come from the fork source until that is designed. Deferred items are recorded on the individual PRs. Closing.