feat(dynamo): opt-in lineage-scoped session affinity

[YAMY1234]

Summary

Adds an opt-in --dynamo-session-affinity-scope {conversation, lineage} endpoint option (default conversation — no behavior change).

With lineage, fork/spawn child conversations bind Dynamo session affinity (nvext.session_control) with the agent-tree root correlation id instead of their own conversation id, so the whole lineage co-locates on the worker (and data-parallel rank) that already holds the shared parent prefix.

Motivation

With --use-dynamo-conv-aware-routing, each child conversation currently binds its own session_id. As the note in dynamo_session_control.py explains, cross-branch KV reuse is expected to survive because a child's first (still-unbound) request is placed by Dynamo's prefix-overlap router onto the worker already holding the shared prefix.

That expectation holds only when the deployment has KV-event prefix indexing configured. On Dynamo deployments without a KV-event index, the router has no prefix knowledge for unbound requests: each child conversation's first turn is placed arbitrarily and re-prefills the shared parent prefix — once per spawned child. Since agentic traces are dominated by subagent requests sharing large parent prefixes, this loses a significant amount of prefix-cache reuse.

lineage scope gives such deployments an explicit, client-side way to keep an agent tree together, without requiring any server-side configuration.

Close semantics

A shared affinity key must not be torn down while sibling conversations may still be running — exactly the hazard the existing docstring warns about. Under lineage:

• child conversations always bind (never close); every bind refreshes the TTL;
• only the root conversation's final turn sends close;
• if any child outlives the root's final turn, the TTL reaper cleans up the leftover affinity.

Notes

• Default conversation keeps today's behavior unchanged.
• Trade-off: lineage-scoped affinity trades some placement freedom (whole trees pin to one worker) for prefix locality — hence deliberately opt-in.
• Draft: happy to add tests/docs if the direction looks right.

[github-actions]

Try out this PR

Quick install:

pip install --upgrade --force-reinstall git+https://github.com/ai-dynamo/aiperf.git@ea59b0137aba0a8e894d3bb268954bf236914930

Recommended with virtual environment (using uv):

uv venv --python 3.12 && source .venv/bin/activate
uv pip install --upgrade --force-reinstall git+https://github.com/ai-dynamo/aiperf.git@ea59b0137aba0a8e894d3bb268954bf236914930

---

Last updated for commit: ea59b01 • Browse code

[YAMY1234]

Positioning note: this flag is intended as an interim, client-side measure. The longer-term home for subagent co-location is a router-side placement policy consuming the dedicated parent/agentic identity hints — filed as ai-dynamo/dynamo#11343. Once such a policy exists, this flag simply stays off (default behavior is unchanged either way).

[ajcasagrande]

Heads-up: #15 (session-routing plugin layer, targeting cquil11/aiperf-agentx-v1.0) now carries this PR's lineage-scoped affinity semantics as --session-routing dynamo_nvext --session-routing-opt scope=lineage, with co-author credit — the plugin layer replaces the --use-dynamo-conv-aware-routing machinery this PR patches, so on the v1.0 line the option surface moved into the plugin's Options model (the entire config plumbing here collapses into one field). One semantic refinement over the root-final-turn close: the shared key closes only on a request stamped provably-last for the whole tree (is_tree_final, agentic replay), so a root's final turn can never tear down affinity while sibling subagents are still running; indeterminate modes fall back to the TTL reaper exactly as described here. Wire-level comparison runs of both implementations are summarized in #15. This PR still applies as-is to its cjq/agentx-v0.4 base.

🤖 Generated with Claude Code

[ajcasagrande]

Update: your lineage-scoped affinity now lives as a dedicated, co-authored commit on the #15 branch — 074a67d87 feat(routing): lineage-scoped affinity for dynamo_nvext (Co-Authored-By: YAMY1234).

What carried over vs. what changed, relative to this PR:

• Key selection: identical. Under --session-routing dynamo_nvext --session-routing-opt scope=lineage, every session in an agent tree binds nvext.session_control with the tree root's correlation ID; conversation remains the default with today's per-session behavior.
• Config surface: collapsed. --dynamo-session-affinity-scope + EndpointDefaults + EndpointInfo plumbing become one field on the plugin's Options model (scope: Literal["conversation","lineage"]), validated/canonicalized by the existing --session-routing-opt machinery.
• Close discipline: refined. Instead of closing on the root conversation's final turn (which can tear down the shared key while sibling subagents still run), close fires only on a request the credit issuer stamped is_tree_final — provably the last request of the whole tree, available under agentic replay. Indeterminate modes (e.g. reactive dag_jsonl) never close and fall back to the session_control TTL, the same backstop this PR uses for stragglers.

Verified with back-to-back --export-level raw mock-server runs of --scenario inferencex-agentx-mvp against this PR's branch: child binds are wire-identical (tree-root-keyed in both), and the refined close fired exactly once, on the provably-last wire request of a fully-drained tree. Also exercised on a synthetic dag_jsonl fork+spawn tree: all children bind the root key; no early close.

Thanks for the design — the motivation writeup (KV-event-index-less placement, the sibling-teardown hazard) transferred directly. This PR still applies as-is to its cjq/agentx-v0.4 base if that line needs it before v1.0 lands.

🤖 Generated with Claude Code