Bidirectional Context Branching with Selective Inheritance

[f0the999th]

Labels: enhancement, architecture, context-management, compute-efficiency, research-tooling
Status: Open for discussion
Summary
Proposal to replace linear, monolithic conversation histories with a Directed Acyclic Graph (DAG) primitive supporting bidirectional selective context inheritance. This architecture allows users to branch upward (selectively inheriting subsets of prior context), branch downward (partitioning future context into scoped sub-threads), and merge/partition vector namespaces without destructive edits or token wastage.

1. The Problem
   Current conversation models rely on a linear, monolithic paradigm where every turn re-transmits the full history. When exploring alternative directions, users face three suboptimal paths:

• Message Editing: Destructive; overwrites the original path.
• New Chat: Lossy; discards all prior context.
• Manual Copy-Pasting: Error-prone; burns tokens and breaks continuity.
  Token Processing Inefficiency
  For a conversation of length N, a linear model processes:

For a 50-turn conversation, this yields ~1,275 message-equivalents processed when only ~50 were authored. This architecture forces the platform to absorb unnecessary inference costs while delivering a restricted user experience.
2. Architectural Solution: Bidirectional Selective Branching
Replace the linear message list with a Directed Acyclic Graph (DAG) of message nodes. Each new turn evaluates as a parameterized query over the graph rather than a monolithic replay.
Core Affordances

Operation	Mechanism
Branch Up	Selectively include an arbitrary subset of ancestor messages as context from any new turn point.
Branch Down	Pin a scoped subset of context for all future descendant turns.
Merge	Combine selected nodes from disparate branches into a unified working context.
Partition	Tag a branch as an independent vector namespace for scoped retrieval.
Paradigm Shift
Dimension	Traditional Chat (Linear)
---	---
History Topology	Linear sequence ([msg_1 \dots msg_N])
Forking Mechanism	Destructive edit/overwrite
Context Resolution	Full replay per turn
Vector Namespace	Monolithic workspace store

3. Compute Optimization & Prefix-Cache Economics
   Mathematical Impact

• Shared Ancestors: Transmitted once and reused via persistent prefix caching.
• Divergence Delta: Branches transmit only their delta plus explicitly selected ancestors.
• Cache Utilization: Prefix-cache hit rates increase from an estimated ~60% to ~90%+, scaling efficiency linearly with branch multiplicity.
  Comparative Metrics (Estimated per 50-Turn Thread with 3 Branches)
  | Metric | Linear Model (Current) | Bidirectional DAG (Proposed) |
  |---|---|---|
  | Tokens Processed / Thread | ~375,000 | ~140,000 |
  | Prefix Cache Utilization | ~60% | ~90% |
  | Effective Compute / Branch | Full replay | Delta + selected ancestors |
  | Estimated Savings | — | ~55% – 65% reduction |

4. Two-Tier UX Design
   4.1 Default Mode: "Auto Context" (Average User)

• Zero Cognitive Overhead: System automatically resolves relevant ancestors using recency and semantic relevance.
• Interface: Simple boolean toggle: "Include earlier context? ✅" and a single action: "Explore from here".
• Abstraction: Hides the graph layer entirely while preserving improved output quality.
  4.2 Power User Mode: "Context Composer" (Researchers, Developers, Analysts)
• Graph Visualization: Complete workspace DAG render.
• Granular Control: Per-node include/exclude toggles, branch color-coding, and custom labels.
• Vector Scoping: Direct control over namespace boundaries for retrieval.
• Recombination: Drag-and-drop nodes across branches to synthesize new working contexts.
• Exporting: Isolate and export branches as standalone collections, JSON payloads, or shareable artifacts.

5. Technical Primitives Required

• Node-Edge Graph Engine: Messages stored as immutable nodes with labeled directed edges supporting multi-parent relationships (merge points).
• Context Resolution Pipeline: context_ref payload evaluated per turn via stable topological sort to maximize prefix-cache hits.
• Vector Namespace Isolation: Query filters bound to branch_id or explicit node sets.
• API Endpoints: Explicit state transitions for branching and turn-submission.

6. Proposed API Sketch
   POST /conversations/{id}/branch
   {
   "parent_node_id": "msg_20",
   "direction": "up",
   "context_ref": ["msg_03", "msg_07", "msg_20"],
   "label": "alt-hypothesis-A"
   }

POST /conversations/{id}/turn
{
"branch_id": "br_abc123",
"content": "...",
"context_ref": ["msg_03", "msg_07", "msg_20"],
"vector_scope": "branch"
}

7. Open Engineering Questions

• Model Evaluation: Do models trained on linear sequential chat degrade when exposed to sparse or topologically reordered selective context? What validation suites are required?
• Graph-Aware Moderation: How to prevent cross-branch context laundering of restricted or unsafe content?
• Heuristics: What weighting function (recency vs. semantic distance vs. manual pin) should drive default-mode auto-selection?
• Migration Strategy: Mapping legacy linear threads to single-path DAG structures.