docs(rfc): amend the served stream transport to graph scope (P8)

[aaltshuler]

What changed

• RFC-026: new §4.7 P8 — Transport scope decision (graph-scoped; per-type is an operator filter), following the P1–P7 grammar: one multiplexed NDJSON ingest per graph, transport-internal prepare with a single graph-level raw-HTTP endpoint, graph-scoped operator verbs reusing existing compositions (driver round; the disable owner's deterministic all-lane sweep with a lifecycle-only exit), the multiplexed response contract (caller order; lane-scoped vs root-scoped blocking; flush-interval-bounded head-of-line delay, measured not guessed), and an explicit statement that P2's engine invariants and all physical/persisted grammar are untouched. §4.6 gains a note that its per-type route sketch remains the full-product design of record while the experimental activation uses P8.
• firehose-path-specs §8 (F7): the wire table rewritten to the graph-scoped routes (/graphs/{id}/stream/ingest|prepare|fold|quiesce|resume, GET /graphs/{id}/stream), with the amendment note and the response-contract summary; the five-workflow paragraph updated; F4's remote-client bullet clarified (per-lane prepare beneath one graph call; engine seams stay per-lane); a new closed-decisions row records the selection.

Why

The prior F7 draft leaked the per-lane physical machine into the caller surface, contradicting P2's own one-connected-model rationale and the direct lane's mixed-type load precedent — and per-type streams are worse for correctness: they destroy producer cross-type arrival order and maximize the cross-arrival skew that P6 dead-letters, while one multiplexed stream preserves it for the driver's node-before-edge rounds.

Timing: deliberately before any F7 implementation, while no OpenAPI/CLI/DTO surface is frozen — the amendment shrinks the public grammar (prepare exits the workflow set; per-type routes collapse to per-graph) instead of costing a deprecation cycle later.

Scope

Docs-only; no engine, format, or recovery change. Cluster-only per-lane surfaces (block show/correct, dead-letter list/export, retirement) deliberately keep their existing grammar.

Validation

• scripts/check-agents-md.sh — OK (73 links, 69 docs)
• anchored patch application (every replaced block asserted verbatim before edit)

Greptile Summary

The documentation amends RFC-026’s planned served-stream transport from per-type routes to a graph-scoped contract.

• Defines one complete-catalog graph prepare occurrence and one multiplexed NDJSON ingest stream.
• Moves status and lifecycle controls to graph-level routes with opaque preparation and control tokens.
• Specifies durable partial-progress, replay, blocking, cancellation, ordering, and deadline behavior.
• Aligns the firehose path specification and future F6c/F7 implementation milestones with the graph boundary.

Confidence Score: 5/5

The PR appears safe to merge.

The previously reported prepare-outcome gap is resolved by explicit durable partial progress, immutable terminal receipts, deterministic child retry identities, token issuance rules, and same-ID versus new-ID replay behavior; no blocking failure remains.

Important Files Changed

Filename	Overview
docs/rfcs/0026-memwal-streaming-ingest.md	Defines the graph-scoped transport decision and closes the previously unspecified bulk-prepare outcome, partial-progress, and replay contracts.
docs/dev/firehose-path-specs.md	Updates the implementation roadmap, wire-route table, coordinator requirements, and operator surfaces to consistently use graph-scoped opaque-token contracts.

Sequence Diagram

sequenceDiagram
    participant Client
    participant API as Graph Stream API
    participant Coordinator as Graph Prepare Coordinator
    participant Lanes as Private Lane Adapters
    Client->>API: "POST /graphs/{id}/stream/prepare"
    API->>Coordinator: prepare_id + graph_control_token
    Coordinator->>Lanes: deterministic complete-catalog children
    alt all children prepared
        Lanes-->>Coordinator: completed
        Coordinator-->>Client: completed + ingest_prepare_token
        Client->>API: "POST /graphs/{id}/stream/ingest (NDJSON)"
    else work remains
        Coordinator-->>Client: in_progress + ordered logical progress
    else child blocked
        Coordinator-->>Client: blocked + opaque block token
    else disable cancels prepare
        Coordinator-->>Client: cancelled_by_disable, no prepare token
    end

Loading

_{Reviews (2): Last reviewed commit: "docs(rfc): make the firehose surface gra..." | Re-trigger Greptile}

[greptile-apps]

Define bulk prepare outcomes

The new graph-level endpoint composes independently effectful per-lane prepare operations, but it does not define atomicity, partial results, result ordering, or how request IDs map to per-lane retry identities. For a request covering multiple types where one lane succeeds and another is absent, ineligible, or stale, clients cannot determine whether successful enrollment effects remain, causing subsequent ingest to target lanes the caller assumed were prepared and receive avoidable per-line refusals.

[aaltshuler]

Closing this version as superseded. The graph-level thesis is sound, but this patch rewrites too much settled F6 history, conflicts with the completed F6b7/F6b8 work, and carries contradictory public contracts. I’m rebuilding the amendment from current main as a smaller graph-scoped documentation change: public row-or-graph blocking only, complete-catalog preparation, consistent authorization, explicit terminal cancellation, and opaque graph-level control/status tokens.

[aaltshuler]

Replacement is now open as #436. One correction to my close note above: the final design does not use complete-catalog preparation. It removes the public prepare phase entirely and lazily invokes the existing private recoverable enrollment child only after a graph-native row is boundedly parsed and validated. That eliminates the N-child partial-prefix transaction called out in review while keeping the graph as the only public resource.