Skip to content

feat(streaming): complete graph-first firehose controls - #443

Merged
aaltshuler merged 2 commits into
mainfrom
codex/graph-stream-control
Aug 5, 2026
Merged

feat(streaming): complete graph-first firehose controls#443
aaltshuler merged 2 commits into
mainfrom
codex/graph-stream-control

Conversation

@aaltshuler

@aaltshuler aaltshuler commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • expose selector-free graph-wide stream resume, EnsureIndices, and Optimize through the existing recovery-v15/v16/v17 owners
  • add graph-logical CLI, HTTP, OpenAPI, status guidance, and offline dead-letter/block projections without leaking datasets or lanes
  • batch dead-letter receipt lookup and bind pagination to a tamper-resistant logical cursor
  • add a real-binary golden journey covering cluster enablement, mixed node/edge ingest, automatic fold/query visibility, disable/re-enable, productive sealed maintenance, convergent resume, and successor ingest

Design boundaries

  • reuses the existing graph coordinator and Lance machinery; no new WAL, coordinator, recovery grammar, or storage-format strand
  • exposes no type/table/lane selector and no public physical rebind
  • retains the selected experimental unbounded retain-all profile; bounded reclamation remains separate work

Validation

  • cargo check --workspace --locked
  • cargo test -p omnigraph-cli --test system_local -- --nocapture (28 passed, 1 network-dependent test ignored)
  • focused engine graph-resume, sealed-maintenance, dead-letter pagination/tamper, and forbidden-API tests
  • focused server route, auth/default-deny, and OpenAPI drift tests
  • scripts/check-agents-md.sh
  • git diff --check

Greptile Summary

This PR exposes selector-free, graph-wide stream resume and sealed maintenance through the engine, HTTP API, CLI, and cluster control surfaces while retaining the existing recovery protocols.

  • Adds graph-wide resume, EnsureIndices, and Optimize orchestration.
  • Adds corresponding HTTP routes, shared DTOs, CLI commands, output rendering, and OpenAPI definitions.
  • Adds batched dead-letter receipt lookup and tamper-resistant logical pagination.
  • Extends integration, authorization, recovery, and real-binary journey coverage.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
crates/omnigraph/src/db/omnigraph/stream_management.rs Adds graph-wide preflight and orchestration for resume and sealed maintenance using the existing recovery owners.
crates/omnigraph/src/db/manifest/token_store.rs Adds batched token-ledger receipt lookup supporting bounded logical dead-letter projections.
crates/omnigraph-cluster/src/stream_block.rs Extends offline block and dead-letter operations with logical pagination and graph-level controls.
crates/omnigraph-server/src/handlers.rs Adds authorized graph-scoped handlers for stream resume and maintenance.
crates/omnigraph-server/src/lib.rs Registers the new protected stream-control routes and OpenAPI operations.
crates/omnigraph-cli/src/client.rs Adds embedded and remote client parity for graph-wide stream controls.
crates/omnigraph-api-types/src/lib.rs Defines shared graph-logical request and response DTOs for the new control operations.
crates/omnigraph-server/tests/openapi.rs The follow-up commit completes expected-path coverage for all three new stream-control routes.
crates/omnigraph/tests/memwal_stream.rs Expands engine coverage for resume convergence, sealed maintenance, and dead-letter cursor behavior.

Sequence Diagram

sequenceDiagram
    participant Operator
    participant CLI
    participant Server
    participant Engine
    participant Recovery
    participant Lance
    Operator->>CLI: stream resume / maintenance
    CLI->>Server: Graph-scoped HTTP request
    Server->>Server: Authenticate and authorize stream_manage
    Server->>Engine: Graph-wide control operation
    Engine->>Engine: Preflight enrolled declarations
    Engine->>Recovery: Compose existing v15/v16/v17 owner
    Recovery->>Lance: Apply productive physical effects
    Recovery->>Engine: Atomically publish manifest state
    Engine-->>Server: Graph-logical result
    Server-->>CLI: Shared response DTO
    CLI-->>Operator: Human or JSON output
Loading

Reviews (2): Last reviewed commit: "test(server): cover graph stream control..." | Re-trigger Greptile

Context used (4)

@aaltshuler
aaltshuler merged commit ea381c5 into main Aug 5, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant