Skip to content

Raft query peer event bus unification - #1427

Merged
zonotope merged 9 commits into
mainfrom
fix/raft-query-peers
Jul 9, 2026
Merged

Raft query peer event bus unification#1427
zonotope merged 9 commits into
mainfrom
fix/raft-query-peers

Conversation

@zonotope

@zonotope zonotope commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

In raft mode, the StateMachineAdapter emits NameServiceEvents on RaftIntegration::event_bus, but the /v1/fluree/events SSE endpoint (and every other consumer that calls Fluree::event_bus()) subscribes on a different bus — the one Fluree allocates internally during build. Nothing bridged the two, so runtime raft commits never reached SSE subscribers. The bug hid because the endpoint's connection-time snapshot reads straight from the nameservice: data present at connect appeared, subsequent inserts did not. Query peers subscribed to a raft cluster therefore never heard about new commits.

Separately, Fluree's local cache event listener (which reconciles the LedgerManager on commit/index events and evicts on retract) was only spawned when background indexing was enabled. Query peers run without indexing, so they had no listener at all and depended on PeerSyncTask manually re-notifying the LedgerManager after every nameservice update.

Fix

FlureeBuilder gains with_event_bus(Arc<LedgerEventBus>): a caller-supplied bus is used instead of allocating a fresh one, uniformly across every build_* path (via a single resolve_event_bus helper). The raft server assembly threads RaftIntegration::event_bus through build_fluree_with_nameservice / build_default_fluree into the builder, so the state-machine adapter (publisher) and the SSE events endpoint, registry maintenance task, and cache listener (subscribers) all share one broadcast channel.

The local cache event listener is now spawned whenever a LedgerManager exists, independent of indexing mode. This gives query peers cache reconciliation driven by the events their NotifyingNameService already emits on successful CAS operations.

Consequent removals

  • The server assembly's explicit spawn_local_cache_event_listener call on the raft bus is gone — the builder-spawned listener now covers it (this also removes a duplicate reconcile task that previously ran on raft nodes with indexing enabled).
  • PeerSyncTask::refresh_cached_ledger and its manual LedgerManager notification are gone: fast_forward_commit dispatches through NotifyingNameService::compare_and_set_ref, which emits LedgerCommitPublished / LedgerIndexPublished on Updated, and the always-running listener reconciles from the just-updated local nameservice.
  • The explicit cache eviction in the peer retract path is gone: NotifyingNameService::retract emits LedgerRetracted, and the listener's ledger_manager.disconnect is exactly what Fluree::disconnect_ledger did.

Behavior change for embedders

The synchronous builders (build_memory(), build_with(), etc.) previously spawned no tasks when indexing was disabled and worked outside an async context. They now spawn the cache listener whenever ledger caching is enabled (the default), so all build_* methods must be called within a tokio runtime. The FlureeBuilder docs state this, and the affected unit tests were converted to #[tokio::test].

@zonotope
zonotope requested review from aaj3f and bplatz July 3, 2026 17:00
Comment thread fluree-db-api/src/lib.rs
/// - **Dynamic build** (`build_client()`) returns `FlureeClient` (type-erased) —
/// used when the storage backend is determined at runtime from config.
///
/// All `build*` methods (including the synchronous ones) must be called

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heads up: this makes all build_* methods (including sync ones like build_memory()) spawn a tokio task whenever ledger caching is on (the default), so they now panic outside a runtime. docs/contributing/benches.md:368 still claims build_memory() has no reactor requirement, and fluree-db-api/benches/insert_formats.rs:279 calls it bare per that guidance (no rt.block_on) — that bench will panic as-is. Worth fixing the call site and updating the doc in this PR.

@bplatz bplatz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple comments inline.

@zonotope
zonotope merged commit 932cfdf into main Jul 9, 2026
13 checks passed
@zonotope
zonotope deleted the fix/raft-query-peers branch July 9, 2026 15:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants