feat(replication): WS plane — shard-0-pinned deterministic WS.CREATE/DROP + live streaming + PSYNC registry blob (Wave B)#293
Merged
Conversation
Wave B ws-plane groundwork. Adds `workspace::repl` — thin RESP-array wrapping of the existing WAL payload bytes (`encode_workspace_create`/`encode_workspace_drop`) into two internal pseudo-commands, `WS.CREATE.APPLY` / `WS.DROP.APPLY`, so the exact bytes already written to shard 0's WAL can flow over the replication wire and be recognized by the replica apply path before generic dispatch (mirrors `GRAPH.ADDNODE` and friends). No new encoding is invented: the payload is always the verbatim output of the WAL encoders. Not yet wired into any write path or replica apply arm — that lands in a follow-up commit alongside the shard-0 affinity hop. author: Tin Dang
Wave B ws-plane groundwork. Adds `replication::ws_sync` — export/install of the process-global `WorkspaceRegistry` as a new `MOON_AUX_WORKSPACE_REGISTRY` RDB aux blob (`persistence::redis_rdb`), for the FULLRESYNC full-resync leg. Unlike the per-shard `MOON_AUX_GRAPH_STORE`/vector/text blobs, the registry is a single process-global structure (one `Mutex` on `ShardDatabases`, visible from every shard's thread) rather than sharded state, so this blob is singular and shard-0-authoritative: exactly one blob belongs in the merged RDB regardless of `--shards`. Versioned little-endian format (version byte + entry count + per-entry ws_id/name/created_at), bounds- checked decoder returning `None` on any malformed input (truncated header, truncated entry, unknown version) rather than panicking — a corrupted or adversarial PSYNC stream must never crash a replica. `None` and `Some(empty)` registries encode identically so a replica can distinguish "master authoritatively has zero workspaces" from "pre-WS-sync master, aux absent entirely" once wired into `load_snapshot`. Includes 5 unit tests (round-trip, none/empty equivalence, 4 malformed-input shapes, truncated-full-blob, empty-name/negative-created_at edge case) and a new `ws_registry_record` cargo-fuzz target for the decoder (Testing rule: every new decoder needs a fuzz target). Not yet wired into any PSYNC capture site or `load_snapshot` — that lands in a follow-up commit. author: Tin Dang
Wave B ws-plane. Adds the cross-shard hop that will let any connection thread route the process-global WorkspaceRegistry mutation onto shard 0's own OS thread, mirroring the existing `WsDropCleanup` hop shape. New `ShardMessage::WsRegistryCreate(Box<WsRegistryCreatePayload>)` / `WsRegistryDrop` variants (boxed vs inline sized per the existing 64-byte enum budget) and their `handle_shard_message_shared` arms run the WHOLE mutation + WAL-append + replication-record sequence in one synchronous stretch on shard 0's thread — this is what keeps the replication offset advance atomic with the mutation w.r.t. a concurrent PSYNC snapshot capture, the same argument `record_local_write` relies on for per-shard state, applied here to the one piece of state that is global. A new `record_ws_registry_write` helper duplicates (rather than shares) `record_local_write`'s backlog-append + offset-advance + self-queue-push steps, because this call site has no `ConnectionContext` — only the raw per-shard pieces already threaded through `handle_shard_message_shared`'s signature (same precedent as `wal_append_and_fanout` being an independent, not shared, copy of the same steps). Also wires the WS registry export into `PrepareReplicaSync`'s reply (`PreparedShardSync::ws_registry_blob`, `Some` only on shard 0's leg, `None` elsewhere) so the master-side PSYNC capture sites can pick it up in a follow-up commit. Not yet reachable from any command handler — `WsRegistryCreate`/`Drop` are constructed nowhere yet; that lands with the write-path wiring. author: Tin Dang
Wave B ws-plane. Threads `Arc<ShardDatabases>` through the replica's apply
path so it can reach the process-global `WorkspaceRegistry`, which lives on
`ShardDatabases` rather than the thread-local `ShardSlice`:
- `apply::apply_local` recognizes the `WS.CREATE.APPLY` / `WS.DROP.APPLY`
internal pseudo-commands (`workspace::repl`) before generic dispatch.
`apply_ws_create` installs the MASTER's `ws_id` + `created_at` VERBATIM —
UUIDv7 is nondeterministic, so a replica must never mint its own id.
`apply_ws_drop` removes the entry and reruns the master's best-effort
`{ws_hex}:`-prefix key sweep across every db on this shard (R0 replication
is single-shard-only, so sweeping the one shard IS the whole sweep — no
cross-shard hop to mirror). Both are in-memory only, matching
`apply_graph`'s no-local-WAL-persistence model: a restarted replica
resyncs the registry from its master rather than replaying a local copy.
- `apply::load_snapshot` reads the (singular, shard-0-authoritative)
`MOON_AUX_WORKSPACE_REGISTRY` aux blob and installs it authoritatively
after a successful keyspace load — same convention as the graph store
(empty blob = master authoritatively has none, replaces local state;
absent aux = pre-WS-sync master, warn-and-keep-local).
- `ReplicaTaskConfig` gains a `shard_databases` field (both runtime
variants' 4 `apply_local`/`load_snapshot` call sites updated); its two
construction sites (`handler_monoio`/`handler_sharded` `dispatch.rs`)
now pass `ctx.shard_databases.clone()`.
Not yet reachable — the master-side PSYNC capture and the write-path shard-0
hop wiring land in the next two commits.
author: Tin Dang
Wave B ws-plane. Wires `replication::ws_sync::export_workspace_registry` into the two FULLRESYNC capture sites: - `handle_psync_inline_single_shard`: shard 0 IS this thread's shard, so the capture is trivially in the same synchronous stretch as the offset read; always written to `moon_aux` (empty blob tells the replica the master authoritatively has zero workspaces), same convention as the graph blob. - `handle_psync_inline_multi_shard`: only shard 0's `PreparedShardSync` leg populates `ws_registry_blob` (every other shard replies `None`, wired in the previous commit); the merge loop keeps the first `Some`, matching the `vector_defs`/`text_defs` convention. Combined with the previous commit's `load_snapshot` install, a replica attaching to a master that already has workspaces now backfills them from the snapshot leg. author: Tin Dang
Wave B ws-plane — the write-path wiring that makes everything from the previous five commits reachable. `try_handle_ws_command` (monoio) now branches on `ctx.shard_id == 0`: a connection already on shard 0 runs the mutation + WAL-append + replication- record sequence inline (as before, just now also emitting the live replication record); every other connection sends `WsRegistryCreate` / `WsRegistryDrop` to shard 0 via `spsc_send` and awaits the bounded oneshot reply — same dual-path shape `WsDropCleanup` already used for the key-sweep half of WS.DROP, now applied to the registry mutation itself. Command semantics (return value, error cases) are unchanged; only the mutation's execution thread and its now-replicated side effect are new. Live replication emission happens only on the inline (shard-0) branch, gated on `replication_fanout_active(ctx)` exactly like the graph plane: the WorkspaceCreate/WorkspaceDrop WAL payload is wrapped via `workspace::repl::serialize_ws_create_apply`/`serialize_ws_drop_apply` and pushed with `record_local_write_db`. The foreign-hop branch's equivalent push happens on shard 0's own thread inside `handle_shard_message_shared` (landed in commit 3) — so every WS.CREATE/DROP is replicated exactly once regardless of which connection thread received it. Also splits the former combined `warn_unreplicated_plane` fail-loud marker (`handler_monoio/ft.rs`) into `warn_mq_unreplicated`: the WS half is deleted now that this plane replicates; the MQ half survives (MQ.* is not yet wired into replication — separate branch/task). author: Tin Dang
Wave B ws-plane. New `tests/replication_ws.rs`, self-contained (own copies of `start_moon`/`Killer`/`send_cmd`/`send_resp`/`wait_until`, modeled on `tests/replication_graph.rs`): - `replica_syncs_ws_create_live_stream` — replica attaches to an EMPTY master, then WS.CREATE runs; asserts the replica resolves the MASTER's exact `ws_id` via WS.INFO/WS.AUTH (the verbatim-apply / no-re-mint correctness guard). - `replica_backfills_ws_registry_from_snapshot` — WS.CREATE runs BEFORE the replica ever attaches; asserts FULLRESYNC backfill produces an identical WS.INFO on both sides, and that a live write past the snapshot boundary still replicates. - `replica_syncs_ws_drop` — WS.DROP with a replica attached; asserts the workspace disappears from WS.LIST and WS.AUTH fails identically on both sides. - `ws_create_shard0_hop_consistent_across_shards` — `--shards 4`, no replica: WS.CREATE followed by 16 connections' WS.INFO/WS.LIST, exercising the shard-0 hop from whichever shard SO_REUSEPORT lands each connection on. Also adds the `[Unreleased]` CHANGELOG entry summarizing the six-commit Wave B ws-plane series. Verified: all 4 new tests green against a `release-fast` build; existing `replication_graph` (3), `replication_planes` (11), and `shardslice_live` (6, including both `test_ssm3_ws_*` legs) stay green — no regressions. Both default and `runtime-tokio,jemalloc` clippy matrices clean at `-D warnings`; `cargo fmt --check` clean; full `cargo test --profile release-fast --lib` green (4156 passed, 1 ignored, 0 failed). author: Tin Dang
…overy
Adversarial review of the ws-plane branch (verdict SHIP-WITH-FIXES):
P0: the WS.CREATE shard-0 hop discarded both the spsc_send and
recv_reply_bounded outcomes and unconditionally replied with the ws_id —
a wedged ring or 30s reply timeout returned a phantom id the registry
never saw (WS AUTH/INFO against it 404s), directly contradicting the
sibling WS.DROP path that already gated on the reply. Fixed: the hop
outcome is checked, and on failure the process-global registry is probed
to disambiguate "shard 0 executed but the reply was slow/lost" (entry
present, success) from "the mutation never happened" (entry absent,
fail closed with a retryable error).
P1: a WS.DROP hop timeout resolved `removed = false` even when shard 0
completed the drop (timeout means "no reply in 30s", not "failed" — the
handler's synchronous O(keys × databases) WsDropCleanup scan makes slow
replies plausible). The registry entry being gone meant no retry could
ever re-trigger the {ws_hex}: key-prefix sweep — the workspace's keys
leaked permanently. Fixed with the same registry probe: entry gone →
the drop landed, run the (idempotent) sweep; entry present → genuine
hop failure, fall through to the not-found path as before.
Both fixes are conn-thread-only; the shard-0 handler and replication
paths are untouched. Gates re-run green: lib 4156/4156, replication_ws
4/4 (incl. the 16-iteration cross-shard hop consistency test), clippy
both matrices, fmt.
author: Tin Dang
The ws-plane branch added the fuzz target binary but not its CI matrix entries (Testing rule: every new decoder fuzz target runs in CI). author: Tin Dang
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
📝 WalkthroughWalkthroughAdds Wave B replication for the workspace registry. WS.CREATE and WS.DROP now route mutations through shard 0, replicate via WAL and live fanout, transfer registry state in full-resync snapshots, and apply deterministic workspace metadata on replicas. ChangesWorkspace registry replication
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant Client
participant WSCommandHandler
participant ShardZero
participant ReplicationBacklog
participant Replica
Client->>WSCommandHandler: WS.CREATE or WS.DROP
WSCommandHandler->>ShardZero: route registry mutation
ShardZero->>ReplicationBacklog: append WAL payload and advance offset
ReplicationBacklog->>Replica: deliver WS apply command
Replica->>Replica: install workspace metadata or remove workspace keys
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (1)
src/server/conn/handler_monoio/write.rs (1)
98-106: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick winConfirm the inline vs. shard-0-hop replication paths are intentionally divergent.
The same logical WS.CREATE/DROP produces a different replication record depending on which shard the client connection landed on:
- Inline (shard 0 connection):
record_local_write_db(ctx, conn.selected_db, ...)— in a multi-shard master this fuses aSELECT <selected_db>prefix ahead of the (db-agnostic)WS.*.APPLYrecord, and is gated onreplication_fanout_active(ctx).- Hop (non-shard-0 connection): the
WsRegistryCreate/WsRegistryDroparm insrc/shard/spsc_handler.rscallsrecord_ws_registry_write, which emits noSELECTprefix and is gated only onfanout_hint_active().Two consequences worth verifying: (1) a spurious
SELECTon the inline path for a db-agnostic record, and (2) the differing fanout gate means the master replication offset can advance under different conditions for the two paths. Since these encode the same operation, aligning both paths (prefer the db-agnosticrecord_ws_registry_writeshape and one shared gate) avoids stream/offset divergence between otherwise-identical clients.Also applies to: 177-186
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/server/conn/handler_monoio/write.rs` around lines 98 - 106, Align the inline WS.CREATE/DROP replication handling in the write path with the shard-0-hop handling: use the same db-agnostic registry-write recording mechanism as the WsRegistryCreate/WsRegistryDrop arms and the same fanout gate as the hop path. Update both highlighted inline branches, replacing record_local_write_db and its replication_fanout_active check while preserving the existing serialized operation payload.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/server/conn/handler_monoio/write.rs`:
- Around line 98-106: Align the inline WS.CREATE/DROP replication handling in
the write path with the shard-0-hop handling: use the same db-agnostic
registry-write recording mechanism as the WsRegistryCreate/WsRegistryDrop arms
and the same fanout gate as the hop path. Update both highlighted inline
branches, replacing record_local_write_db and its replication_fanout_active
check while preserving the existing serialized operation payload.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 7b5b106b-6566-4076-9766-17bb69bc6e46
📒 Files selected for processing (19)
.github/workflows/fuzz.ymlCHANGELOG.mdfuzz/Cargo.tomlfuzz/fuzz_targets/ws_registry_record.rssrc/persistence/redis_rdb.rssrc/replication/apply.rssrc/replication/master.rssrc/replication/mod.rssrc/replication/replica.rssrc/replication/ws_sync.rssrc/server/conn/handler_monoio/dispatch.rssrc/server/conn/handler_monoio/ft.rssrc/server/conn/handler_monoio/write.rssrc/server/conn/handler_sharded/dispatch.rssrc/shard/dispatch.rssrc/shard/spsc_handler.rssrc/workspace/mod.rssrc/workspace/repl.rstests/replication_ws.rs
pilotspacex-byte
added a commit
that referenced
this pull request
Jul 12, 2026
… Stream codec data-loss fix (Wave B stage 2b, task #34) (#294) * fix(persistence): PSYNC RDB Stream codec discarded all content (data loss) The FULLRESYNC RDB codec (persistence::redis_rdb, distinct from the persistence::rdb SAVE/BGSAVE codec) serialized Stream values as a placeholder "__stream__:<len>" string — entries, last_id, consumer groups, PEL state, durable flag, and max_delivery_count were all silently discarded on every replica full resync. A durable MQ queue snapshotted this way would come back empty on the replica. Add a full RDB_TYPE_STREAM_MOON (0xC8) codec, ported from persistence::rdb's battle-tested Stream serializer: entries, last_id, consumer groups (PEL + per-consumer pending), durable flag, and max_delivery_count round-trip losslessly. Every new count field is bounds-checked via check_alloc_bound before allocating, matching the existing DoS-avoidance discipline in this module. Also adds the MOON_AUX_MQ_REGISTRY aux-field constant used by the Wave B stage 2b MQ replication work landing in subsequent commits. author: Tin Dang * feat(mq): replication wire-framing constants + generalized apply engine Preparatory plumbing for Wave B stage 2b MQ replication: - src/mq/wal.rs: five synthetic MQ._REPL.CREATE/PUSH/POP/ACK/TRIGGER pseudo-command names plus is_mq_replay_command(), used to frame the existing encode_mq_*/decode_mq_* WAL payload bytes on the wire and to route them on the replica side (never dispatched to real clients). - src/mq/trigger.rs: TriggerRegistry::iter(), needed to snapshot every registered trigger for the FULLRESYNC registry blob landing in a later commit. - src/shard/shared_databases.rs: generalize apply_mq_create/push/pop/ ack/trigger over a new MqApplyTarget trait (mq_databases_mut() / mq_durable_queue_registry_mut() / mq_trigger_registry_mut()), implemented for both ShardSliceInit (boot-time WAL replay) and ShardSlice (live replica apply, added next). One apply engine now serves both boot recovery and replica replication instead of two. author: Tin Dang * feat(replication): MQ replica-side apply engine + FULLRESYNC snapshot leg Wave B stage 2b, replica-consuming half. Two pieces: 1. Live-record apply (src/replication/apply.rs): apply_local routes any MQ._REPL.* record to a new apply_mq(), which decodes the single bulk-string payload with the existing decode_mq_* functions and dispatches to the SAME apply_mq_* engine (shard::shared_databases, generalized last commit) boot-time WAL replay uses. A replica never fires MQ.TRIGGER callbacks on apply — registrations are stored as opaque data, matching boot-time replay. 2. FULLRESYNC snapshot leg (src/replication/mq_sync.rs, new): a per-shard MOON_AUX_MQ_REGISTRY blob carries the durable-queue registry and trigger registry — shard-level bookkeeping that lives outside the keyspace (Stream CONTENT rides the ordinary RDB body, covered by the redis_rdb.rs Stream codec fix). Wired at both master capture sites (single-shard PSYNC in master.rs, and the merged multi-shard path via a new PreparedShardSync.mq_blob field populated in spsc_handler.rs) and installed additively into every replica shard by load_snapshot (apply.rs), mirroring graph_sync::install_graph_store_many's precedent exactly: an EMPTY blob list (pre-MQ-replication master) warns-and-keeps local state; a non-empty list is authoritative and replaces it. author: Tin Dang * feat(replication): MQ master-side live-stream emission (Wave B stage 2b) Emits MQ.* effect records to attached replicas at the moment they're durably WAL-logged on the owner shard, closing the plane gap warn_unreplicated_plane previously fail-loud-warned about for MQ.* (the WS.* half of that warning is unaffected — a separate agent tracks retiring it once WS replicates). - src/replication/state.rs: record_local_write_global / record_local_write_db_global — ctx-free counterparts of record_local_write(_db) that resolve the same per-process ReplicationState Arc via admin::metrics_setup::get_global_repl_state_arc(), needed because shard::mq_exec runs on the shard's own OS thread without a ConnectionContext in scope. Also adds ReplicationState::num_shards(). - src/shard/mq_exec.rs: replicate_mq_record(), called from all five owner-shard handlers (CREATE/PUSH/POP/ACK/TRIGGER) right where each already builds its WAL payload — ships the IDENTICAL encoded bytes as one of the MQ._REPL.* pseudo-commands, gated single-shard-only (num_shards == 1), the same posture graph's own live path takes for its multi-shard gap. monoio-only; a no-op under runtime-tokio. - src/server/conn/handler_monoio/txn.rs: MQ.PUBLISH's TXN-materialization leg replicates the same way at commit, on the connection's own thread (which — since owner == ctx.shard_id in this leg — IS the owning shard's thread). - src/server/conn/handler_monoio/write.rs: retire the MQ half of warn_unreplicated_plane's call site now that MQ replicates in the single-shard configuration; the WS.CREATE/DROP call site is untouched. A multi-shard master still does not live-stream MQ writes (durability is unaffected — WAL + FULLRESYNC still cover it); tracked as a known follow-up alongside graph's own multi-shard live-stream gap. author: Tin Dang * test(mq): replication_mq.rs integration suite + 2 new fuzz targets tests/replication_mq.rs, modeled on tests/replication_graph.rs + tests/replication_planes.rs (single-shard master scope, matching graph's own precedent): - replica_syncs_mq_live_stream: MQ.CREATE/PUSH/ACK on the master reach an already-attached replica, verified via MQ.POP on the replica side (proves both registry state and message content replicated, not just the raw key) plus a stream-health assertion (zero reconnects). - replica_backfills_mq_from_snapshot: a replica attaching to a master that already has MQ state backfills it from PSYNC FULLRESYNC (the redis_rdb.rs Stream codec fix is what makes this pass), then a live write after the snapshot boundary still lands. - multi_shard_live_mq_write_not_streamed: pins the documented num_shards==1 live-stream gate down with an explicit RED-if-lifted test rather than leaving the limitation untested. Two new cargo-fuzz targets (wired into both the PR and nightly matrices in .github/workflows/fuzz.yml, per the project's "any new parser/decoder needs a fuzz target" rule): - mq_registry_blob: fuzzes install_mq_registry_many (the new MOON_AUX_MQ_REGISTRY blob decoder) on a throwaway per-iteration ShardSlice, mirroring mq_wal_record.rs's coverage of the sibling WAL op-blob decoders. - redis_rdb_load: fuzzes persistence::redis_rdb::load_rdb directly (the FULLRESYNC RDB loader, including the new RDB_TYPE_STREAM_MOON tag) — distinct from the existing rdb_load.rs target, which covers the separate persistence::rdb SAVE/BGSAVE codec. author: Tin Dang * docs(changelog): MQ-plane replication entry (Wave B stage 2b) author: Tin Dang * docs(changelog): mixed-version FULLRESYNC operator note for Stream codec fix Adversarial review of the mq-repl branch (verdict SHIP, P2 finding): the RDB_TYPE_STREAM_MOON fix changes the PSYNC wire format for durable Streams, so mixed-version pairs misbehave in both directions — old replica vs new master hard-fails FULLRESYNC (reconnect-loops until upgraded), new replica vs old master absorbs the legacy placeholder as a corrupted string. Document the upgrade-together requirement where operators will see it. author: Tin Dang * docs: retire warn_mq_unreplicated remnants after WS+MQ plane composition Rebase-composition tidy: with both the WS plane (PR #293) and this branch's MQ live stream shipped, the round-2 warn_unreplicated_plane fail-loud marker has no surviving half — the renamed warn_mq_unreplicated lost its last call site during the rebase, so the function is removed (dead code under -D warnings) and the doc comment in command/mq.rs plus the WS-plane CHANGELOG entry that still claimed "warn_mq_unreplicated survives for MQ" are corrected. author: Tin Dang * test(mq): replica observation via reads + failover-promotion proof Rebase-composition fix: the replication_mq suite verified replica state by issuing MQ POP against the replica, and readonly enforcement (PR #292, merged after this branch was cut) now correctly rejects that with -READONLY — MQ POP claims PEL entries and advances last_delivered, a genuine write. The enforcement is right; the tests were exercising a hole it closed. Replica-side verification now uses read-only stream commands (XRANGE for content, XPENDING summary for the consumer-group PEL), which also makes the assertions STRONGER: the live-stream test now observes the master's MqPop claim land in the replica PEL (1 pending) and the MqAck drain it (0 pending) directly, instead of inferring both from what a replica-local POP re-surfaced. Both tests then add the operational failover proof — REPLICAOF NO ONE and a real MQ POP on the promoted node — which exercises the replicated durable flag, consumer-group state, and last_delivered cursor end-to-end the way an actual failover would. 3/3 green locally (macOS release, --include-ignored). author: Tin Dang * fix(replication): clamp MQ apply db index + assert truncation rejection Two CodeRabbit findings on PR #294: 1. apply_mq used the payload's db index unclamped — a replica configured with fewer logical dbs than its master would silently no-op the stream mutation (MqApplyTarget::mq_databases_mut().get_mut(oob) returns None): CREATE registered the queue but never created the stream, PUSH/POP/ACK were dropped. Now clamped with the same posture (and debug log) as the generic KV clamp in apply_local. Boot-time WAL replay is unaffected — a shard replays its own records, whose db indices are valid by construction. 2. The RDB stream truncation test discarded the decode result, proving only "no panic". Every strict prefix of the length-prefixed encoding must hit EOF, so it now asserts is_err() at every cut point. Gates: redis_rdb unit tests 30/30, replication_mq 3/3, clippy, fmt. author: Tin Dang --------- Co-authored-by: Tin Dang <tindang.ht97@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Wave B WS plane (task #34): the process-global workspace registry is now deterministic, replicated, and snapshot-covered. Before this, WS.CREATE/DROP mutated the registry from any conn thread, minted nondeterministic UUIDv7s, never streamed to replicas, and had no FULLRESYNC coverage — a replica's registry was permanently empty (and, until PR #292, silently mutable by clients).
Design
ShardMessage::WsRegistryCreate/Drop, mirroring the WsDropCleanup hop). Zero.awaitinside the write arm and the PrepareReplicaSync arm → a WS.CREATE racing a snapshot lands in exactly one of {blob, stream} (R2 offset-cut lesson; verified by line-trace in adversarial review).WS.CREATE.APPLY/WS.DROP.APPLYpseudo-commands carry the exact WAL-encoder bytes; recognized only inapply_local(unreachable from client dispatch — traced both runtimes; not in any phf table). Replica applies the MASTER's ws_id + created_at verbatim (from_bytes, nevernew_v7).MOON_AUX_WORKSPACE_REGISTRYblob (versioned, bounds-checked decode, new fuzz targetws_registry_recordregistered in both CI fuzz matrices); install is authoritative-replace; multi-shard PSYNC aborts if shard 0's leg fails rather than proceeding blobless.Adversarial review (independent agent): SHIP-WITH-FIXES — both findings fixed in-branch
removed=falseeven when shard 0 completed the drop, permanently skipping the{ws_hex}:key sweep (unretryable leak — the registry entry is already gone). Same registry probe: entry gone → run the (idempotent) sweep.Verification
tests/replication_ws.rs: live-stream, snapshot-backfill, WS.DROP propagation, 16-iteration cross-shard hop consistency — green macOS + Linux VM (fresh ELF, pinned MOON_BIN), plusreplication_graph,replication_planes(11),shardslice_live(6, incl. both ssm3 WS restart legs), lib 4181/4181 post-rebase.-D warnings, fmt, CHANGELOG.Follow-up PR (last Wave B piece): MQ plane replication (stage 2b), already built and under review.
Summary by CodeRabbit
New Features
Bug Fixes
Tests