feat(ipc): MPSC shared-memory transport with multi-client echo test#23040
Merged
Conversation
ludamad
approved these changes
May 7, 2026
1344f0d to
17b93f4
Compare
Adds a multi-producer, single-consumer shared-memory IPC transport. Internally this is **N SPSC rings (one per producer) + a shared doorbell futex** — there is no shared producer ring. Each client writes only to its own SPSC ring; producers ring the doorbell to wake the consumer, which then checks which ring has data and reads from it. Each client also has its own SPSC response ring. This is the same pattern that existed in earlier IPC iterations and is what \`aztec-wsdb\` will could use when serving both the TS layer and a C++ AVM worker pool from the same world state if we need the performance. Right now though we'll use UDS. This is just if needed in future.
17b93f4 to
e668757
Compare
5 tasks
This was referenced May 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part 1 — MPSC shared-memory transport (originally cl/ipc-1-mpsc-shm)
Adds a multi-producer, single-consumer shared-memory IPC transport. Internally this is N SPSC rings (one per producer) + a shared doorbell futex — there is no shared producer ring. Each client writes only to its own SPSC ring; producers ring the doorbell to wake the consumer, which then checks which ring has data and reads from it. Each client also has its own SPSC response ring.
This is the same pattern that existed in earlier IPC iterations and is what
aztec-wsdbwill/could use when serving both the TS layer and a C++ AVM worker pool from the same world state if we need the performance.Right now though we'll use UDS. This is just if needed in future.
Part 2 — aztec-wsdb binary + IPC supporting code (originally #23035 / cl/ipc-2-wsdb, merged into this PR)
Summary
Adds the standalone
aztec-wsdbbinary plus all supporting code (C++ client library, TS spawner, IPC adapter) needed to move world state out of the Node.js process. This part is inert: nothing yet uses the new binary. A follow-up PR (cl/ipc-3-avm-wsdb-cutover, #23036) cuts the NAPI AVM and the TS world state over to use it.What's added
C++:
barretenberg/cpp/src/barretenberg/wsdb/—aztec-wsdbstandalone binary that runs the world state DB as an IPC server. Same WorldState surface as the in-process NAPI module, but exposed over msgpack via UDS or shared memory.barretenberg/cpp/src/barretenberg/wsdb_client/—WsdbIpcMerkleDBimplementsLowLevelMerkleDBInterfaceover WSDB IPC. The standalone AVM (or NAPI AVM after cutover) will use this in place of an in-processWorldStatereference.barretenberg/cpp/src/barretenberg/ipc/mpsc_shm_{client,server}.hpp— multi-producer single-consumer shared-memory transport (see Part 1 above). Lower latency than UDS for the AVM↔WSDB hop.TypeScript (bb.js):
barretenberg/ts/src/aztec-wsdb/—WsdbBackendspawns theaztec-wsdbbinary and routes msgpack commands via the generatedAsyncApi. ImplementsIMsgpackBackendAsync.barretenberg/ts/src/cbind/cpp_codegen.ts— C++ codegen used byaztec-wsdb'sgenerate.tsto producewsdb_ipc_client_generated.{cpp,hpp}. Small shared updates toschema_visitor/typescript_codegen/rust_codegen.yarn-project:
yarn-project/world-state/src/native/ipc_world_state_instance.ts—IpcWorldStateimplementsNativeWorldStateInstanceover WSDB IPC. Not yet wired in.Why split this way
The full WSDB-out-of-process cutover involves rewiring the NAPI AVM (which currently dereferences an in-process
WorldState*pointer) to talk toaztec-wsdbover IPC, plus replacing TS NAPI WorldState withIpcWorldStateeverywhere it's used. This PR keeps the diff bounded by landing the binary and supporting code first; the cutover lands separately and should be a tiny diff.Verification
aztec-wsdbbuilds:cd barretenberg/cpp/build && ninja aztec-wsdbwsdb_clientstatic library builds:ninja wsdb_clientcd barretenberg/ts && yarn build:esm && yarn build:cjs && yarn build:browser@aztec/world-statetypechecks clean (the only TS errors in the build output are pre-existing onnextin unrelated packages)Stack
This is part of a stack splitting up #21331. Plan:
/mnt/user-data/charlie/.claude/plans/glittery-snuggling-horizon.md.aztec-avm+ CDB IPC server (inert)