feat: add URMA transport support - #3428
Open
zchuango wants to merge 3 commits into
Open
Conversation
feat: add URMA transport support
Contributor
|
I'm not familiar with URMA, but it seems that the code in |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds an optional URMA (UMDK Unified Remote Memory Access) transport backend to bRPC’s existing Socket/Transport architecture, including a TCP-based negotiation/handshake with transparent TCP fallback when URMA is unavailable or negotiation fails.
Changes:
- Introduces
SOCKET_MODE_URMA,UrmaTransport, and per-connectionurma::UrmaEndpoint, plus handshake (v2 binary + v3 protobuf) and a credit-based flow-control model. - Adds build-system integration for URMA across CMake/Make/Bazel (including fetching UMDK headers and a link-time URMA mock when
liburma/hardware is unavailable). - Adds URMA unit tests, documentation (EN/CN), and a new
urma_performanceexample.
Reviewed changes
Copilot reviewed 36 out of 37 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| WORKSPACE | Adds Bazel external umdk repo for URMA headers. |
| MODULE.bazel | Adds bzlmod umdk repo rule for URMA headers. |
| BUILD.bazel | Wires URMA define and exposes URMA protos/headers to Bazel build. |
| bazel/config/BUILD.bazel | Adds brpc_with_urma config setting. |
| bazel/third_party/umdk/umdk.BUILD | Defines urma_headers cc_library for UMDK header export. |
| CMakeLists.txt | Adds WITH_URMA build option, header discovery/download, library detection, mock handling, and proto compilation. |
| Makefile | Includes src/brpc/urma sources conditionally; excludes mock when linking real liburma. |
| config_brpc.sh | Adds --with-urma configuration, header/lib detection, and mock selection. |
| src/brpc/socket_mode.h | Adds SOCKET_MODE_URMA. |
| src/brpc/transport_factory.h | Minor API/comment cleanup and parameter naming. |
| src/brpc/transport_factory.cpp | Adds URMA transport creation/context-init plumbing. |
| src/brpc/socket.h | Adds URMA forward decls and friend access for new endpoint/handshake types. |
| src/brpc/channel.h | Updates socket_mode comments to reference SocketMode generally. |
| src/brpc/channel.cpp | Incorporates URMA into channel signature hashing. |
| src/brpc/server.h | Clarifies socket_mode comment for accepted sockets. |
| src/brpc/input_messenger.h | Adds friend access for URMA endpoint. |
| src/brpc/input_messenger.cpp | Ensures URMA-delivered messages are executed off poller threads. |
| src/brpc/urma_transport.h | Declares UrmaTransport composing TcpTransport fallback and managing URMA state. |
| src/brpc/urma_transport.cpp | Implements transport dispatch, epoll-out waiting, message queueing, and context init constraints. |
| src/brpc/urma/urma_endpoint.h | Declares per-socket endpoint state machine, resources, polling/event CQ handling, and data-path APIs. |
| src/brpc/urma/urma_endpoint.cpp | Implements endpoint handshake, CQ polling, SEND/RECV processing, and connect driver. |
| src/brpc/urma/urma_helper.h | Declares global URMA init, polling-mode setup, and memory registration APIs. |
| src/brpc/urma/urma_helper.cpp | Implements global URMA init, buffer-pool + IOBuf allocator hijack, and user memory registration. |
| src/brpc/urma/urma_handshake.proto | Defines protobuf v3 handshake message (UrmaHello). |
| src/brpc/urma/urma_handshake.h | Declares handshake strategy abstraction + v2/v3 implementations and negotiation helpers. |
| src/brpc/urma/urma_handshake.cpp | Implements v2 binary + v3 protobuf handshake serialization/parsing and validation. |
| src/brpc/urma/urma_bonding.h | Optional provider-extension include gate for URMA bonding. |
| src/brpc/urma/mock_urma.cpp | Provides link-time URMA C-API mock for builds without liburma/hardware. |
| test/brpc_urma_unittest.cpp | Adds URMA handshake/helper/mock unit coverage. |
| example/cmake/BrpcExample.cmake | Updates example toolchain to handle newer Protobuf/Abseil and URMA include/defines. |
| example/urma_performance/CMakeLists.txt | Adds URMA performance example build (links liburma if present). |
| example/urma_performance/test.proto | Proto definitions for performance example RPC service. |
| example/urma_performance/server.cpp | URMA/TCP selectable server for performance testing. |
| example/urma_performance/client.cpp | URMA/TCP selectable benchmark client with warmup + worker bthreads. |
| docs/en/urma.md | English documentation for building/using UrmaTransport and configuration flags. |
| docs/cn/urma.md | Chinese documentation for building/using UrmaTransport and configuration flags. |
| .gitignore | Ignores local URMA proposal notes and graph artifacts. |
Suppressed comments (1)
src/brpc/urma/urma_helper.cpp:103
- kIOBufBlockHeaderLen is currently hard-coded to 32, which assumes the current IOBuf::Block layout. Once butil/iobuf_inl.h is included, this can be derived from sizeof(butil::IOBuf::Block) to stay correct if the layout ever changes.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| // --------------------------------------------------------------------------- | ||
| // 4-byte ACK: HELLO_ACK_URMA_OK bit. | ||
| // --------------------------------------------------------------------------- | ||
| TEST(UrmaHandshakeTest, ack_bit_is_rdma_ok) { |
Comment on lines
+36
to
+39
| #include "butil/atomicops.h" | ||
| #include "butil/containers/flat_map.h" | ||
| #include "butil/iobuf.h" | ||
| #include "butil/logging.h" |
Comment on lines
+282
to
+287
| git_repository( | ||
| name = "umdk", | ||
| build_file = "//bazel/third_party/umdk:umdk.BUILD", | ||
| remote = "https://atomgit.com/openeuler/umdk.git", | ||
| tag = "v26.06.0_CAM", | ||
| ) |
Comment on lines
+60
to
+65
| git_repository( | ||
| name = 'umdk', | ||
| build_file = '//bazel/third_party/umdk:umdk.BUILD', | ||
| remote = 'https://atomgit.com/openeuler/umdk.git', | ||
| tag = 'v26.06.0_CAM', | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
About discussion:
Issue Number: Related to #3401
Problem Summary:
brpc does not currently provide a remote-memory transport based on the UMDK
URMA API.
This PR adds URMA (Unified Remote Memory Access) as an optional transport in
the existing
Transportframework. TCP is used for connection establishmentand URMA capability/resource negotiation. After both peers successfully
negotiate URMA support, the data path switches to URMA. When URMA is
unavailable or negotiation does not succeed, the connection falls back to TCP.
The initial implementation supports the
baidu_stdprotocol only and isdisabled by default.
What is changed and the side effects?
Changed:
SOCKET_MODE_URMAandUrmaTransport, and integrate them with theexisting transport, socket, and message-processing framework.
UrmaEndpointfor URMA resource management, SEND/RECV processing,completion handling, and credit-based flow control.
and v3 protobuf handshake formats, with TCP fallback.
memory registration.
is available.
a link-time URMA mock for build and test environments without
liburmaorURMA hardware.
urma_performanceexample.Side effects:
Performance effects:
default behavior remain unchanged.
default configuration is 65,536 buffers of 8 KiB each (512 MiB in total).
actively poll the JFC.
configuration. This PR does not claim a specific hardware performance
improvement.
Breaking backward compatibility:
SOCKET_MODE_URMAand the URMA memory-registration APIs are additive.Existing socket modes and their default behavior remain unchanged.
Check List: