Skip to content

feat: add URMA transport support - #3428

Open
zchuango wants to merge 3 commits into
apache:masterfrom
LinQuickDev:urma_transport
Open

feat: add URMA transport support#3428
zchuango wants to merge 3 commits into
apache:masterfrom
LinQuickDev:urma_transport

Conversation

@zchuango

@zchuango zchuango commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

About discussion:

#3167 —— Socket-Transport 架构改造共识
#3217 —— 路线 B(URMA-UrmaTransport)与路线 C(双后端)共识
#3226 —— OBMM/ShmTransport 实现方案
本提案对应 #3217 中的路线 B,并与#3226 的 OBMM-ShmTransport 形成路线 C(双后端协同)。

Issue Number: Related to #3401

Problem Summary:

brpc does not currently provide a remote-memory transport based on the UMDK
URMA API.

This PR adds URMA (Unified Remote Memory Access) as an optional transport in
the existing Transport framework. TCP is used for connection establishment
and URMA capability/resource negotiation. After both peers successfully
negotiate URMA support, the data path switches to URMA. When URMA is
unavailable or negotiation does not succeed, the connection falls back to TCP.

The initial implementation supports the baidu_std protocol only and is
disabled by default.

What is changed and the side effects?

Changed:

  • Add SOCKET_MODE_URMA and UrmaTransport, and integrate them with the
    existing transport, socket, and message-processing framework.
  • Add UrmaEndpoint for URMA resource management, SEND/RECV processing,
    completion handling, and credit-based flow control.
  • Add TCP-based URMA capability and resource negotiation, including v2 binary
    and v3 protobuf handshake formats, with TCP fallback.
  • Add a registered buffer pool for IOBuf blocks and APIs for explicit user
    memory registration.
  • Support completion-event mode and optional busy-polling mode.
  • Support the UMDK bonding extension when the corresponding provider extension
    is available.
  • Add optional URMA build integration for CMake, Make, and Bazel, together with
    a link-time URMA mock for build and test environments without liburma or
    URMA hardware.
  • Add URMA unit tests, English and Chinese documentation, and an
    urma_performance example.

Side effects:

  • Performance effects:

    • URMA is optional and disabled by default. Existing socket modes and their
      default behavior remain unchanged.
    • Enabling URMA maps and registers a configurable IOBuf buffer pool. The
      default configuration is 65,536 buffers of 8 KiB each (512 MiB in total).
    • Busy-polling mode may increase CPU usage because completion pollers
      actively poll the JFC.
    • URMA performance depends on the provider, driver, hardware, and polling
      configuration. This PR does not claim a specific hardware performance
      improvement.
  • Breaking backward compatibility:

    • No intended breaking changes.
    • SOCKET_MODE_URMA and the URMA memory-registration APIs are additive.
      Existing socket modes and their default behavior remain unchanged.

Check List:

@wwbmmm

wwbmmm commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

I'm not familiar with URMA, but it seems that the code in src/brpc/urma is similar to src/brpc/rdma, is that anything we can reuse?

@wwbmmm
wwbmmm requested review from chenBright and a lite review from Copilot August 9, 2026 07:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds an optional URMA (UMDK Unified Remote Memory Access) transport backend to bRPC’s existing Socket/Transport architecture, including a TCP-based negotiation/handshake with transparent TCP fallback when URMA is unavailable or negotiation fails.

Changes:

  • Introduces SOCKET_MODE_URMA, UrmaTransport, and per-connection urma::UrmaEndpoint, plus handshake (v2 binary + v3 protobuf) and a credit-based flow-control model.
  • Adds build-system integration for URMA across CMake/Make/Bazel (including fetching UMDK headers and a link-time URMA mock when liburma/hardware is unavailable).
  • Adds URMA unit tests, documentation (EN/CN), and a new urma_performance example.

Reviewed changes

Copilot reviewed 36 out of 37 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
WORKSPACE Adds Bazel external umdk repo for URMA headers.
MODULE.bazel Adds bzlmod umdk repo rule for URMA headers.
BUILD.bazel Wires URMA define and exposes URMA protos/headers to Bazel build.
bazel/config/BUILD.bazel Adds brpc_with_urma config setting.
bazel/third_party/umdk/umdk.BUILD Defines urma_headers cc_library for UMDK header export.
CMakeLists.txt Adds WITH_URMA build option, header discovery/download, library detection, mock handling, and proto compilation.
Makefile Includes src/brpc/urma sources conditionally; excludes mock when linking real liburma.
config_brpc.sh Adds --with-urma configuration, header/lib detection, and mock selection.
src/brpc/socket_mode.h Adds SOCKET_MODE_URMA.
src/brpc/transport_factory.h Minor API/comment cleanup and parameter naming.
src/brpc/transport_factory.cpp Adds URMA transport creation/context-init plumbing.
src/brpc/socket.h Adds URMA forward decls and friend access for new endpoint/handshake types.
src/brpc/channel.h Updates socket_mode comments to reference SocketMode generally.
src/brpc/channel.cpp Incorporates URMA into channel signature hashing.
src/brpc/server.h Clarifies socket_mode comment for accepted sockets.
src/brpc/input_messenger.h Adds friend access for URMA endpoint.
src/brpc/input_messenger.cpp Ensures URMA-delivered messages are executed off poller threads.
src/brpc/urma_transport.h Declares UrmaTransport composing TcpTransport fallback and managing URMA state.
src/brpc/urma_transport.cpp Implements transport dispatch, epoll-out waiting, message queueing, and context init constraints.
src/brpc/urma/urma_endpoint.h Declares per-socket endpoint state machine, resources, polling/event CQ handling, and data-path APIs.
src/brpc/urma/urma_endpoint.cpp Implements endpoint handshake, CQ polling, SEND/RECV processing, and connect driver.
src/brpc/urma/urma_helper.h Declares global URMA init, polling-mode setup, and memory registration APIs.
src/brpc/urma/urma_helper.cpp Implements global URMA init, buffer-pool + IOBuf allocator hijack, and user memory registration.
src/brpc/urma/urma_handshake.proto Defines protobuf v3 handshake message (UrmaHello).
src/brpc/urma/urma_handshake.h Declares handshake strategy abstraction + v2/v3 implementations and negotiation helpers.
src/brpc/urma/urma_handshake.cpp Implements v2 binary + v3 protobuf handshake serialization/parsing and validation.
src/brpc/urma/urma_bonding.h Optional provider-extension include gate for URMA bonding.
src/brpc/urma/mock_urma.cpp Provides link-time URMA C-API mock for builds without liburma/hardware.
test/brpc_urma_unittest.cpp Adds URMA handshake/helper/mock unit coverage.
example/cmake/BrpcExample.cmake Updates example toolchain to handle newer Protobuf/Abseil and URMA include/defines.
example/urma_performance/CMakeLists.txt Adds URMA performance example build (links liburma if present).
example/urma_performance/test.proto Proto definitions for performance example RPC service.
example/urma_performance/server.cpp URMA/TCP selectable server for performance testing.
example/urma_performance/client.cpp URMA/TCP selectable benchmark client with warmup + worker bthreads.
docs/en/urma.md English documentation for building/using UrmaTransport and configuration flags.
docs/cn/urma.md Chinese documentation for building/using UrmaTransport and configuration flags.
.gitignore Ignores local URMA proposal notes and graph artifacts.
Suppressed comments (1)

src/brpc/urma/urma_helper.cpp:103

  • kIOBufBlockHeaderLen is currently hard-coded to 32, which assumes the current IOBuf::Block layout. Once butil/iobuf_inl.h is included, this can be derived from sizeof(butil::IOBuf::Block) to stay correct if the layout ever changes.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

// ---------------------------------------------------------------------------
// 4-byte ACK: HELLO_ACK_URMA_OK bit.
// ---------------------------------------------------------------------------
TEST(UrmaHandshakeTest, ack_bit_is_rdma_ok) {
Comment on lines +36 to +39
#include "butil/atomicops.h"
#include "butil/containers/flat_map.h"
#include "butil/iobuf.h"
#include "butil/logging.h"
Comment thread WORKSPACE
Comment on lines +282 to +287
git_repository(
name = "umdk",
build_file = "//bazel/third_party/umdk:umdk.BUILD",
remote = "https://atomgit.com/openeuler/umdk.git",
tag = "v26.06.0_CAM",
)
Comment thread MODULE.bazel
Comment on lines +60 to +65
git_repository(
name = 'umdk',
build_file = '//bazel/third_party/umdk:umdk.BUILD',
remote = 'https://atomgit.com/openeuler/umdk.git',
tag = 'v26.06.0_CAM',
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants