feat(storage): add optional RDMA piece transport over libfabric

[YQ-Wang]

Description

Adds an optional peer-to-peer piece transport that carries piece bytes over an RDMA
fabric instead of the TCP piece server. It is behind the rdma cargo feature, disabled
by default, Linux-only, and every failure falls back to the TCP piece server for that
piece.

Design, in short (full write-up in docs/rdma-p2p.md):

• libfabric, not ibverbs directly. AWS EFA uses Scalable Reliable Datagram and has
  no reliable connected queue pairs, so an RC-based ibverbs implementation cannot drive
  it. libfabric covers both EFA and RoCE/InfiniBand (verbs;ofi_rxm) behind one API.
• Split planes. Control messages (request, capability negotiation, metadata, flow
  control, errors) go over a TCP rendezvous connection; bulk bytes go as two-sided
  tagged messages on a shared FI_EP_RDM endpoint.
• Two-sided, not one-sided. RDMA READ/WRITE would mean handing remote-access memory
  keys to peers, which in a P2P system means handing them to anyone who can reach the
  daemon. With two-sided tagged messaging the receiver posts its own buffers and no
  remote key leaves the process.
• Discovery on the existing TCP piece port, via a four-byte discriminator, so no
  announcement or scheduler change is needed. It is fail-closed: an older peer, a peer
  built without the feature, or a peer whose fabric has failed all leave the client on
  TCP. Answers are cached for 60s.
• Fabric tag. Peers speak RDMA only when they agree on the concrete provider and
  on an operator-supplied reachability label, because libfabric will happily report a
  working provider for two nodes that cannot actually reach each other.
• Zero bounce buffer on receive. Windows are handed to storage still resident in
  registered memory, so a piece goes from the NIC to the page cache without an
  intermediate copy. The parent may not send a window until the client reports its
  receives posted, and the parent recomputes each window itself, so a peer cannot
  replay a window, skip ahead, or over-claim chunks.
• Bounded pinned memory. Registrations are pooled under a configurable budget;
  exhausting it degrades a transfer to one window at a time rather than failing it.

New config lives under storage.server.rdma (12 options, all documented in
docs/rdma-p2p.md), plus download.protocol: rdma to download over it. Serving and
downloading are enabled independently.

Scope: 24 files, +7507/−17. No behaviour changes when the feature is off — the
default build and the TCP path are untouched.

Related Issue

There is no existing issue for this; I am opening this as a draft to start the
discussion, and happy to file one if you would prefer that first.

Related prior art in the project: dragonflyoss/api defines
ExchangeIBVerbsQueuePairEndpoint on DfdaemonUpload, and piece_collector.rs still
carries a comment describing download_ip as the channel for exchanging IBVerbs queue
pair endpoints. This PR deliberately does not use that RPC (see below), so if the
IBVerbs direction is the intended one I would like to know before going further.

Motivation and Context

On a host with an RDMA fabric, a single TCP flow leaves most of that fabric idle, and
the piece protocol opens one connection per piece. For model and image distribution on
GPU nodes — exactly where the fast fabric was bought — that shortfall is the bottleneck.

Measured on two p6-b200.48xlarge EFA nodes (one rail), 24 GiB of 512 MiB pieces served
from tmpfs, best of three runs at each concurrency, in Gbps:

Concurrent pieces	1	2	4	8	16	32
RDMA, transport only	44.7	78.9	122.5	198.9	261.0	277.0
TCP, transport only	4.1	8.0	14.6	27.6	50.2	74.4
Speedup	10.8×	9.9×	8.4×	7.2×	5.2×	3.7×
RDMA, CRC32 + write	22.1	39.0	72.0	118.1	139.2	130.1
TCP, CRC32 + write	3.5	6.8	13.1	24.2	42.3	61.0
Speedup	6.2×	5.7×	5.5×	4.9×	3.3×	2.1×

"CRC32 + write" is the work dfdaemon actually does per piece; "transport only" isolates
the wire. The gap narrows with concurrency because a single TCP flow on this network
caps near 5 Gbps while the piece server uses one connection per piece, so TCP scales
with streams as RDMA saturates. Past roughly 16 streams the RDMA side is bound by
receive-side CPU (digest and write), not by the fabric, which accounts for under 1% of
a transfer.

Testing

• 11 integration tests in dragonfly-client-storage/tests/rdma_transfer.rs plus unit
  tests covering framing, capability negotiation, window validation, and the write path.
• Builds and passes clippy both with and without --features rdma.
• On hardware: 48 pieces verified against SHA-256 digests across the sweep above.

Known limitations

• Linux only; the feature rejects other targets at compile time.
• Measured on a single EFA rail — no striping across multiple devices yet.
• fabricTag is an operator assertion; a wrong tag degrades to TCP fallback rather
  than failing loudly.
• The fabric is unauthenticated, as fabrics generally are. A peer on the same fabric
  could aim bytes at another peer's endpoint; the digest check catches it, so the
  effect is a failed download and a TCP retry rather than corruption.
• Two follow-ups I have deliberately left out of this PR: the client collapses the
  parent's BUSY and INCOMPATIBLE rendezvous codes into one generic transport
  failure, so a merely loaded parent gets the same escalating backoff as a broken one;
  and the downloader returns success once the Ready frame parses, so a transfer that
  dies mid-stream never reaches the backoff bookkeeping.