feat(io): single-transfer chunking + adaptive multi-NIC striping

[maning00]

Summary

A single RDMA transfer was posted as one WR on one QP/NIC, capping at single-outstanding bandwidth (~28 GB/s) regardless of message size. This PR:

• Single-transfer chunking (size-adaptive): split a transfer into ~chunkBytes chunks (default 64KB, capped by maxChunks) and round-robin them across the session's QPs to pipeline. Messages ≤ chunkBytes are unchanged. Enabling chunking uses the inline post path (single posting thread), avoiding worker-thread handoff latency.
• Adaptive multi-NIC striping by memory type: GPU stays single-NIC (PCIe-bound); host memory stripes across NUMA-local NICs.

All new behavior is config/env gated.

Performance

All numbers: 2 nodes, 400G ionic, qp_per_transfer=4, op=write (deployment config).

GPU memory (single NIC) — before = main (4 worker), after = chunking (auto single-worker inline)

Scenario	Before (main)	After
1 MB, single transfer	76 us / 13.7 GB/s	38 us / 27.8 GB/s (~2x)
1 GB, single transfer	25.0 GB/s	47.2 GB/s (~1.9x)
batch (1 MB blocks)	48.0 GB/s	47.7 GB/s

A single QP never saturates the NIC (~28 GB/s) even at GB scale; chunking lifts it to the PCIe/line ceiling (~47 GB/s). GPU multi-NIC is PCIe-bound (single GPU Gen5x16 ≈ one NIC) → defaults to 1.

Host/CPU memory (chunking on) — multi-NIC + QP-depth scaling

Host DRAM has no single-x16 chokepoint, so throughput scales with both NIC count and QPs-per-NIC across NUMA-local NICs.

Single transfer (8 MB block):

	1 NIC	NUMA-local 4 NIC
8 MB	44.3 GB/s	88.9 GB/s (~2x)

Batch write (256 × 1 MB blocks), peak BW @ 1 MB block:

NICs × QP/NIC (total qp_per_transfer)	Max BW	Avg BW
1 NIC × 4 QP (baseline)	47.2	44.8
4 NIC × 1 QP (qp=4)	124.3	97.2
4 NIC × 2 QP (qp=8)	164.7	162.7
4 NIC × 4 QP (qp=16)	174.7	171.9
4 NIC × 8 QP (qp=32)	176.1	175.0
8 NIC × 2 QP (qp=16)	192.8	177.1
8 NIC × 4 QP (qp=32)	197.6	192.1

• Multi-NIC striping scales host batch throughput ~4x (47 → 176 GB/s on 4 NICs, → 198 GB/s on 8 NICs).
• For a fixed NIC set the 1→2 QP/NIC step matters most; ≥4 QP/NIC saturates the links (4-NIC ceiling ≈176, 8-NIC ≈198) and converges Avg→Max (run-to-run stability). The remaining ceiling is NIC count, not QP depth.
• The deployment default qp_per_transfer=4 over 4 NICs is only 1 QP/NIC; raising qp_per_transfer lets chunking spread more QPs/NIC and unlocks the headroom above.

Tests

• New C++/Python unit tests: chunk planning, QP-count distribution, NIC interleave, memory-type NIC resolution, config validation, defaults.
• Functional byte-for-byte validation across read/write x GPU/CPU x single/multi-NIC; batch and single-transfer.