Skip to content

Fix RDMA chunking for oversized multi-SGE WRs#409

Merged
maning00 merged 2 commits into
mainfrom
fix-rdma-multi-sge-chunking
Jun 18, 2026
Merged

Fix RDMA chunking for oversized multi-SGE WRs#409
maning00 merged 2 commits into
mainfrom
fix-rdma-multi-sge-chunking

Conversation

@maning00

@maning00 maning00 commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Fix RDMA chunk expansion so oversized multi-SGE WRs are split before ibv_post_send.
  • Carry the local ibverbs max_msg_sz capability on RdmaEndpoint without serializing it to peers.
  • Add coverage for multi-SGE chunk geometry, max message limits, SGE boundaries, and disabled-chunking failure behavior.

Background

A large batch can be merged into a multi-SGE WR whose total message length exceeds the local provider max_msg_sz. The old chunker only split oversized single-SGE WRs, so these multi-SGE WRs could bypass chunking and be rejected by the verbs provider. This keeps the merge fast path for small WRs while enforcing the local message-size limit when transfer chunking is enabled.

Validation

  • BUILD_TESTS=1 pip install .
  • ./build/tests/cpp/test_engine

@maning00 maning00 force-pushed the fix-rdma-multi-sge-chunking branch from e638877 to 718a34f Compare June 17, 2026 12:23
@maning00 maning00 merged commit fbde9ae into main Jun 18, 2026
13 checks passed
@maning00 maning00 deleted the fix-rdma-multi-sge-chunking branch June 18, 2026 04:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant