Hello, thanks for maintaining sideway.
We are integrating sideway into a transfer engine that needs to register both host memory and CUDA GPU memory for RDMA. Host memory works well with the existing ProtectionDomain::reg_mr API. For GPU memory exported as DMA-BUF, the expected verbs API is ibv_reg_dmabuf_mr(pd, offset, length, iova, fd, access).
Would you be open to exposing this through sideway, for example as a ProtectionDomain::reg_dmabuf_mr(...) / MemoryRegion::reg_dmabuf_mr(...) API?
One related detail: the current rdma-mummy-sys 0.2.3 wrapper used by sideway does not appear to export ibv_reg_dmabuf_mr, even though modern system libibverbs.so does. I am filing the lower-level request there as well.
Our desired usage is:
- host memory:
reg_mr
- GPU DMA-BUF memory:
reg_dmabuf_mr
- both through the same sideway / rdma-mummy binding stack, avoiding a second libibverbs binding in the application
Project-side tracking issue: novitalabs/pegaflow#296
Thank you!
Hello, thanks for maintaining sideway.
We are integrating sideway into a transfer engine that needs to register both host memory and CUDA GPU memory for RDMA. Host memory works well with the existing
ProtectionDomain::reg_mrAPI. For GPU memory exported as DMA-BUF, the expected verbs API isibv_reg_dmabuf_mr(pd, offset, length, iova, fd, access).Would you be open to exposing this through sideway, for example as a
ProtectionDomain::reg_dmabuf_mr(...)/MemoryRegion::reg_dmabuf_mr(...)API?One related detail: the current
rdma-mummy-sys0.2.3 wrapper used by sideway does not appear to exportibv_reg_dmabuf_mr, even though modern systemlibibverbs.sodoes. I am filing the lower-level request there as well.Our desired usage is:
reg_mrreg_dmabuf_mrProject-side tracking issue: novitalabs/pegaflow#296
Thank you!