Skip to content

RDMA fixes, including memory deregistration - #191

Merged
ritvikrao merged 2 commits into
mainfrom
rdma-ack-fix
Jul 28, 2026
Merged

RDMA fixes, including memory deregistration#191
ritvikrao merged 2 commits into
mainfrom
rdma-ack-fix

Conversation

@ritvikrao

@ritvikrao ritvikrao commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Intended to fix issue #188. Reconverse was inadvertently overrwriting CMK_SRC_DEST_ACK to CMK_SRC_ACK, causing an error with destination ACKing. This fixes that (allowing the pingpong results to show for nodegroups) but isn't complete yet, there needs to be code to clean up RDMA buffers. currently this error happens at the end:

------- Processor 0 Exiting: Called CmiAbort ------
Reason: Unhandled C++ exception in user code: 0:55075:0:/Users/ritvik/charm_reconverse/reconverse-darwin-arm8/_deps/lci-src/src/network/ofi/backend_ofi.cpp:~ofi_device_impl_t:327<lci:Assert failed: false> err : Resource busy (/Users/ritvik/charm_reconverse/reconv
/Users/ritvik/charm_reconverse/reconverse-darwin-arm8/_deps/lci-src/lcrun: line 39: 55075 Segmentation fault: 11  "$@"
/Users/ritvik/charm_reconverse/reconverse-darwin-arm8/_deps/lci-src/lcrun: line 39: 55076 Segmentation fault: 11  "$@"

update: All RDMA tests now pass and pingpong runs correctly. This pr changes 2 things:

  • adds CK_BUFFER_DEREG to charm-config.h, and other changes to allow Charm++ to use conv-rdma.h from reconverse, not old converse
  • adds logic to deregister all RDMA buffers on exit

Using these changes requires the reconverse-rdma-fixes branch of Charm++; once this is merged into main, the reconverse-rdma-fixes branch will be merged into reconverse-specific-build in the charm repo

@ritvikrao
ritvikrao requested a review from lvkale July 28, 2026 15:21
@ritvikrao
ritvikrao marked this pull request as ready for review July 28, 2026 15:25
@ritvikrao ritvikrao changed the title WIP: trying to fix ack issue in RDMA implementation RDMA fixes, including memory deregistration Jul 28, 2026
@lvkale

lvkale commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Create a sort-of benign dependency on pup, but it is ok I think.

@ritvikrao
ritvikrao merged commit f1e3f0f into main Jul 28, 2026
2 checks passed
@ritvikrao
ritvikrao deleted the rdma-ack-fix branch July 28, 2026 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants