refactor(bnxt): collapsed-CQ drain, drop outstandingWqe, fix per-lane PSN#395
Merged
Conversation
bnxt runs a one-CQE collapsed CQ (maxCqeNum=1): the NIC keeps the latest completion in CQE[0]. Mirror the mlx5 collapsed-CQ approach so bnxt no longer needs the per-WQE wq.outstandingWqe[] slot->counter table. - Add BnxtCollapsedCqDrain<DrainToLive>: reconstruct the 32-bit monotonic doneIdx arithmetically from CQE[0].con_indx instead of a table lookup. bnxt wraps by sqWqeNum (a power of two), so anchor on dbTouchIdx and pick the unique V <= dbTouchIdx with V % sqWqeNum == con_indx % sqWqeNum (unambiguous because outstanding <= sqWqeNum). Lock-free / multi-warp safe via atomicMax. - ShmemQuietThreadKernelSerialImpl is now templated on DrainToLive and just calls the drain; drop the per-warp poll lock and cq_consumer bookkeeping. The final per-pe quiet now waits on the live postIdx (DrainToLive=true). - Remove all bnxt wq->outstandingWqe[] writes. - Drop the now-dead outstandingWqe[OUTSTANDING_TABLE_SIZE] member (and the define) from WorkQueueHandle: 512 KB per work queue that was copied to the GPU, no longer referenced by any provider. Validated on MI300X + bnxt (MORI_DISABLE_P2P=ON): p2p put/get bandwidth and latency across block/warp/thread scopes, concurrent put/get/imm correctness examples, and the EP internode_v1 dispatch/combine suite (113 passed).
Removing the outstandingWqe[OUTSTANDING_TABLE_SIZE] member outright broke the dist_rdma_ops example (the only other user), which fails the BUILD_EXAMPLES=ON build. Instead, make WorkQueueHandle::outstandingWqe a pointer (8 bytes) left null by mori_shmem, so the GPU-resident handle no longer carries the 512 KB inline table. The dist_rdma_ops example allocates a per-QP device buffer and attaches it; its kernels are unchanged (pointer indexing == array indexing). Verified the full ci.yml-style build (BUILD_BENCHMARK=ON BUILD_EXAMPLES=ON pip install .) and the MORI-CPP IBGDA suite on MI300X + bnxt: put/get/imm, put-imm, put-signal, atomic non-fetch and atomic fetch all pass, plus EP internode_v1 (113 passed).
…rite Remove the outstandingWqe member from WorkQueueHandle entirely and reconstruct the completed counter arithmetically in the dist_rdma_ops example, matching how mori_shmem now drains the CQ.
The bnxt put/get/putSignal thread kernels assumed all active lanes transfer the same size (psnCnt * laneId / psnCnt * num_active_lanes), which gaps/overlaps the MSN-table PSN ranges when lanes differ. Reserve the warp-wide sum of per-lane psnCnt and place each lane at the exclusive prefix sum. Identical to the old values for uniform sizes; correct when sizes vary.
Avoid every warp polling the uncached CQE[0] and contending on doneIdx: take cq.pollCqLock so one warp drains. Recycle gate losers return (caller retries); final-quiet losers wait on doneIdx until their reserved WQEs complete.
jhchouuu
changed the title
jhchouuu
added a commit
that referenced
this pull request
Jun 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Apply the mlx5 collapsed-CQ approach to bnxt (one-CQE mode) and fix two related issues.
Validated on MI300X + bnxt: IBGDA put/get/imm/signal/atomic, p2p bandwidth/latency, and EP internode_v1 (113 passed).