refactor(cco): single-header consolidation + shmem decoupling + self-contained bootstrap#375
Merged
Conversation
Adds coverage for ccoMemAlloc and ccoWindowRegister to the SPMT
(single-process multi-thread) test, alongside the existing
ccoDevCommCreate connection-type validation:
Phase 1 ccoCommCreate
Phase 1.5 ccoMemAlloc edge cases:
- size==0 returns nullptr
- alloc/free/alloc reuses the same slot (freelist)
Phase 2 ccoWindowRegister × 2 overloads:
- overload B: pre-allocated ptr (4 MiB; exercises VMM
sub-buffer regression path from rocm-systems#2516)
- overload A: convenience (library does internal alloc)
- flat-VA reverse-lookup verifies the registered window
resolves back to the original buffer
Phase 3 ccoDevCommCreate × {NONE, FULL, RAIL} (existing)
Teardown reverse order: DevCommDestroy → WindowDeregister →
MemFree → CommDestroy
All 8 ranks pass in SPMT mode on patched ROCm 7.2.x.
Add application::RdmaEndpointDevice — the device-visible projection of application::RdmaEndpoint — to the device-safe application_device_types.hpp, beside the host RdmaEndpoint it mirrors. It is composed purely of application:: + core:: types (vendorId, qpn, work/completion-queue handles, atomic ibuf), so RDMA-driving backends (shmem, cco, ...) depend on it directly rather than on each other. shmem::ShmemRdmaEndpoint becomes a using-alias of application::RdmaEndpointDevice, preserving existing shmem code unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Merge the cco public surface into one header, include/mori/cco/cco.hpp
(OPUS-style single include), and delete the former split headers:
cco_types / cco_coop / cco_team / cco_lsa_types / cco_lsa_impl / cco_device
gda/{gda_device, gda_device_types, gda_device_primitives, gda_device_api}
Structure inside cco.hpp (single-file ordering = dependency layering):
1. shared types (host+device; host-only STL structs under !__HIPCC__ guard)
2-5. device-side API (coop, teams, LSA barrier, GDA layer) under __HIPCC__
6. host control-plane API prototypes (impl stays in src/cco/cco_init.cpp)
Also in this change:
- Decouple cco entirely from shmem: use application::RdmaEndpointDevice,
drop the mori/shmem/internal.hpp include (and from the gda tests).
- Drop the nested mori::cco::gda namespace; GDA now lives directly in
mori::cco (consistent with the LSA layer; the ccoGda* prefix is the
namespace). Unifies the two ccoWindow_t typedefs onto ccoWindowDevice*.
- Move the GDA implementation (primitive layer + peer<->world translation)
into mori::cco::impl, keeping the public surface (ccoGda facade, ABI
types, action structs) clean.
- Trim includes: forward-declare application::BootstrapNetwork (referenced
only by pointer) instead of pulling the mpi/torch/socket bootstrap headers;
drop anvil_device/hip_compat direct includes (transitively provided by
application_device_types.hpp).
- Update cco tests and examples to include only mori/cco/cco.hpp.
Verified: builds (host CXX + device HIP) and passes the cco test suite
single-node (test_cco_host, test_cco_gda_modes) and 2-node internode
(test_cco_multiprocess, 16 ranks) on MI300X + BNXT.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
cco.hpp now forward-declares application::BootstrapNetwork instead of pulling the bootstrap headers, so the LSA examples (which construct an application::MpiBootstrapNetwork) must include it themselves — matching how the cco tests already include the bootstrap header they use. Verified: lsa_barrier (10/10), lsa_memcheck, lsa_allreduce all build and pass under mpirun -np 8 on MI300X + BNXT. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a POD ccoUniqueId plus ccoGetUniqueId() and an overload ccoCommCreate(const ccoUniqueId&, int nRanks, int rank, size_t vmm, ccoComm**) so a communicator can be bootstrapped with only mori/cco/cco.hpp — no bootstrap header at the call site. Rank 0 generates the id (free port chosen by random probe-bind, interface from MORI_SOCKET_IFNAME), broadcasts it out-of-band, and every rank passes it in; cco builds and owns the socket bootstrap internally. The ccoCommCreate(BootstrapNetwork*, ...) overload is retained for pluggable transports. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… coverage lsa_barrier / lsa_memcheck / lsa_allreduce now bootstrap through ccoUniqueId: MPI is only the launcher plus a one-shot MPI_Bcast of the id, and each example finalizes MPI itself (the self-contained socket bootstrap does not). Also add a [twait] case to lsa_barrier that drives the timeout path through a direct arrive() + wait(coop, timeoutCycles), complementing the existing [timeo] case that uses sync(coop, timeoutCycles); lsaBarrierCount bumped 10 -> 11. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
jhchouuu
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.