Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Refactor operations into separate components by framework #826
This change pulls out the various framework-specific preprocessor conditional logic into a class hierarchy that selects the appropriate operations (CUDA, NCCL, DDL, Hierarchical) based on available libraries and parameter selections.
Additionally, it abstracts much of the MPI-specific concepts into a CommunicationContext interface that can be substituted for a different framework for orchestration as desired. This PR still retains coupling between operations.cc and MPI, but subsequent changes in the roadmap will further decouple operations.cc to use the CommunicationContext interface.
alsrgv left a comment
It may also help to introduce this change more gradually via a series of pull requests. E.g., first do the change to rename