CO2: Efficient Distributed Training with Full Communication-Computation Overlap
This repo will be updated soon. Please refer to the below repos for trying CO2:
- fairseq-CO2: shows an example of utilizing CO2 within Fairseq.
- fairscale-CO2: integrates CO2 within Fairscale.