ScaleApex This repo implements a simple way to combine apex and fairscale. Some background can be found here. Installation git clone https://github.com/amsword/scaleapex cd scaleapex python setup.py build develop Example mpirun -n 2 python example/example.py