Performance
- Smart schedule of FasterMoE is updated with correct stream management, and becomes faster.
Testing
- All unit tests are checked and they run correctly now.
Adaption
- Megatron-LM 3.2 supported.
Documentation
- README is updated with some bugs fixed.
- A detailed document for process groups.