Warning: This project is still under construction and is not production ready!
This project aims to (i) port optimizations from Microsoft's DeepSpeed library to TensorFlow through various XLA-compiler optimizations like GSPMD, (ii) explore better strategies for mixed-precision quantization for facilitating sparser model architectures, and (iii) provide an extensible API for scalable training and inference.
WarpSpeed is developed in C++ and Python using TensorFlow with API compatibility for JAX + Flax.
The code in this repository is licensed under the Apache License 2.0.