DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
-
Updated
Jul 20, 2024 - Python
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Fast and easy distributed model training examples.
Making large AI models cheaper, faster and more accessible
LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training
Towards Rehearsal-based Continual Learning at Scale: distributed CL with Horovod + PyTorch
飞桨大模型开发套件,提供大语言模型、跨模态大模型、生物计算大模型等领域的全流程开发工具链。
Distributed training (multi-node) of a Transformer model
Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*
SC23 Deep Learning at Scale Tutorial Material
Easy Parallel Library (EPL) is a general and efficient deep learning framework for distributed model training.
Batch Partitioning for Multi-PE Inference with TVM (2020)
WIP. Veloce is a low-code Ray-based parallelization library that makes machine learning computation novel, efficient, and heterogeneous.
Distributing Deep Learning Hyperparameter Tuning for 3D Medical Image Segmentation
A fully distributed hyperparameter optimization tool for PyTorch DNNs
Official Repository for the paper: Distributing Deep Learning Hyperparameter Tuning for 3D Medical Image Segmentation
Understanding the effects of data parallelism and sparsity on neural network training
Distributed Keras Engine, Make Keras faster with only one line of code.
A decentralized and distributed framework for training DNNs
Example of Distributed pyTorch
Add a description, image, and links to the data-parallelism topic page so that developers can more easily learn about it.
To associate your repository with the data-parallelism topic, visit your repo's landing page and select "manage topics."