Mesh TensorFlow: Model Parallelism Made Easier
-
Updated
Dec 18, 2018 - Python
Mesh TensorFlow: Model Parallelism Made Easier
distributed tensorflow (model parallelism) example repository
A decentralized and distributed framework for training DNNs
performance test of MNIST hand writings usign MXNet + TF
PyTorch implementation of 3D U-Net with model parallel in 2GPU for large model
A GPipe implementation in PyTorch
Slicing a PyTorch Tensor Into Parallel Shards
A fully distributed hyperparameter optimization tool for PyTorch DNNs
Model parallelism for NN architectures with skip connections (eg. ResNets, UNets)
WIP. Veloce is a low-code Ray-based parallelization library that makes machine learning computation novel, efficient, and heterogeneous.
Adaptive Tensor Parallelism for Foundation Models
Easy Parallel Library (EPL) is a general and efficient deep learning framework for distributed model training.
Official implementation of DynPartition: Automatic Optimal Pipeline Parallelism of Dynamic Neural Networks over Heterogeneous GPU Systems for Inference Tasks
SC23 Deep Learning at Scale Tutorial Material
Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*
Distributed training (multi-node) of a Transformer model
飞桨大模型开发套件,提供大语言模型、跨模态大模型、生物计算大模型等领域的全流程开发工具链。
Fast and easy distributed model training examples.
LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training
Add a description, image, and links to the model-parallelism topic page so that developers can more easily learn about it.
To associate your repository with the model-parallelism topic, visit your repo's landing page and select "manage topics."