Description of Framework for Efficient Fused-layer Cost Estimation, Legion (2021)
-
Updated
Dec 18, 2022
Description of Framework for Efficient Fused-layer Cost Estimation, Legion (2021)
pipeDejavu: Hardware-aware Latency Predictable, Differentiable Search for Faster Config and Convergence of Distributed ML Pipeline Parallelism
A fully distributed hyperparameter optimization tool for PyTorch DNNs
A simple graph partitioning algorithm written in Go. Designed for use for partitioning neural networks across multiple devices which has an added cost when crossing device boundaries.
WIP. Veloce is a low-code Ray-based parallelization library that makes machine learning computation novel, efficient, and heterogeneous.
Adaptive Tensor Parallelism for Foundation Models
Official implementation of DynPartition: Automatic Optimal Pipeline Parallelism of Dynamic Neural Networks over Heterogeneous GPU Systems for Inference Tasks
Mesh TensorFlow: Model Parallelism Made Easier
Model parallelism for NN architectures with skip connections (eg. ResNets, UNets)
Development of Project HPGO | Hybrid Parallelism Global Orchestration
distributed tensorflow (model parallelism) example repository
A decentralized and distributed framework for training DNNs
performance test of MNIST hand writings usign MXNet + TF
PyTorch implementation of 3D U-Net with model parallel in 2GPU for large model
Torch Automatic Distributed Neural Network (TorchAD-NN) training library. Built on top of TorchMPI, this module automatically parallelizes neural network training.
The project is focused on parallelising pre-processing, measuring and machine learning in the cloud, as well as the evaluation and analysis of the cloud performance.
Serving distributed deep learning models with model parallel swapping.
Fast and easy distributed model training examples.
An MPI-based distributed model parallelism technique for MLP
Add a description, image, and links to the model-parallelism topic page so that developers can more easily learn about it.
To associate your repository with the model-parallelism topic, visit your repo's landing page and select "manage topics."