DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
-
Updated
Jun 19, 2024 - Python
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Making large AI models cheaper, faster and more accessible
Fast and easy distributed model training examples.
飞桨大模型开发套件,提供大语言模型、跨模态大模型、生物计算大模型等领域的全流程开发工具链。
LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training
Single-node data parallelism in Julia with CUDA
The project utilizes OpenMP to implement parallelism in a large dataset by leveraging multicore processor architectures to concurrently execute code sections, optimizing performance and scalability for efficient database processing
Distributed training (multi-node) of a Transformer model
The Levenshtein edit-distance algorithm, in Javascript, parallelised across workers [WIP]
Towards Rehearsal-based Continual Learning at Scale: distributed CL with Horovod + PyTorch
SIMD multithreaded Monte Carlo options pricer in Rust 🦀
A mostly POSIX-compliant utility that scans a given interval for vampire numbers.
Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*
A state-of-the-art multithreading runtime: message-passing based, fast, scalable, ultra-low overhead
SC23 Deep Learning at Scale Tutorial Material
Scaling Unet in Pytorch
Scaling Unet in Tensorflow
MapReduceSimulator for Scheduling and Provisioning Algorithms
This repository provides hands-on labs on PyTorch-based Distributed Training and SageMaker Distributed Training. It is written to make it easy for beginners to get started, and guides you through step-by-step modifications to the code based on the most basic BERT use cases.
Add a description, image, and links to the data-parallelism topic page so that developers can more easily learn about it.
To associate your repository with the data-parallelism topic, visit your repo's landing page and select "manage topics."