Faster large mini-batch distributed training w/o. squeezing devices

This project contains scripts/modules for distributed training

A GitHub repository showcasing the implementation of AI scaling techniques and integration with MLflow for streamlined experiment tracking and management in machine learning workflows.

Everything is born from a simple experiment.

Implementation of a Transformer model from scratch in PyTorch for language translation.

Training Using Multiple GPUs

Example of Distributed pyTorch

Distributed training of a CNN using MNIST dataset, Tensorflow and Horovod

Official DGL Implementation of "Distributed Graph Data Augmentation Technique for Graph Neural Network". KSC 2023

Distributed DP-Helmet: Scalable Differentially Private Non-interactive Averaging of Single Layers

General purpose Kubernetes operator for DL frameworks written in Python

Distributed Machine Learning for Bio-marker Prediction from Big Data Stream collected from Multi-modal Wearable Sensor Data

Messing with Distributed TensorFlow and Kubernetes

In this project, I implement and compare the different distributed training techniques from data parallelization and model parallelization from scratch using PyTorch

A lightweight wrapper that bootstraps PyTorch's Distributed (Data) Parallel.

Tensorflow implementation of U-Net model with TPU Estimator support.

Distributed deep learning framework based on pytorch/numba/nccl and zeromq.

📜 A python library for distributed training of a Transformer neural network across the Internet to solve the Running Key Cipher, widely known in the field of Cryptography.

A hands-on tutorial to dive deep into PyTorch's RPC (Remote Procedure Call) framework. This repository offers a comprehensive guide developed with the assistance of OpenAI's ChatGPT. Whether you're a beginner or an advanced user, this tutorial will provide insights and practices to effectively use PyTorch RPC in your projects.

基于PyTorch的分布式强化学习框架