NCCL Examples from Official NVIDIA NCCL Developer Guide.

Experiments with low level communication patterns that are useful for distributed training.

jupyter/scipy-notebook with CUDA Toolkit, cuDNN, NCCL, and TensorRT

golang wrapper for nccl

nccl built on centos6

Librería de operaciones matemáticas con matrices multi-gpu utilizando Nvidia NCCL.

Tool to run rccl-tests/nccl-tests based on from an application and gather performance.

Nvidia NCCL2 Python bindings using ctypes and numba.

High performance NCCL plugin for Bagua.

use ncclSend ncclRecv realize ncclSendrecv ncclGather ncclScatter ncclAlltoall

Installation script to install Nvidia driver and CUDA automatically in Ubuntu

Blink+: Increase GPU group bandwidth by utilizing across tenant NVLink.

Default Docker image used to run experiments on csquare.run.

An open collection of implementation tips, tricks and resources for training large language models

N-Ways to Multi-GPU Programming

Distributed deep learning framework based on pytorch/numba/nccl and zeromq.

Hands-on Labs in Parallel Computing

Python Distributed Non Negative Matrix Factorization with custom clustering

Sample examples of how to call collective operation functions on multi-GPU environments. A simple example of using broadcast, reduce, allGather, reduceScatter and sendRecv operations.

NCCL Fast Socket is a transport layer plugin to improve NCCL collective communication performance on Google Cloud.