Popular repositories Loading

1. Finetune_llama2 Finetune_llama2 Public

   Build a llama fine-tuning script from scratch using PyTorch and transformers API. It needs to support 4 optional features: gradient checkpointing, mixed precision, data parallelism, tensor parallel…

   Python 2

2. ColossalAI ColossalAI Public

   Forked from hpcaitech/ColossalAI

   Making large AI models cheaper, faster and more accessible

   Python 1

3. BandWidth_Test BandWidth_Test Public

   Test the GPU bandwidth of collectives operators like all-reduce, all-gather, broadcast and all-to-all primitives on single-node multi-GPU (2, 4, 8 cards) and multi-node multi-GPU (16 cards) setups,…

   Python 1

4. Finetune_llama2_Megatron Finetune_llama2_Megatron Public

   Python 1