Repository navigation
Installation
- Python 3.10+; Python 3.12 is a good option on Windows.
- PyTorch 2.6 or newer compatible with your OS and accelerator.
-
transformers,peft,accelerateandsafetensorsfor Qwen LoRA mode. - Network access for downloading pretrained Qwen weights (Qwen mode only).
py -3.12 -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -r requirements.txt
python -m pip checkThe uploaded manifest is named llm_ddp_requirements_.txt. The publication package includes an identical requirements.txt alias because the original README and notebook refer to that conventional filename. No original dependency specification was modified.
For a GPU, install an official CUDA-enabled PyTorch package appropriate to the driver before installing optional ML libraries. torch>=2.6 alone does not guarantee a CUDA build.
python -c "import torch; print('Torch:',torch.__version__,'CUDA:',torch.cuda.is_available(),'GPUs:',torch.cuda.device_count())"For a CPU-only first test, see Quick-Start. On Windows, PowerShell activation might be blocked by an execution policy; running .venv\Scripts\python.exe directly is an alternative to modifying policy settings.
Distributed LLM Training · PyTorch DDP (educational) — Home · System-Architecture · Low-Level-Design · Publishing-the-Wiki
This project is licensed under the Apache License, Version 2.0.© 2026 Sudhanshu Shukla · Licensed under the Apache License, Version 2.0.