Jupyter and Colab

Jupyter, VS Code, Colab and Kaggle

The supplied LLM_DDP_Parallel_Training.ipynb is a self-contained notebook. It writes llm_ddp_parallel_training.py and a requirements file to its working directory, checks hardware availability, and launches training through subprocess calls.

Recommended notebook workflow

1. Open the .ipynb in JupyterLab, VS Code, Colab or Kaggle.
2. Run setup / inspection cells. Install missing dependencies in a separate notebook cell using %pip install -r requirements.txt if needed.
3. Execute the %%writefile cell to create the Python CLI script.
4. Run the single CPU or two CPU processes through subprocess.run in the provided cells.
5. Enable Qwen training only after confirming available CUDA devices and package compatibility.

The separate-process approach avoids common notebook issues with torchrun and IPython argparse injection.

Important: The notebook writes files to the current working directory. If you run it in a clone of this repository, ensure you do not unintentionally overwrite work-in-progress source changes. The notebook's two-CPU demonstration works without a second GPU; standard Colab sessions may only expose one GPU.