Repository navigation
Security and Limitations
Sudhanshu edited this page Nov 11, 2026
·
1 revision
- No API keys, credentials or access tokens should be committed. This repository does not require an OpenAI API key.
- Review local corpora for confidential, licensed or personal information before publishing.
- Model checkpoints and adapter weights may be large and contain learned information from training samples; publish them deliberately, not by default.
- Pretrained Qwen weights are downloaded from Hugging Face on demand. Review the relevant model card and license before redistribution.
- This source ZIP does not include a software license. Add one only after making an intentional licensing decision.
- Full model replica per worker; no sharded optimizer/model state (no FSDP/ZeRO).
- Built-in corpus is small and educational. No validation split or meaningful generation benchmark.
- In Qwen mode loss covers all non-padding tokens; response-only masking is absent.
- Checkpointing happens only at the end, rank 0; optimizer/rank RNG state is not saved for resume.
- Failure of one worker can stop the distributed collective; no elastic recovery policy is implemented.
- Training data resides in process memory; no network data ingestion or distributed streaming storage.
- CUDA/NCCL and Qwen adapter mode were not validated in this package.
See Future-Enhancements.
Distributed LLM Training · PyTorch DDP (educational) — Home · System-Architecture · Low-Level-Design · Publishing-the-Wiki
This project is licensed under the Apache License, Version 2.0.© 2026 Sudhanshu Shukla · Licensed under the Apache License, Version 2.0.