Repository navigation
Low Level Design
Sudhanshu edited this page Nov 11, 2026
·
1 revision
This UML-like overview maps Python entry points and functions to the third-party Trainers they instantiate. These are modules/functions, not user-defined classes where the source contains no such class.
classDiagram
class QwenCLI {
+build_parser() ArgumentParser
+validate_jsonl(path) int
+main(argv) None
}
class LegacyCLI {
+validate_data() None
+load_dependencies(mode) dict
+check_hardware(args, torch) tuple
+make_model(args, deps, cuda) tuple
+build_sft_dataset(Dataset, eos) Dataset
+build_preference_dataset(Dataset, eos) Dataset
+run_sft_dpo(args, deps, tokenizer, model, fp16, bf16) None
+run_orpo(args, deps, tokenizer, model, fp16, bf16) None
+main(argv) None
}
class Trainer {
+train()
+save_model(path)
}
class SFTTrainer {
+train()
+save_model(path)
}
class DPOTrainer {
+train()
}
class ORPOTrainer {
+train()
}
class PEFTModel {
+load_adapter(path, adapter_name)
+set_adapter(name)
}
QwenCLI ..> Trainer : configures
QwenCLI ..> PEFTModel : applies LoRA
LegacyCLI ..> SFTTrainer : mode sft-dpo
LegacyCLI ..> DPOTrainer : mode sft-dpo
LegacyCLI ..> ORPOTrainer : mode orpo
LegacyCLI ..> PEFTModel : adapter handling
-
build_parser()defines--validate-data,--data,--model,--output, epochs, batch size, sequence length, CPU/LoRA switches. -
validate_jsonl()checks one JSON object per line and the nonemptyinstructionandoutputstrings. -
main()loads a Hugging Face dataset, formats Qwen chat messages, tokenizes, applies LoRA toq_projandv_proj, and createsTrainer. -
Trainer.train()performs training, and the entry point saves the LoRA adapter and tokenizer.
-
main()parsesvalidate,check,sft-dpoororpo. -
load_dependencies()checks exact pinned versions before importing the version-sensitive TRL trainers. -
check_hardware()selects CPU/FP16/BF16 and checks 4-bit prerequisites. -
make_model()loads the pretrained model/tokenizer and applies LoRA. -
run_sft_dpo()performs completion-only SFT, then loads a frozen SFT reference adapter and trains DPO. -
run_orpo()trains a separate preference experiment using ORPOTrainer.
flowchart TD
X["Hidden states X"] --> Projection["Attention projection q_proj/v_proj"]
W["Frozen base weights W"] --> Projection
A["Trainable low-rank matrix A"] --> Delta["Low-rank update ΔW = B·A"]
B["Trainable low-rank matrix B"] --> Delta
Delta --> Projection
Projection --> Attention["Attention computation within pretrained causal LM"]
Attention --> Loss["Causal LM loss over unmasked tokens"]
Loss --> Optim["Update LoRA A/B only"]
Only LoRA matrices receive trainable updates by default. The base model is pretrained externally; attention and Transformer blocks are not implemented from scratch in this repository.
Pre-Training and Fine-Tuning · Real prodcution grade PyTorch/Hugging Face training examples · Wiki Home · Architecture. This project is licensed under the Apache License, Version 2.0.© 2026 Sudhanshu Shukla · Licensed under the Apache License, Version 2.0