Skip to content

Low Level Design

Sudhanshu edited this page Nov 11, 2026 · 1 revision

Low-Level Design (LLD)

This UML-like overview maps Python entry points and functions to the third-party Trainers they instantiate. These are modules/functions, not user-defined classes where the source contains no such class.

classDiagram
    class QwenCLI {
        +build_parser() ArgumentParser
        +validate_jsonl(path) int
        +main(argv) None
    }
    class LegacyCLI {
        +validate_data() None
        +load_dependencies(mode) dict
        +check_hardware(args, torch) tuple
        +make_model(args, deps, cuda) tuple
        +build_sft_dataset(Dataset, eos) Dataset
        +build_preference_dataset(Dataset, eos) Dataset
        +run_sft_dpo(args, deps, tokenizer, model, fp16, bf16) None
        +run_orpo(args, deps, tokenizer, model, fp16, bf16) None
        +main(argv) None
    }
    class Trainer {
        +train()
        +save_model(path)
    }
    class SFTTrainer {
        +train()
        +save_model(path)
    }
    class DPOTrainer {
        +train()
    }
    class ORPOTrainer {
        +train()
    }
    class PEFTModel {
        +load_adapter(path, adapter_name)
        +set_adapter(name)
    }
    QwenCLI ..> Trainer : configures
    QwenCLI ..> PEFTModel : applies LoRA
    LegacyCLI ..> SFTTrainer : mode sft-dpo
    LegacyCLI ..> DPOTrainer : mode sft-dpo
    LegacyCLI ..> ORPOTrainer : mode orpo
    LegacyCLI ..> PEFTModel : adapter handling
Loading

Qwen workflow

  1. build_parser() defines --validate-data, --data, --model, --output, epochs, batch size, sequence length, CPU/LoRA switches.
  2. validate_jsonl() checks one JSON object per line and the nonempty instruction and output strings.
  3. main() loads a Hugging Face dataset, formats Qwen chat messages, tokenizes, applies LoRA to q_proj and v_proj, and creates Trainer.
  4. Trainer.train() performs training, and the entry point saves the LoRA adapter and tokenizer.

Legacy workflow

  1. main() parses validate, check, sft-dpo or orpo.
  2. load_dependencies() checks exact pinned versions before importing the version-sensitive TRL trainers.
  3. check_hardware() selects CPU/FP16/BF16 and checks 4-bit prerequisites.
  4. make_model() loads the pretrained model/tokenizer and applies LoRA.
  5. run_sft_dpo() performs completion-only SFT, then loads a frozen SFT reference adapter and trains DPO.
  6. run_orpo() trains a separate preference experiment using ORPOTrainer.

Attention and adaptation

flowchart TD
    X["Hidden states X"] --> Projection["Attention projection q_proj/v_proj"]
    W["Frozen base weights W"] --> Projection
    A["Trainable low-rank matrix A"] --> Delta["Low-rank update ΔW = B·A"]
    B["Trainable low-rank matrix B"] --> Delta
    Delta --> Projection
    Projection --> Attention["Attention computation within pretrained causal LM"]
    Attention --> Loss["Causal LM loss over unmasked tokens"]
    Loss --> Optim["Update LoRA A/B only"]
Loading

Only LoRA matrices receive trainable updates by default. The base model is pretrained externally; attention and Transformer blocks are not implemented from scratch in this repository.

Clone this wiki locally