High Level Design

High-Level Design (HLD)

The repository has two independent training environments. Their libraries and API expectations are incompatible. Models are downloaded from Hugging Face when training is requested; the ZIP does not bundle model weights.

flowchart TB
    subgraph Setup["Project execution"]
      CLI["CLI or Jupyter"] --> Branch{"Selected workflow"}
    end
    subgraph Modern["Qwen modern environment · Transformers 5.x"]
      File["data/alien_romulus_qa.jsonl"] --> Validate["validate_jsonl()"]
      Validate --> HFData["datasets.load_dataset()"]
      HFData --> Chat["Qwen chat template"]
      Chat --> Tokens["Tokenizer + causal LM collator"]
      Qwen["Qwen2.5-0.5B-Instruct pretrained weights"] --> LoRA["PEFT LoRA on q_proj, v_proj"]
      Tokens --> Trainer["Transformers Trainer"]
      LoRA --> Trainer
      Trainer --> Adapter["Saved Qwen LoRA adapter"]
    end
    subgraph Legacy["Legacy environment · TRL 0.8.6"]
      Pairs["6 SFT + 5 preference examples"] --> LegacyTrain["Legacy training pipeline"]
      Smol["SmolLM-135M base model"] --> LegacyTrain
      LegacyTrain --> Decision{"Training mode"}
      Decision -->|sft-dpo| SFT["Completion-only SFT"]
      SFT --> DPO["DPO vs frozen SFT adapter"]
      Decision -->|orpo| ORPO["Standalone ORPO"]
      DPO --> SaveD["SFT & DPO adapters"]
      ORPO --> SaveO["ORPO adapter"]
    end
    Branch -->|Qwen LoRA| Validate
    Branch -->|Legacy TRL| LegacyTrain

Loading

Runtime boundaries

Boundary	Qwen LoRA	Legacy SFT/DPO/ORPO
Entry point	qwen_alien_romulus_finetune.py or Qwen notebook	LLM_Training_Fixed_v2.py or legacy notebook
Dependencies	Transformers 5.x, PEFT	Transformers 4.40, TRL 0.8.6, PEFT 0.10
Pretrained checkpoint	Qwen2.5-0.5B-Instruct	SmolLM-135M (Phi-2 optional)
Dataset	JSONL, 49 Q&A records	6 embedded SFT, 5 preference pairs
Output	LoRA adapter/tokenizer/metrics	SFT+DPO or separate ORPO adapters

There is no web API, message broker, database, RAG retrieval, or distributed training system in this implementation. See Low-Level Design for the function boundaries.