Home

Pre-Training and Fine-Tuning — Engineering Wiki

This Wiki documents the actual code in this repository: Qwen2.5-0.5B-Instruct LoRA fine-tuning, a separate pinned legacy SmolLM SFT → DPO workflow, and standalone ORPO. The project name is historical: there is no base-model pre-training loop here, no PPO, and no end-to-end production LLM service. Pre-training will be introduced in the next iteration with our own self-hosted or GPT-style small LLM that fits into consumer GPUs.

Start here

• High-Level Design · Low-Level Design · Mermaid Diagram Index
• Repository Map · Installation · Quick Start
• Qwen LoRA · SFT to DPO · ORPO
• Datasets & Validation · Data Contracts
• Training & Artifacts · Configuration
• Testing · Troubleshooting · Security and Licenses
• Comparison · Publishing to GitHub · Limitations

System context

flowchart LR
    Dev["Developer / Researcher"] --> CLI["PowerShell CLI / Jupyter notebook"]
    CLI --> Q["Qwen LoRA workflow"]
    CLI --> P["Legacy TRL workflow"]
    D["49 Q&A examples · JSONL"] --> Q
    S["6 SFT examples · embedded"] --> P
    R["5 preference pairs · embedded"] --> P
    HF[("Hugging Face model hub")] --> Q
    HF --> P
    Q --> QA["LoRA adapter + tokenizer + training metrics"]
    P --> PA["SFT / DPO / ORPO adapters"]

Loading

All Mermaid diagrams are embedded in Wiki Markdown and also available as editable .mmd files in the main repository's diagrams/ folder.