System Architecture

High-Level System Design (HLD)

Architectural scope

Two workflows appear in the source but have different maturity:

1. Implemented: DistilBERT supervised four-class classification, validation, inference and frozen-vs-full fine-tuning comparison.
2. Design-stage only: Phi-2 parameter-efficient SFT/DPO draft; cannot execute as supplied. ORPO is presented as a printable example string, not a trained model. No PPO exists.

Actual DistilBERT data and control architecture

flowchart TD
    A[30 hardcoded labeled support messages] --> B[Stratified train/validation split 24/6]
    B --> C[Hugging Face DatasetDict]
    C --> D[AutoTokenizer: distilbert-base-uncased]
    D --> E[DataCollatorWithPadding]
    E --> F{Experiment}
    F --> G[Full fine-tuning: DistilBERT + head trainable]
    F --> H[Frozen DistilBERT + trainable pre_classifier/classifier]
    G --> I[Trainer: train/evaluate]
    H --> J[Trainer: train/evaluate]
    I --> K[Accuracy, weighted F1, report, confusion matrix]
    J --> L[Accuracy and weighted F1 comparison]
    I --> M[Saved best full model and tokenizer]
    M --> N[Inference on sample queries]

Loading

System context

flowchart LR
    Dev[Developer / Researcher] --> CLI[Python CLI]
    Dev --> NB[Jupyter / Colab Notebook]
    CLI --> Pipeline[DistilBERT Training Workflow]
    NB --> Pipeline
    HF[(Hugging Face Model Hub)] -->|download tokenizer and pretrained weights| Pipeline
    Pipeline --> Models[(Local Model Checkpoints)]
    Pipeline --> Metrics[(Validation Reports / Logs)]
    Dev -.-> Draft[Phi-2 SFT / DPO Draft: not runnable]
    Draft -.-> HF

Loading

Conceptual planned alignment architecture (not implemented end to end)

flowchart TD
    A[6 chat-style SFT examples] -.-> B[Format prompts for Phi-2 tokenizer]
    C[Phi-2 model weights] -.-> D[4-bit NF4 + LoRA adapter]
    B -.-> E[SFTTrainer]
    D -.-> E
    E -.-> F[Saved SFT adapter]
    F -.-> G[Trainable policy and reference configuration]
    H[5 prompt/chosen/rejected pairs] -.-> I[DPOTrainer with beta]
    G -.-> I
    I -.-> J[Proposed DPO adapter and qualitative evaluation]
    H -.-> K[ORPO code-string concept only]
    K -.-> L[Not executed]
    subgraph Warning[Design-stage ONLY — current DPO_SFT.py fails syntax validation]
      E
      I
    end

Loading

Boundaries

• Data is embedded directly in Python source; there is no external dataset service or DB.
• Model and tokenizer are downloaded using the Hugging Face APIs.
• Hugging Face Trainer handles optimization; the repository does not implement attention kernels itself.
• Local model outputs are not uploaded or served automatically.
• No authentication, REST API, queue, microservice, or external inference gateway is present.

See Low-Level-Design, Training-Configuration, and Known-Issues.