Glossary

Glossary

DistilBERT — A smaller pretrained bidirectional Transformer encoder used here for text classification.

Fine-tuning — Updating pretrained model weights on a new supervised task.

Frozen backbone — Holding the pretrained encoder's parameters constant while updating the classification head.

SFT — Supervised Fine-Tuning on input/target responses.

DPO — Direct Preference Optimization using preferred and dispreferred responses, generally relative to a reference policy.

LoRA — Low-Rank Adaptation, a parameter-efficient method for training adapter matrices.

NF4 — NormalFloat 4-bit representation used in some model-quantization training pipelines.

ORPO — Odds Ratio Preference Optimization; included only as draft sample text in this project.

PPO — Proximal Policy Optimization; not implemented in this repository.

HLD / LLD — High-Level Design and Low-Level Design architectural documents.