Repository navigation
Full vs Frozen
Sudhanshu edited this page Nov 11, 2026
·
1 revision
This comparison studies the number of trainable weights and the impact of allowing the encoder to update. It is not a controlled proof of model superiority.
flowchart LR
M["Pretrained DistilBERT checkpoint"] --> A["Full fine-tune model"]
M --> B["Frozen-backbone model"]
A --> A1["distilbert: trainable"]
A --> A2["pre_classifier + classifier: trainable"]
B --> B1["distilbert: frozen"]
B --> B2["pre_classifier + classifier: trainable"]
A1 --> C["4 epochs, LR 2e-5"]
A2 --> C
B1 --> D["6 epochs, LR 1e-3"]
B2 --> D
C --> E["Accuracy / weighted F1 comparison"]
D --> E
| Aspect | Full | Frozen |
|---|---|---|
| Encoder weights update | Yes | No |
| Classification head update | Yes | Yes |
| Learning rate | 2e-5 | 1e-3 |
| Epochs | 4 | 6 |
| Best model explicitly saved | Yes | No |
| Metrics | Accuracy, weighted F1 | Accuracy, weighted F1 |
The program logs the numbers that result when run, but this Wiki does not invent accuracy, loss, latency or throughput values. Repeated runs across independent seeds and a much larger held-out test set would be needed for reliable comparison.
Getting started
Architecture
- System Context
- High-Level Design
- Low-Level Design
- Model Components
- Training Sequence
- Data & Labels
- Tokenization
Training & results
Engineering reference