Skip to content

Full vs Frozen

Sudhanshu edited this page Nov 11, 2026 · 1 revision

Full versus frozen-backbone comparison

This comparison studies the number of trainable weights and the impact of allowing the encoder to update. It is not a controlled proof of model superiority.

flowchart LR
    M["Pretrained DistilBERT checkpoint"] --> A["Full fine-tune model"]
    M --> B["Frozen-backbone model"]
    A --> A1["distilbert: trainable"]
    A --> A2["pre_classifier + classifier: trainable"]
    B --> B1["distilbert: frozen"]
    B --> B2["pre_classifier + classifier: trainable"]
    A1 --> C["4 epochs, LR 2e-5"]
    A2 --> C
    B1 --> D["6 epochs, LR 1e-3"]
    B2 --> D
    C --> E["Accuracy / weighted F1 comparison"]
    D --> E
Loading
Aspect Full Frozen
Encoder weights update Yes No
Classification head update Yes Yes
Learning rate 2e-5 1e-3
Epochs 4 6
Best model explicitly saved Yes No
Metrics Accuracy, weighted F1 Accuracy, weighted F1

The program logs the numbers that result when run, but this Wiki does not invent accuracy, loss, latency or throughput values. Repeated runs across independent seeds and a much larger held-out test set would be needed for reliable comparison.

Clone this wiki locally