Frozen Backbone Comparison

Full Fine-Tuning vs Frozen Backbone

The second experiment freezes model_frozen.distilbert.parameters() while leaving the pre_classifier and classifier head trainable. It does not freeze the entire model.

flowchart TD
    A[Same tokenizer and stratified data split] --> B{Training mode}
    B --> C[Full update]
    B --> D[Backbone frozen]
    C --> E[DistilBERT weights trainable]
    E --> F[Classification head trainable]
    D --> G[DistilBERT parameters requires_grad=False]
    G --> H[Pre-classifier + classifier trainable]
    F --> I[4 epochs; learning rate 2e-5]
    H --> J[6 epochs; learning rate 1e-3]
    I --> K[Validation accuracy + weighted F1]
    J --> K
    K --> L[Compare only as tiny-sample demonstration]

Loading

Different learning rates and epoch counts make this an instructional comparison, not a tightly controlled experiment. The script prints accuracy and weighted F1 from both runs; the Wiki intentionally does not invent results.