Assignment_7

by Evelyn Bushell

Model

distilbert-base-uncased

Dataset

IMDB

LoRA configuration parameters

My LoRA-R was a small baseline of 8 and my LoRA-Alpha was double that, so 16. The parameters I targeted were those around attention. Because DistilBERT uses odd naming schemes, they were q_lin, k_lin, v_lin, and out_lin.

Evaluation Metrics

Accuracy: 90.86% F1-Score: 0.9103

Explanation

LoRA is a fine-tuning technique that saves time by saves computational resources by freezing the majority of parameters and only adjusting the specific weights needed for the specific training.