ReviewPulse v2.0.0 — DistilBERT Extension

ReviewPulse v2.0.0 — DistilBERT Extension

Release date: 2026-04-29

Summary

ReviewPulse v2.0.0 adds a Hugging Face DistilBERT sentiment classifier as a third model option alongside the v1.0.0 baseline and BiLSTM. The pretrained transformer clears both v1 models by a meaningful margin on the held-out test set and is live in the Streamlit app.

This release covers Issues #21–#27 and PR #22 (implemented by Victor Meneses, hardened by Luis Faria).

What Shipped

• Hugging Face distilbert-base-uncased wrapper (src/model_bert.py) with frozen-encoder default and partial unfreezing support.
• Full fine-tuning pipeline (src/train_bert.py): head-first training, partial encoder unfreezing for the last 2 layers, Adam with BCEWithLogitsLoss, ReduceLROnPlateau, and checkpoint serialisation to outputs/distilbert.pt.
• DistilBERT evaluation path in src/evaluate.py with confusion matrix and error analysis outputs.
• predict_distilbert() wired into src/inference.py as model_name="distilbert".
• Streamlit app updated with DistilBERT as a third model option; graceful unavailable state if checkpoint or dependency is missing.
• pytest.importorskip("transformers") guards on BERT-specific test files so baseline/BiLSTM tests run cleanly without the optional dependency.
• Updated presentation outline, individual report template, and docs/issueBreakdown.md to reflect v2.0.0 results.

Model Results

All results on the held-out test split (1,159 reviews, seed=42):

Model	Accuracy	F1	Misclassified
TF-IDF + Logistic Regression	82.7%	81.9%	201
BiLSTM + GloVe	81.0%	80.3%	220
DistilBERT	88.2%	88.6%	137

Validation result:

Model	Best validation F1	Best epoch
DistilBERT	87.8%	12

DistilBERT is available in the app but not the default model — the TF-IDF baseline remains the default because it is faster, has no optional dependency, and its held-out behaviour is well understood. DistilBERT is selectable and degrades gracefully if unavailable.

Application Experience

The Streamlit app now offers three model options:

• TF-IDF + Logistic Regression (default) — fastest, no GPU, best understood
• BiLSTM + GloVe — neural model satisfying the rubric architecture requirement
• DistilBERT — highest accuracy; requires transformers and outputs/distilbert.pt

If DistilBERT fails to load, the app shows a clear warning and keeps baseline/BiLSTM available.

Test Coverage

Scope	Fast tests	Slow integration
v1.0.0 (baseline, BiLSTM, parser, preprocess, dataset, inference, evaluate)	120	5
v2.0.0 BERT (model_bert, train_bert)	23	0
Total	143	5

Fast test command:

pytest tests/ -q -m "not slow"

BERT tests are skipped automatically when transformers is absent.

Known Limitations

• DistilBERT confidence is uncalibrated — 95% confidence does not imply 95% accuracy. Platt scaling is the recommended next step.
• Negation and sarcasm remain partially handled; DistilBERT is better than v1 models but not robust.
• The checkpoint (outputs/distilbert.pt, ~29 MB) is committed to the repository for now. Long-term hosting on the luisfaria webserver is tracked in Issue #28.
• The dataset is from 2007; modern review language, emoji, and slang are not represented in training.

Issue Coverage

• Issue #21: DistilBERT implementation (Victor Meneses, PR #22).
• Issue #23: BERT test guard with pytest.importorskip.
• Issue #24: Graceful DistilBERT loading failure in Streamlit app.
• Issue #25: Verified metrics — confirmed test F1 88.6% from held-out evaluation.
• Issue #26: Training/evaluation implementation cleanup (dead params, device consistency, exception narrowing).
• Issue #27: Presentation outline, individual report template, and issueBreakdown updated with v2.0.0 evidence.

Next Milestone

• Issue #28: host outputs/distilbert.pt on the luisfaria webserver to move the binary out of git long-term.
• Issue #29: v2.0.0 GitHub release tag, changelog, and submission packaging.
• Future: RoBERTa benchmarking (v2.1.0), confidence calibration, LIME explainability.