v0.10.0 — Pretrained Embeddings (GloVe)
Pretrained Embeddings. A from-scratch GloVe loader and vocabulary-aligned
embedding-matrix builder, wired into both models. The benchmark honestly shows
GloVe does not break the ceiling on IMDB (+0.001 for the pooling model; the
transformer overfits harder and slips to 0.849) — with 25,000 labeled reviews the
model learns good embeddings from scratch anyway. Pretrained word vectors help
when labeled data is scarce; the lever that actually moves this task is
self-supervised (contextual) pretraining, which is the next milestone.
Added
load_gloveandbuild_embedding_matrix(polaris.embeddings): load pretrained GloVe vectors and align them to aVocabulary(GloVe vector per known word, seeded-random for out-of-vocabulary, zeros for padding).pretrained_embeddings/freeze_embeddingsparameters onMeanPoolingClassifierandTransformerEncoderClassifier, initializing the embedding layer from a matrix.
Changed
- The IMDB example has a
GLOVE_PATHsetting: point it at a GloVe file (with the whitespace tokenizer) to initialize the model's embeddings from pretrained vectors. - README benchmark table gains a GloVe column and the third honest finding (pretrained word embeddings do not break the ~86% ceiling here).