Skip to content
Leveraging Sparse and Dense Feature Combinations for Sentiment Classification
OpenEdge ABL Python
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.


Source code for the paper: Leveraging Sparse and Dense Feature Combinations for Sentiment Classification

Compatibility and Dependencies

Python 2.7 and 3.x sklearn scipy numpy pandas nltk

Running the Model

usage: python --train [path to train in json] --test [path to test in json] --we [path to word2vec] --ngram [e.g. 123]

train/test file should in json with attributes: text: text string, y: labels

Running Example

usage: python --train ../data/mr_train_cv2.json --test ../data/mr_test_cv2.json --ngram 123 --we GoogleNews-vectors-negative300.bin


NBSVM+POS word embedding (NBSVM+POS wemb) model outperforms most recent published sentiment models such CNN, LSTM etc. excepting the model proposed on Self-Adaptive Hierarchical Sentence Model. Comparing with other models, NBSVM+POS wemb is simple and fast-to-train since it’s basically SVM using ngram log-count raitos and POS word embedding features.


Naive Bayes Support Vector Machine (NBSVM) is a simple and good approach introduced by Wang & Manning, 2012. This approach computes a log-ratio vector between the average word counts extracted from positive documents and the average word counts extracted from negative documents. The input to the logistic regression/SVM classifier corresponds to the log-ratio vector multiplied by the binary pattern for each word in the document vector. NBSVM often outperforms regular SVM using uni/bi-gram counts directly.

Improvements of NBSVM+POS wemb

The performance of NBSVM (originally only use log-ratio vectors for its features) increases about 0.5-1% by naively incorporating averaged word embeddings into log-ratio feature vectors (NBSVM+avg wemb). To push the scores higher, by concatenating averaged word embeddings for different POS tags (concatenate several averaged word embedding vectors for nouns, verbs, and adjectives so on) instead of the whole sentence to the log-ratio feature vectors (NBSVM + POS wemb), NBSVM + POS wemb outperforms NBSVM by 2-3%, and becomes a state-of-the-art model on most of sentiment benchmarks.

You can’t perform that action at this time.