Implement some state-of-the-art text classification models with TensorFlow.
- TensorFlow >= 1.4
Note: Original code is written in TensorFlow 1.4, while the
VocabularyProcessor is depreciated, updated code changes to use
tf.keras.preprocessing.text to do preprocessing. The new preprocessing function is named
You can load the data with
dbpedia = tf.contrib.learn.datasets.load_dataset('dbpedia', test_with_fake_data=FLAGS.test_with_fake_data)
Or download it from Baidu Yun.
Attention is All Your Need
Paper: Attention Is All You Need
Use self-attention where Query = Key = Value = sentence after word embedding
Multihead Attention module is implemented by Kyubyong
IndRNN for Text Classification
IndRNNCell is implemented by batzener
Attention-Based Bidirection LSTM for Text Classification
Hierarchical Attention Networks for Text Classification
Attention module is implemented by ilivans/tf-rnn-attention .
Adversarial Training Methods For Supervised Text Classification
Convolutional Neural Networks for Sentence Classification
RMDL: Random Multimodel Deep Learning for Classification
See: RMDL.py See: RMDL Github
Note: The parameters are not fine-tuned, you can modify the kernel as you want.
|Attention-based Bi-LSTM||98.23 %|
|HAN||89.15%||1080Ti 10 epochs 12 min|
|Adversarial Attention-based Bi-LSTM||98.5%||AWS p2 2 hours|
|IndRNN||98.39%||1080Ti 10 epochs 10 min|
|Attention is All Your Need||97.81%||1080Ti 15 epochs 8 min|
|RMDL||98.91%||2X Tesla Xp (3 RDLs)|
Welcome To Contribute
If you have any models implemented with great performance, you're welcome to contribute. Also, I'm glad to help if you have any problems with the project, feel free to raise a issue.