RUBER

Implementation of RUBER: An Unsupervised Method for Automatic Evaluation of Open-Domain Dialog Systems

Using pre-trained word2vec embeddings to initialize bidirectional RNN.

Performance:

1. RUBER (10 times avg):
   Unrefer Avg pearson: 0.36928, Unrefer Avg spearman: 0.36722 RUBER Avg pearson: 0.29075, RUBER Avg spearman: 0.28677

2. BERT-RUBER (10 times avg): Unrefer Avg pearson: 0.42702, Unrefer Avg spearman: 0.43126 RUBER Avg pearson: 0.42218, RUBER Avg spearman: 0.42163

It should be noted that this repo is not finished, I will push a tutorial few days later.