NLP 对话领域的常用自动评价指标

BLEU

关于BLEU的讲解:BLEU 关于BLEU的代码:BLEU 相关依赖: nltk 使用方法:

import bleu

from bleu import cal_bleu

# sentence: a str separated by space; e.g. "Hello World."
# references: a list of sentences: e.g. ["Hello Me", "Good World"]
# weights: a list of number which be summed together is 1; [0.1, 0.2, 0.3, 0.4]. This is for BLEU-4
  The weight for unigram, bigram, 3-gram, 4-gram is [0.1, 0.2, 0.3, 0.4]

BLEU-SCORE = cal_bleu(sentence, references, weights)

ROUGE

DISTINCT

distinct is firstly proposed by Jiwei Li et.al in paper for the diversity evaluation of generated responses.