The leaderboard on the main page is the results on the hidden test set for the ConvAI2 competition, which will serve as the main results dictating which models graduate to the Mechanical Turk human evaluation round on September 30th. On this page will we add additional scores of interest: currently, results on the validation set which are good for calibration, and are the actual numbers you would see if you run the evaluation scripts locally (as you do not have access to the hidden test set for the main leaderboard; those are run by us when you submit your model). We will add the revised persona leaderboard here soon too.
ConvAI2 Validation Set Leaderboard
|Lost in Conversation||-||17.3||17.79|
|Khai Mai Alt||-||85.3
|Mohd Shadab Alam||34.12||13.4||17.08|
|Clova Xiaodong Gu||-||-||15.39|
|Seq2Seq + Attention||ParlAI team||35.07||12.5||16.82|
|Language Model||ParlAI team||51.1||-||15.31|
|KV Profile Memory||ParlAI team||-||55.1||11.72|
Models by ParlAI team are baselines, and not entries into the competition; code is included for those models.