Skip to content
baseline system of knowledge driven dialogue competition
Branch: master
Clone or download
Latest commit 2007cdc Mar 15, 2019
Type Name Latest commit message Commit time
Failed to load latest commit information.
generative_pt fix Mar 15, 2019
retrieval_paddle Update Mar 15, 2019 Update Mar 15, 2019
task_description.pdf update task description file Mar 10, 2019


License: MIT

about the competition

Human-machine conversation is one of the most important topics in artificial intelligence (AI) and has received much attention across academia and industry in recent years. Currently dialogue system is still in its infancy, which usually converses passively and utters their words more as a matter of response rather than on their own initiatives, which is different from human-human conversation. Therefore, we set up this competition on a new conversation task, named knowledge driven dialogue, where machines converse with humans based on a built knowledge graph. It aims at testing machines’ ability to conduct human-like conversations.
Please refer to competition website for details of the competition.

about the task

Given a dialogue goal g and a set of topic-related background knowledge M = f1 ,f2 ,..., fn , a participating system is expected to output an utterance "ut" for the current conversation H = u1, u2, ..., ut-1, which keeps the conversation coherent and informative under the guidance of the given goal. During the dialogue, a participating system is required to proactively lead the conversation from one topic to another. The dialog goal g is given like this: "Start->Topic_A->TOPIC_B", which means the machine should lead the conversation from any start state to topic A and then to topic B. The given background knowledge includes knowledge related to topic A and topic B, and the relations between these two topics.
Please refer to task description for details of the task.

about the baseline

We provide retrieval-based and generation-based baseline systems.The retrieval-based system was implemented by PaddlePaddle (the Baidu deeplearning framework). The generation-based system was implemented by Pytorch (the Facebook deeplearning framework). The performance of the two systems is as follows:

baseline system F1/BLEU1/BLEU2 DISTINCT1/DISTINCT2
retrieval-based 31.72/0.291/0.156 0.118/0.373
generation-based 32.65/0.300/0.168 0.062/0.128
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.