Skip to content
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
src
 
 
 
 
 
 
 
 
 
 
 
 

TransDG

The implementation of TransDG proposed in Improving Knowledge-aware Dialogue Generation via Knowledge Base Question Answering (AAAI-2020).

In this paper, we propose a novel knowledge-aware dialogue generation model (called TransDG), which transfers question representation and knowledge matching abilities from knowledge base question answering (KBQA) task to facilitate the utterance understanding and factual knowledge selection for dialogue generation.

Requirements

  • Python3
  • Tensorflow >= 1.8
  • Stanford CoreNLP
  • NLTK
  • PyLucene

Datasets

The SimpleQuestions (v2) dataset and FB2M can be downloaded from here, the Reddit dialogue dataset associated with ConceptNet can be downloaded from here. Also, please download the Freebase metadata from here (extraction code: iwq1) and unpack to the folder data/freebase-metadata/, download the Glove embeddings from here (extraction code: vxc3) and unpack to the folder data/kbqa_emb/.

Quickstart

Step 1: Data Processing

Data Processing for KBQA Pre-training:

(1) Entity linking. Please set data_name=SimpQ and data_dir=data/SimpleQuestions_v2 in the gen_linkings.sh, then run:

sh gen_linkings.sh

(2) Candidates building. Please set data_name=SimpQ in the gen_candidates.sh, then run:

sh gen_candidates.sh

Data Processing for Dialogue Generation:

(1) Top-k similar responses retrieving. Please set mode=train/valid/test in the gen_retrieving.sh, then run:

sh gen_retrieving.sh

(2) Entity linking. Please set data_name=Reddit, data_dir=data/Reddit, and mode=train/valid/test in the gen_linkings.sh, then run:

sh gen_linkings.sh

(3) Candidates building. Please set data_name=Reddit and mode=train/valid/test in the gen_candidates.sh, then run:

sh gen_candidates.sh

(4) Schema dataset building. Please set mode=train/valid/test in the gen_schema_dataset.sh, then run:

sh gen_schema_dataset.sh

(5) Final dataset building. Please set mode=train/valid/test in the gen_final_dataset.sh, then run:

sh gen_final_dataset.sh

Step 2: KBQA Pre-training

Please refer to the script train_kbqa.sh and set parameters accordingly. Then run:

sh train_kbqa.sh

Step 3: Dialogue Generation

For model training, please refer to the script run_train.sh and set parameters accordingly. Then run:

sh run_train.sh

For model testing, please refer to the script run_test.sh and set parameters accordingly. Then run:

sh run_test.sh

Citation

If you use any source code included in this project in your work, please cite as:

@inproceedings{wang2020improving,
  title={Improving Knowledge-Aware Dialogue Generation via Knowledge Base Question Answering.},
  author={Wang, Jian and Liu, Junhao and Bi, Wei and Liu, Xiaojiang and He, Kejing and Xu, Ruifeng and Yang, Min},
  booktitle={AAAI},
  pages={9169--9176},
  year={2020}
}

About

Source code for the paper "Improving Knowledge-aware Dialogue Generation via Knowledge Base Question Answering".

Topics

Resources

License

Releases

No releases published

Packages

No packages published