Skip to content
master
Switch branches/tags
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Attention Encoder-Decoder Dialog Model with GRU

Model Architecture Image

Goal

  • generic and easy method using knowledge graph
  • improve information, diversity, appropriateness
  • reform decoding method and objective function

Example

Type Sentence
Post so instead of adding a simple key combination, they make you use their backwards mouse .
Reference no there is a dedicated button for it on the keyboard , or you can just keep your hand on your input device .
Beam Search you 're right . i 'm not a smart guy , but i do n't think it 's necessary to be a mouse
+ Proposed i do n't know if you can use a mouse pad , but it 's not easy to control the mouse .
Method BLEU-1 BLEU-2 DIST-1 DIST-2
Beam Search 14.545 2.827 3.328 12.872
+ Proposed 14.285 2.771 3.934 17.619

Dataset

  • use dataset for CCM (Commonsense Knowledge Aware Conversation Generation with Graph Attention)
  • Commonsense Conversation Dataset (English)
  • embedding vector is GloVe
  • dialog data from Reddit (not Twitter)
  • commonsense is ConceptNet

Environment

  • Windows10 (coding and run), Ubuntu18.04 (run)
  • python 3.6.8
  • pytorch 1.1.0
  • CUDA 10.1

Run

  1. download CCM dataset
  2. create folders (./data and ./log and ./model)
  3. dataset into ./data folder
  4. edit param.py
  5. $ python train.py
  6. (stop learning with 'Ctrl+C')
  7. $ python test.py
  8. $ pip install rouge (for ROUGE eval)
  9. $ nltk.download('wordnet') (for METEOR eval)
  10. $ python eval.py

Implement

model

  • prepare data
  • create dictionary
  • GRU encoder
  • GRU decoder
  • attention

decoding method

  • greedy search
  • sampling
  • top-k sampling
  • top-p sampling
  • beam search
  • diverse beam search
  • maximum mutual information
  • + with knowledge graph

objective function

  • softmax cross-entropy loss
  • inverse token frequency loss
  • inverse N-gram frequency loss
  • + with knowledge graph

reranking method

  • beam score
  • entity score
  • breakdown possibility
  • auto evaluation

automatic evaluation

  • Length
  • BLEU
  • NIST
  • ROUGE
  • DIST
  • Repeat
  • METEOR
  • Entity score

others

  • length normalization
  • repetitive suppression
  • IDF

Reference (random order)

Site

Paper

About

Dialog Encoder-Decoder Model by Pytorch. With Attention and Bi-GRU.

Resources

Releases

No releases published

Packages

No packages published

Languages