G-SAT: Scalable Neural Dialogue State Tracking

This is a PyTorch implementation of the paper: Scalable Neural Dialogue State Tracking. Vevake Balaraman and Bernardo Magnini. ASRU 2019. PDF

Abstract

A Dialogue State Tracker (DST) is a key component in a dialogue system aiming at estimating the beliefs of possible user goals at each dialogue turn. Most of the current DST trackers make use of recurrent neural networks and are based on complex architectures that manage several aspects of a dialogue, including the user utterance, the system actions, and the slot-value pairs defined in a domain ontology. However, the complexity of such neural architectures incurs into a considerable latency in the dialogue state prediction, which limits the deployments of the models in real-world applications, particularly when task scalability (i.e. amount of slots) is a crucial factor. In this paper, we propose an innovative neural model for dialogue state tracking, named Global encoder and Slot-Attentive decoders (G-SAT), which can predict the dialogue state with a very low latency time, while maintaining high-level performance. We report experiments on three different languages (English, Italian, and German) of the WOZ2.0 dataset, and show that the proposed approach provides competitive advantages over state-of-art DST systems, both in terms of accuracy and in terms of time complexity for predictions, being over 15 times faster than the other systems.

Citation

The bibtex is below.

@article{balaraman2019scalable,
    title={Scalable Neural Dialogue State Tracking},
    author={Vevake Balaraman and Bernardo Magnini},
    year={2019},
    eprint={1910.09942},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

Dataset

The WoZ2.0 dataset is used for the experiments. Thanks to Nikola Mrksic for making it publicly available.

Requirements

The required packages to run the program are available in requirements.txt and can be installed using following command.

pip install requirements.txt

Executing the program

1. Download and preprocess the data. The following script downloads datset for all 3 languages (en, it and de).

   python preprocess_data.py
   

2. Train the model

   python train.py --lang <en/it/de> --seed <SEED NUMBER> --data_dir <DIR. TO SAVE PREPROCESSED DATA> --save_dir <DIR. TO SAVE TRAINED MODEL> --emb_dim <EMBEDDING DIM> 
   

   All parameters are optional. To run in default config, run python train.py. Default random seed is 123, to run with different random seeds, use --seed parameter. More options for the model can be found in config.py file.

3. Test the performance on testset

   python test.py 
   

   If default parameters were modified while training, input the same while testing as well.

Result

The results of the proposed approach are averaged over 10 different random initializations. Table 1 shows results for pre-trained embeddings and Table 2 & 3 are results without pre-trained embeddings.

Time complexity of different models

Time complexity of various models for each batch of size 50 during training and testing (low execution time means low latency in prediction).

Contact

Please feel free to contact me at balaraman@fbk.eu for any queries.

Acknowledgement

We thank all researchers who made the source code for their works on dialogue state tracker publicly available. In particular we thank Victor Zhong for making the GLAD model public (Source code for processing and loading the dataset are based on this).