Skip to content
Deep Unknown Intent Detection with Margin Loss (ACL2019)
Jupyter Notebook Python
Branch: master
Clone or download
Latest commit 304402d Aug 11, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data
img
README.md
experiment.ipynb initial commit Aug 11, 2019
experiment.py initial commit Aug 11, 2019
models.py initial commit Aug 11, 2019
requirements.txt
utils.py

README.md

Deep Unknown Intent Detection with Margin Loss

Implementation of the research paper Deep Unknown Intent Detection with Margin Loss (ACL2019)

Identifying the unknown (novel) user intents that have never appeared in the training set is a challenging task in the dialogue system. In this paper, we present a two-stage method for detecting unknown intents. We use bidirectional long short-term memory (BiLSTM) network with the margin loss as the feature extractor.

With margin loss, we can learn discriminative deep features by forcing the network to maximize inter-class variance and to minimize intra-class variance. Then, we feed the feature vectors to the density-based novelty detection algorithm, local outlier factor (LOF), to detect unknown intents.

The architecture of the proposed method:

Architecture

Usage

  1. Install all required library
pip install -r requirements.txt
  1. Get the GloVe embedding and modify the embedding_path in experiment.py/ipynb
wget http://nlp.stanford.edu/data/glove.6B.zip
unzip glove.6B.zip
  1. Run the experiments by:
python experiment.py <dataset> <proportion>
python experiment.py SNIPS 50
python experiment.py ATIS 25

Result

% of known intents 25% 50% 75% 25% 50% 75%
SNIPS ATIS
MSP - 6.2 8.3 8.1 15.3 17.2
DOC 72.5 67.9 63.9 61.6 62.8 37.7
DOC (Softmax) 72.8 65.7 61.8 63.6 63.3 38.7
LOF (Softmax) 76.0 69.4 65.8 67.3 61.8 38.9
LOF (LMCL) 79.2 84.1 78.8 69.6 63.4 39.6

Citation

If you mentioned the method in your research, please cite this article:

@inproceedings{lin-xu-2019-deep,
    title = "Deep Unknown Intent Detection with Margin Loss",
    author = "Lin, Ting-En  and
      Xu, Hua",
    booktitle = "Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics",
    month = jul,
    year = "2019",
    address = "Florence, Italy",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/P19-1548",
    pages = "5491--5496",
}
You can’t perform that action at this time.