Skip to content
No description, website, or topics provided.
Python Shell
Branch: master
Clone or download
Latest commit 085fcbc Aug 13, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data sparc Aug 13, 2019
data_util first commit Jun 22, 2019
eval_scripts Update evaluation_sqa.py Jul 17, 2019
model Update schema_interaction_model.py Jul 17, 2019
README.md Update README.md Jul 28, 2019
logger.py first commit Jun 22, 2019
model_util.py first commit Jun 22, 2019
parse_args.py first commit Jun 22, 2019
postprocess_eval_sparc.py first commit Jun 22, 2019
preprocess_sparc.py first commit Jun 22, 2019
requirements.txt first commit Jun 22, 2019
run.py
run_atis.sh first commit Jun 22, 2019
run_sparc_cdseq2seq.sh first commit Jun 22, 2019
run_sparc_cdseq2seq_segment_copy.sh first commit Jun 22, 2019

README.md

CD-Seq2Seq for SParC

This is a pytorch implementation of the CD-Seq2Seq baseline in our ACL 2019 paper "SParC: Cross-Domain Semantic Parsing in Context".

Please cite this paper if you use our data/code.

@InProceedings{Yu2019,
  author =      "Tao Yu, Rui Zhang, Michihiro Yasunaga, Yi Chern Tan, Xi Victoria Lin, Suyi Li, Heyang Er, Irene Li, Bo Pang, Tao Chen, Emily Ji, Shreya Dixit, David Proctor, Sungrok Shim, Jonathan Kraft, Vincent Zhang, Caiming Xiong, Richard Socher, Dragomir Radev",
  title =       "SParC: Cross-Domain Semantic Parsing in Context",
  booktitle =   "Proceedings of The 57th Annual Meeting of the Association for Computational Linguistics",
  year =        "2019",
  address =     "Florence, Italy"
}

Contact Rui Zhang for any question.

Dependency

The model is tested in python 3.6 and pytorch 1.0. We recommend using conda and pip:

conda create -n cdseq2seq python=3.6
source activate cdseq2seq
pip install -r requirements.txt

The evaluation scripts use python 2.7

Run SParC experiment

First, download SParC from here. Then please follow

  • run_sparc_cdseq2seq.sh. We saved our experimental logs at logs/logs_sparc_cdseq2seq
  • use segment copy: run_sparc_cdseq2seq_segment_copy.sh. We saved our experimental logs at logs/logs_sparc_cdseq2seq_segment_copy

This reproduces the SParC result in "SParC: Cross-Domain Semantic Parsing in Context"

Question Match Interaction Match
Dev Test Dev Test
CD-Seq2Seq 21.9 - 8.1 -
CD-Seq2Seq+segment copy (use predicted query) 21.7 - 9.5 -
CD-Seq2Seq+segment copy (use gold query) 27.3 - 10.0 -

Run ATIS experiment

To get ATIS data and get evaluation on the result accuracy, you need get ATIS data from here, set up your mysql database for ATIS and change --database_username and --database_password in parse_args.py.

Please follow run_atis.sh

This reproduces the ATIS result in "Learning to map context dependent sentences to executable formal queries". We saved our experimental logs at logs/logs_atis

Dev Test
Query Relaxed Strict Query Relaxed Strict
Suhr et al., 2018 37.5(0.9) 63.0(0.7) 62.5(0.9) 43.6(1.0) 69.3(0.8) 69.2(0.8)
Our Replication 38.8 63.3 62.8 44.6 68.3 68.2

Acknowledgement

This implementation is based on "Learning to map context dependent sentences to executable formal queries". Alane Suhr, Srinivasan Iyer, and Yoav Artzi. In NAACL, 2018.

You can’t perform that action at this time.