No description, website, or topics provided.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.

README.md

STAGG-Files

This folder contains several intermediate and final output files of our semantic parsing QA system, detailed in:

Wen-tau Yih, Ming-Wei Chang, Xiaodong He and Jianfeng Gao. Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base In Proceedings of ACL-IJCNLP-2015.

@InProceedings{yih-EtAl:2015:ACL-IJCNLP,
  author    = {Yih, Wen-tau  and  Chang, Ming-Wei  and  He, Xiaodong  and  Gao, Jianfeng},
  title     = {Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base},
  booktitle = {Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
  month     = {July},
  year      = {2015},
  address   = {Beijing, China},
  publisher = {Association for Computational Linguistics},
  pages     = {1321--1331},
  url       = {http://www.aclweb.org/anthology/P15-1128}
}

The files include:

  1. The input files

    • The original input files from SEMPRE:

        webquestions.examples.train.json
        webquestions.examples.test.json
      
    • My table format of the original data. The columns are: Question ID, Original Question, Normalized Question (removing punctuations, tokenization, etc.)

        webquestions.examples.train.questionmap.tsv
        webquestions.examples.test.questionmap.tsv
      
  2. The entity linking results from the system detailed in [Yang & Chang, ACL-IJCNLP-2015]

    The columns are: Question ID, Entity Mention, Start Position (of the normalized question), Length, Linked Entity ID, Entity Name, Entity Linking Score. Notice that the entity IDs match the version of Freebase hosted by SEMPRE, which are not standard Freebase entity MIDs.

     webquestions.examples.train.e2e.top10.filter.sid.tsv
     webquestions.examples.test.e2e.top10.filter.sid.tsv
    

    The files with the standard Freebase entity IDs are included in case you are interested.

     webquestions.examples.train.e2e.top10.filter.tsv
     webquestions.examples.test.e2e.top10.filter.tsv
    
  3. DSSM Relation Matching Training Files

    These are the files the "main inferential chains" generated from the pairs of questions and answers. The columns are F1 score, pattern (question), predicate sequence (entity name & predicate sequence), Question ID. We used only the pairs with F1 scores at least 0.5 for training the relation matching models. Please see the paper for detail.

     webquestions.examples.train.e2e.top10.filter.patrel.sid.tsv
     webquestions.examples.train.e2e.top10.filter.q_ep.sid.tsv
    
  4. DSSM Relation Matching Models

    The DSSM training and predicting code can be found at Sent2Vec.

    Here we provided the models used in the paper.

     data.newpr  -- the model trained on pairs of pattern and predicate
     data.newqe  -- the model trained on pairs of question and entity-predicate
    

    Please check "pr.sample.txt" and "qep.sample.txt" for the format of the input data. They basically are the same pairs as in the training files. The differences are that we replaced the "^E1$" symbol with "e1", replaced "." with " " in the predicates and made all letters lower-cased.

    The command lines to run the DSSM predictors are provided in "run-patrel.bat" and "run-q_ep.bat".

     pr.sample.txt  -- sample pairs for testing the pattern--predicate model
     qep.sample.txt -- sample pairs for testing the question--entity-predicate model
    
     run-patrel.bat -- sample command line to run the DSSM predictor on pr.sample.txt
     run-q_ep.bat   -- sample command line to run the DSSM predictor on qep.sample.txt
    
  5. The final system output on the testing data

    This file is in the format that can be consumed by the standard evaluation script provided in SEMPRE. It can also be found at the CodaLab WebQuestions Worksheet:

     webquestions-predictions-acl2015-msr-stagg
    

Please contact Scott Wen-tau Yih if you have any question.

-- Scott Yih