BiMPM: Bilateral Multi-Perspective Matching for Natural Language Sentences
Switch branches/tags
Nothing to show
Clone or download
zhigwang
zhigwang tf 1.4 to tf 1.5
Latest commit 33cc8fe Feb 5, 2018
Permalink
Failed to load latest commit information.
.settings Init Mar 2, 2017
configs updated to tensorflow 1.4 Jan 29, 2018
src updated to tensorflow 1.4 Jan 29, 2018
.project Init Mar 2, 2017
.pydevproject Init Mar 2, 2017
LICENSE.txt Create LICENSE.txt May 24, 2017
README.md tf 1.4 to tf 1.5 Feb 5, 2018

README.md

BiMPM: Bilateral Multi-Perspective Matching for Natural Language Sentences

Updates (Jan 28, 2018)

  • This repository has been updated to tensorflow 1.5
  • The training process speeds up 15+ times without lossing the accuracy.
  • All codes have been re-constructed for better readability and adaptability.

Description

This repository includes the source code for natural language sentence matching. Basically, the program takes two sentences as input, and predict a label for the two input sentences. You can use this program to deal with tasks like paraphrase identification, natural language inference, duplicate questions identification et al. More details about the underneath model can be found in our paper published in IJCAI 2017. Please cite our paper when you use this program! 😍

Requirements

  • python 2.7
  • tensorflow 1.5

Data format

Both the train and test sets require a tab-separated format. Each line in the train (or test) file corresponds to an instance, and it should be arranged as

label sentence#1 sentence#2 instanceID

For more details about the data format, you can download the SNLI and the Quora Question Pair datasets used in our paper.

Training

You can find the training script at BiMPM/src/SentenceMatchTrainer.py

First, edit the configuration file at ${workspace}/BiMPM/configs/snli.sample.config (or ${workspace}/BiMPM/configs/quora.sample.config ). You need to change the "train_path", "dev_path", "word_vec_path", "model_dir", "suffix" to your own setting.

Second, launch job using the following command line

python ${workspace}/BiMPM/SentenceMatchTrainer.py --config_path ${workspace}/BiMPM/configs/snli.sample.config

Testing

You can find the testing script at BiMPM/src/SentenceMatchDecoder.py

python ${workspace}/BiMPM/src/SentenceMatchDecoder.py --in_path ${your_path_to}/dev.tsv --word_vec_path ${your_path_to}/wordvec.txt --out_path ${your_path_to}/result.json --model_prefix ${model_dir}/SentenceMatch.${suffix}

Where "model_dir" and "suffix" are the variables set in your configuration file.

The output file is a json file with the follwing format.

{
    { 
        "ID": "instanceID",
        "truth": label,
        "sent1": sentence1,
        "sent2": sentence2,
        "prediction": prediciton,
        "probs": probs_for_all_possible_labels
    },
    { 
        "ID": "instanceID",
        "truth": label,
        "sent1": sentence1,
        "sent2": sentence2,
        "prediction": prediciton,
        "probs": probs_for_all_possible_labels
    }
}

Reporting issues

Please let me know, if you encounter any problems.