Skip to content
Multi-hop knowledge graph reasoning learned via policy gradient with reward shaping and action dropout
Branch: master
Clone or download
Type Name Latest commit message Commit time
Failed to load latest commit information.
configs initial commit Sep 28, 2018
src implement flag option for printing beam search paths Apr 11, 2019
.gitignore add .gitignore Sep 28, 2018
Dockerfile initial commit Sep 28, 2018
LICENSE Create LICENSE Sep 28, 2018
Makefile Add command to print inference paths Apr 11, 2019
data-release.tgz initial commit Sep 28, 2018 initial commit Sep 28, 2018 initial commit Sep 28, 2018
requirements.txt initial commit Sep 28, 2018

Multi-Hop Knowledge Graph Reasoning with Reward Shaping

This repository contains the source code release of the paper: Lin et. al. 2018. Multi-Hop Knowledge Graph Reasoning with Reward Shaping.

Quick Start

Environment Variables & Dependencies

Use Docker

Build the docker image

docker build -< Dockerfile -t multi_hop_kg:v1.0

Spin up a docker container and run experiments inside it.

nvidia-docker run -v `pwd`:/workspace/MultiHopKG -it multi_hop_kg:v1.0

The rest of the readme assumes that one works interactively inside a container. If you prefer to run experiments outside a container, please change the commands accordingly.

Mannually Set up

Alternatively, you can install Pytorch (>=0.4.1) manually and use the Makefile to set up the rest of the dependencies.

make setup

Process data

First, unpack the data files

tar xvzf data-release.tgz

and run the following command to preprocess the datasets.

./ configs/<dataset>.sh --process_data <gpu-ID>

<dataset> is the name of any dataset folder in the ./data directory. In our experiments, the five datasets used are: umls, kinship, fb15k-237, wn18rr and nell-995. <gpu-ID> is a non-negative integer number representing the GPU index.

Train models

Then the following commands can be used to train the proposed models and baselines in the paper. By default, dev set evaluation results will be printed when training terminates.

  1. Train embedding-based models
./ configs/<dataset>-<emb_model>.sh --train <gpu-ID>

The following embedding-based models are implemented: distmult, complex and conve.

  1. Train RL models (policy gradient)
./ configs/<dataset>.sh --train <gpu-ID>
  1. Train RL models (policy gradient + reward shaping)
./ configs/<dataset> --train <gpu-ID>
  • Note: To train the RL models using reward shaping, make sure 1) you have pre-trained the embedding-based models and 2) set the file path pointers to the pre-trained embedding-based models correctly (example configuration file).

Evaluate pretrained models

To generate the evaluation results of a pre-trained model, simply change the --train flag in the commands above to --inference.

For example, the following command performs inference with the RL models (policy gradient + reward shaping) and prints the evaluation results (on both dev and test sets).

./ configs/<dataset> --inference <gpu-ID>

To print the inference paths generated by beam search during inference, use the --save_beam_search_paths flag:

./ configs/<dataset> --inference <gpu-ID> --save_beam_search_paths
  • Note for the NELL-995 dataset:

    On this dataset we split the original training data into train.triples and dev.triples, and the final model to test has to be trained with these two files combined.

    1. To obtain the correct test set results, you need to add the --test flag to all data pre-processing, training and inference commands.
    ./ configs/ --process_data <gpu-ID> --test
    ./ configs/ --train <gpu-ID> --test
    ./ configs/ --train <gpu-ID> --test
    ./ configs/ --inference <gpu-ID> --test
    1. Leave out the --test flag during development.

Change the hyperparameters

To change the hyperparameters and other experiment set up, start from the configuration files.

Notes on Implementation Details

We use mini-batch training in our experiments. To save the amount of paddings (which can cause memory issues and slow down computation for knowledge graphs that contain nodes with large fan-outs), we group the action spaces of different nodes into buckets based on their sizes. Description of the bucket implementation can be found here and here.


If you find the resource in this repository helpful, please cite

  author = {Xi Victoria Lin and Richard Socher and Caiming Xiong}, 
  title = {Multi-Hop Knowledge Graph Reasoning with Reward Shaping}, 
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural
               Language Processing, {EMNLP} 2018, Brussels, Belgium, October
               31-November 4, 2018},
  year = {2018} 
You can’t perform that action at this time.