Skip to content
No description, website, or topics provided.
Python C++ C Shell Objective-C
Branch: master
Clone or download

Latest commit

Latest commit ec87851 Mar 27, 2020

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
AMIE AMIE Mar 24, 2020
Cartesian-product Cartesian product relations Mar 24, 2020
ComplEX ComplEx Mar 24, 2020
ConvE ConvE Mar 24, 2020
Models-detailed-results update detailed_results Mar 27, 2020
OpenKE OpenKE Mar 24, 2020
RotatE RotatE Mar 24, 2020
TuckER TuckER Mar 24, 2020
data-redundancy update detailed_results Mar 27, 2020
data data Mar 24, 2020
datasets datasets Mar 24, 2020
README.md Update README.md Mar 27, 2020

README.md

Realistic Re-evaluation of Knowledge Graph Completion Methods: An Experimental Study

This repository contains codes, experiment scripts, and datasets to reproduce results of the following papers:

Realistic Re-evaluation of Knowledge Graph Completion Methods: An Experimental Study Farahnaz Akrami, Mohammed Samiul Saeef, Qingheng Zhang, Wei Hu, and Chengkai Li, SIGMOD 2020

Re-evaluating Embedding-Based Knowledge Graph Completion Methods Farahnaz Akrami, Lingbing Guo, Wei Hu, and Chengkai Li, CIKM 2018

A summary of the paper is published in a Medium blog post.

Cartesian-product

To find Cartesian Product relations in FB15k and perform link prediction on these relations:

python fb15k_cartesian_product.py

Data redundancy

To find the reverse and duplicate relations in FB15k:

python FB15k_redundancy.py

To find redundant information in test set of FB15k:

python FB15k_test_redundancy.py

AMIE

Implementations for conducting link prediction using rules generated by AMIE.

Run the experiments

Head and tail entity link prediction for a specific test relation:

python left.py relation_id
python right.py relation_id

Results obtained for each test relation are stored in ./AMIE_LinkPrediction_results.

To have the results of link prediction using AMIE:

python combine_res.py

Models-detailed-results

To obtain the link prediction results for each test relation of FB15k-237, WN18RR, and YAGO3-10:

python result_by_rel_all.py

To calculate the head and tail entity link prediction results for different relation types:

python result_by_relcategory.py

Percentage of triples on which each method outperforms others, separately for each relation:

python heatmap.py

Percentage of test triples with better performance than TransE that have reverse and duplicate triples in training set:

python reverse_percentage.py

To obtain the number of relations on which each model is the most accurate and charts:

python detailed_results.py

ComplEx

  • Cloned October 2018.
  • Implementations for ComplEx and DistMult.
  • Original repository: (https://github.com/ttrouill/complex).
  • evaluation.py is modified to calculate raw metrics.

ConvE

  • Cloned October 2019
  • Implementations for ConvE.
  • Original repository: (https://github.com/TimDettmers/ConvE).
  • evaluation.py is modified to print results for each test triple and calculate raw metrics.

OpenKE

  • Cloned October 2019.
  • Implementations for TransE.
  • Original repository: (https://github.com/thunlp/OpenKE).
  • Test.h is modified to print results for each test triple.

RotatE

TuckER

You can’t perform that action at this time.