Skip to content
No description, website, or topics provided.
Roff Java Python Other
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.idea
_old
data
libs
mapping_results
src/main/java/edu/umich/templar
.gitignore
LICENSE
README.md
build.gradle
config.properties.example
requirements.txt
templar.iml
word2vec_server.py

README.md

Templar

Build

We use the gradle shadowJar plugin to build the project.

gradle shadowJar

Setup

  • Make sure to run python word2vec_server.py before (this will use port 10000 by default).
  • Ensure your database properties are set correctly in config.properties in the main project folder. If this doesn't exist, you should create it by copying config.properties.example.
  • a functioning MySQL instance with necessary data from each dataset pre-loaded. For example, have MySQL up and running, then create database name mas and follow instructions on the MAS dataset README.

Running Templar tests

  • TemplarCV - Runs a cross-validation test on a specific dataset given some parameters.

After building, we can run:

java -cp build/libs/templar-all.jar edu.umich.templar.TemplarCV <dataset> <log_level> <log_join_on>

Choices for each argument:

  • <dataset>: mas, yelp, imdb
  • <log_level>: full, no_const, no_const_op
  • <log_join_on>: true, false

Disabling the candidates cache

Since a lot of keywords are frequently reused in each dataset, we implemented a cache to speed up testing. This can be enabled/disabled by changing the setting for ENABLE_CACHE in the edu.umich.templar.main.settings.Params.

These caches will be saved in data/<dataset>/<dataset>.cands.cache, so to clear the cache, just delete these files.

Adding new datasets

In order to add new datasets, you need to

  1. Load the dataset with name <dataset> into MySQL.
  2. Create the folder data/<dataset>. Each dataset is required to have the following files (see existing datasets for examples):
    • <dataset>_keywords.csv: pre-parsed keywords, metadata, and answers. See other datasets for examples. Note specifically that we allow multiple correct answers, separated by semicolons, and that pairs are given in comma-separated form. This formatting matters because our accuracy evaluation is done via string comparison.
    • <dataset>_joins.csv: correct join paths for each query. These are in a nested, parenthetical format, where the first table alphabetically is always the first, then a table's children is given by parentheses after it, and multiple children of a tree are separated by commas. For example, author(organization,writes(publication)) is a join path where author is the first alphabetical table name, then its children are organization and writes, and then writes has publication as a child. This formatting matters because our accuracy evaluation is done via string comparison.
    • <dataset>_all.sqls: the correct SQL labels for each NLQ, one query per line. This is fed in as our query log.
    • <dataset>.fkpk.json: a JSON file listing all the foreign key-primary key relationships in the schema
    • <dataset>.main_attrs.json: defining the main/display/default attributes for each relation
    • <dataset>.proj_attrs.json: defining the paired attributes for each relation
You can’t perform that action at this time.