Wheatly

A tuned and customised indexer built on top of Apache Lucene.

Build indexer

You can build the indexer by either running shell script files or the set of commands below:

$ sudo apt-get install maven
$ mvn clean package appassembler:assemble

alBERT

simply, adaptive learning Bidirectional Encoder Representations from Transformers, is a neural network model adapted from BERT to learn patterns in passage rankings.

Pre-processing

BERT only processes a certain format of files , TF-Records. Run corresponding files for each datasets:

• convert_treccar_to_tfrecord.py
• convert_msmarco_to_tfrecord.py

Train and Prediction

We have provided Google Colaboratory environments for the two datasets, TREC-CAR and MS-MARCO. You can copy and train your own model using:

• TREC-CAR colab
• MS-MARCO colab

=======

Why Wheatly?

Wheatly (a play with the name of the founder of Index Society, Henry Benjamin Wheatley)