Skip to content
Permalink
Branch: master
Find file Copy path
Find file Copy path
1 contributor

Users who have contributed to this file

75 lines (46 sloc) 3.21 KB

Anserini: SIGIR Forum 2018 Experiments

This page documents code for replicating results from the following article:

Note that the commit 2c8cd7a referenced in the article is out of date with respect to the latest experimental results. See "History" section below.

Requirements: Python>=2.6 or Python>=3.5 pip install -r src/main/python/requirements.txt

Folds:

Parameter Tuning

First, change the index path at src/main/resources/fine_tuning/collections.yaml. The script will go through the index_roots and concatenate with the collection's index_path and take the first match as the index path.

BM25 Robust04 (runs + eval + print results):

python src/main/python/fine_tuning/run_batch.py --collection robust04 --model bm25 --n 44 --run --use_drr_fold

QL Robust04 (runs + eval + print results):

python src/main/python/fine_tuning/run_batch.py --collection robust04 --basemodel ql --model ql --n 44 --run --use_drr_fold

BM25+RM3 Robust04 (runs + eval + print results):

python src/main/python/fine_tuning/run_batch.py --collection robust04 --model bm25+rm3 --n 44 --run --use_drr_fold

BM25+AxiomaticReranking Robust04 (runs + eval + print results):

python src/main/python/fine_tuning/run_batch.py --collection robust04 --model bm25+axiom --n 44 --run --use_drr_fold

Tuned Run

Tuned parameter values:

To be clear, these are the tuned parameters on that fold, trained on the remaining folds.

The follow script will reconstruct the tuned runs for BM25 + RM3:

python src/main/python/fine_tuning/reconstruct_robus04_tuned_run.py \
 --index lucene-index.robust04.pos+docvectors+rawdocs \
 --folds src/main/resources/fine_tuning/robust04-paper2-folds.json \
 --params src/main/resources/fine_tuning/robust04-paper2-folds-map-params.json

Change paper2 to paper1 to reconstruct using the folds in paper 1.

History

  • commit 407f308 (2019/1/2) - Added results for axiomatic semantic term matching.
  • commit e71df7a (2018/12/18) - Upgrade to Lucene 7.6.
  • commit 18c3211 (2018/12/9) - minor fixes.
  • commit 2c8cd7a (2018/11/16) - commit id referenced in SIGIR Forum article.
You can’t perform that action at this time.