chmod +x run.sh
./run.sh
#Dependencies
TagMe: pip install tagme (GCUBE_TOKEN = "bfbfb535-3683-47c0-bd11-df06d5d96726-843339462")
Argparse: conda install -c anaconda argparse=1.3.0
Copy: For making deepcopy
Itertools: conda install -c auto more-itertools=2.2
TrecCar: https://github.com/TREMA-UNH/trec-car-tools/tree/master/python3
Following results will appear once the shell script is run.
============================================
Generating results file with regular queries
============================================
Gathered Pages
Gathered Paragraphs
Gathered Regular Queries
Gathered Results
=======================================
Running evaluation framework...........
=======================================
runid all test
num_q all 60
num_ret all 2100
num_rel_ret all 174
map all 0.2295310245310245
gm_map all 0.6585904496408844
Rpec all 0.10599206349206347
recip_rank all 0.22433261183261183
P_5 all 0.07
P_10 all 0.04500000000000001
P_15 all 0.03333333333333333
P_20 all 0.026666666666666675
P_30 all 0.018333333333333333
P_100 all 0.005500000000000001
P_200 all 0.0027500000000000007
P_500 all 0.0011000000000000003
P_1000 all 0.0005500000000000001
=================================================================
Generating results file with tagme enchanced queries.........
=================================================================
Gathered Pages
Gathered Paragraphs
Generating TagMe enchanced queries........
Gathered Tagme Enhanced Queries
Gathered Results
=================================================================
Running evaluation framework on results on tagme enhanced results
=================================================================
runid all test
num_q all 60
num_ret all 2100
num_rel_ret all 174
map all 0.2079354904354904
gm_map all 0.6511606087084545
Rpec all 0.11154761904761905
recip_rank all 0.2138481888481888
P_5 all 0.07666666666666667
P_10 all 0.04666666666666667
P_15 all 0.03333333333333333
P_20 all 0.026666666666666675
P_30 all 0.018333333333333333
P_100 all 0.005500000000000001
P_200 all 0.0027500000000000007
P_500 all 0.0011000000000000005
P_1000 all 0.0005500000000000002