Correlation, Prediction and Ranking of Evaluation Metrics in Information Retrieval

Directory Structure

The subfolders contain the codes and data used in the paper.

Folder	Description
Data	Metric Scores for Prediction and Ranking
Figs	Figures used in the paper
Ranking	Code for Metric Ranking
Regression	Code for Metric Regression

All the TREC data has been anonymized.

Python Library Requirements

Pandas, Numpy, Matplotlib, Sklearn, Scipy

Condition of Use

Please cite the [paper] (https://www.ischool.utexas.edu/~ml/papers/gupta-ecir19.pdf) whenever you use the methods or data in any publication or presentation.

The BiBTex is provided below for convenience:

@inproceedings{ecirGuptaKKL19,
  author    = {Soumyajit Gupta and
               Mucahid Kutlu and
               Vivek Khetan and
               Matthew Lease},
  title     = {Correlation, Prediction and Ranking of Evaluation Metrics in Information
               Retrieval},
  booktitle = {Advances in Information Retrieval - 41st European Conference on {IR}
               Research, {ECIR} 2019, Cologne, Germany, April 14-18, 2019, Proceedings,
               Part {I}},
  pages     = {636--651},
  year      = {2019}
}