I maintain NECS, a dependency-light validator for TREC qrels/run structure. Eval4Sim's prepare_data.py writes four-column qrels, while search_colbert.py writes a six-column TREC run.
Would you be willing to run NECS v0.3.1 once on a pair produced by those two paths and paste the JSON result here? I am specifically interested in whether its query-coverage, unjudged-document, or advisory-rank warnings match the assumptions used before adherence scoring.
python -m pip install https://github.com/Madhvansh/Neural-E-Commerce-Search/releases/download/v0.3.1/neural_ecommerce_search_madhvansh-0.3.1-py3-none-any.whl
necs-validate --qrels <PREPARE_DATA_OUTPUT_DIR>/qrels.txt --run <SEARCH_COLBERT_OUTPUT> --format json
The validator does not compute adherence or certify the experiment, and I am not asking for an integration change. A clean pass is useful compatibility evidence too.
Release:
https://github.com/Madhvansh/Neural-E-Commerce-Search/releases/tag/v0.3.1
I maintain NECS, a dependency-light validator for TREC qrels/run structure. Eval4Sim's prepare_data.py writes four-column qrels, while search_colbert.py writes a six-column TREC run.
Would you be willing to run NECS v0.3.1 once on a pair produced by those two paths and paste the JSON result here? I am specifically interested in whether its query-coverage, unjudged-document, or advisory-rank warnings match the assumptions used before adherence scoring.
The validator does not compute adherence or certify the experiment, and I am not asking for an integration change. A clean pass is useful compatibility evidence too.
Release:
https://github.com/Madhvansh/Neural-E-Commerce-Search/releases/tag/v0.3.1