Skip to content

Releases: Clarify-then-Search/clarify-then-search.github.io

Hard518 dataset & eval package

11 Feb 07:46
@Clarify-then-Search Clarify-then-Search
hard518-v1
aa46822
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
Hard518 dataset & eval package Latest
Latest

Hard518 public release package.

Includes:

  • data/hard_518_queries.csv (fused_query, blurred_query)
  • data/gold_public_hard518.jsonl (gold nuggets)
  • results/candidates/*.jsonl (baseline candidate answers)
  • results/clarify_only/*.csv (questions/answers/rewrites)
  • code/ evaluation scripts (eval_gold.py, etc.)

Quick start:
export QIANFAN_API_KEY="YOUR_KEY"
python code/eval_gold.py --gold_a data/gold_public_hard518.jsonl --gold_b results/candidates/ebk1__cand_hard518.jsonl --out_dir outputs/ebk1

Assets 3
Loading