Skip to content

Hard518 dataset & eval package

Latest
Latest

Choose a tag to compare

View all tags
@Clarify-then-Search Clarify-then-Search released this 11 Feb 07:46
· 13 commits to main since this release
hard518-v1
aa46822
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Hard518 public release package.

Includes:

  • data/hard_518_queries.csv (fused_query, blurred_query)
  • data/gold_public_hard518.jsonl (gold nuggets)
  • results/candidates/*.jsonl (baseline candidate answers)
  • results/clarify_only/*.csv (questions/answers/rewrites)
  • code/ evaluation scripts (eval_gold.py, etc.)

Quick start:
export QIANFAN_API_KEY="YOUR_KEY"
python code/eval_gold.py --gold_a data/gold_public_hard518.jsonl --gold_b results/candidates/ebk1__cand_hard518.jsonl --out_dir outputs/ebk1

Assets 3
Loading