Skip to content

VN-GeoQA v3.0.0 — frozen benchmark

Latest
Latest

Choose a tag to compare

View all tags
@itskyf itskyf released this 05 Sep 04:24
· 47 commits to 596948222a20baf8c6f817a71aed1f8bf52a1bb9 since this release
v3.0.0
3d75d30

VN-GeoQA v3.0.0 — frozen Vietnamese GS-QA benchmark.

  • 28 canonical TIDs x 100 = 2,800 questions, seed 42, from Geofabrik vietnam-260901.osm.pbf (sha256 in MANIFEST).
  • Location gold (T13–T20) freezes geometry and native OSM address components and a deterministic canonical address string; candidates are address-bearing POIs (predicate stated verbatim in the Text2SQL prompt).
  • T7/T8 external facts are verifiably out-of-schema (capacity removed from the reference schema).
  • Verified 2,800/2,800 (incl. loc address guards, T7/T8 out-of-schema assertion); second seed-42 run byte-identical; human QC approved (docs/qc_spot_check_v3.0.0.tsv).
  • Assets: vn-geoqa.zip (dataset), osm-vn.dump (PostgreSQL custom-format reference DB dump, restored with pg_restore). Restore via scripts/restore_dataset.sh and scripts/restore_database.sh (SHA-256 pinned).
Assets 9
Loading