Skip to content

SycoBench-600 v1.0.0: ACL Findings camera-ready release

Choose a tag to compare

View all tags
@debu-sinha debu-sinha released this 05 May 03:42
· 11 commits to main since this release
v1.0.0
5193ce4

Stable release of SycoBench-600, the ACL 2026 Findings benchmark for measuring sycophancy and correction selectivity in LLM assistants.

This release pins the exact artifacts behind the camera-ready paper.

Included

  • Dataset: 600 questions across 8 balanced domains (272 unique stems)
  • Raw model logs for 7 models in results/raw_camera_ready/
  • Generated tables and figures in build/camera_ready/
  • Camera-ready paper source and PDF in paper/
  • Reproducibility scripts: scripts/run_eval.py, scripts/validate_paper_full.py
  • Test suite covering release artifacts

Validation

  • pytest: 10 passed
  • Paper consistency check: 0 errors
  • All Table 3 and Table 4 numbers match build/camera_ready/tables/main_results.csv
  • Headline figures regenerated and verified

Licensing

  • Code: MIT
  • Dataset, raw logs, generated tables and figures: CC BY 4.0
  • Paper and camera-ready PDF: ACL / CC BY 4.0

Paper

OpenReview: https://openreview.net/forum?id=ijR52T6rdD

Citation

See CITATION.cff or the README citation block.

Assets 2
Loading