Skip to content

Paper scores and evaluation artifacts

Choose a tag to compare

View all tags
@shichengf shichengf released this 03 Oct 09:54
paper-artifacts-v1
c0b69db

This release provides the paper score archive and the Harbor adapter for hidden evaluation.

  • Paper scores: 2,000 task scores for ten models, model identifiers, API configuration, and a script that recomputes the 130 overall, role, and family means in Tables 4 and 13 for the capability-coverage track.
  • Harbor adapter: code and instructions for running Agents against the 20 private evaluation configurations using the v1.0.0 runtime and scorers.
  • Supporting records: 5×5 shared-market study and catalog provenance.

The paper's task and scorer configuration is preserved at v1.0.0, also tagged kdd2027-submission. Each archive includes file checksums and has a separate SHA-256 checksum attachment.

Assets 6
Loading