Paper scores and evaluation artifacts

This release provides the paper score archive and the Harbor adapter for hidden evaluation.

• Paper scores: 2,000 task scores for ten models, model identifiers, API configuration, and a script that recomputes the 130 overall, role, and family means in Tables 4 and 13 for the capability-coverage track.
• Harbor adapter: code and instructions for running Agents against the 20 private evaluation configurations using the v1.0.0 runtime and scorers.
• Supporting records: 5×5 shared-market study and catalog provenance.

The paper's task and scorer configuration is preserved at v1.0.0, also tagged kdd2027-submission. Each archive includes file checksums and has a separate SHA-256 checksum attachment.

Minimized catalog data

Minimized catalog archive. Full raw-row payloads and unused source-file metadata removed; only ten required fields retained for 271 source records. Compressed archive: 103.22 MB.

Database SHA-256: 558426ddee209b11bd324de159d61a3af11849e7a8a99b01b39fded57866402e
Archive SHA-256: e0b87bb6ed9f9b29ca7bf1fb71a94f743e1b4a3145c4cbfb98e56ea342d31048

ACWorld Large-Catalog Data v1

Prepared data for the 60-task large-catalog benchmark. Contains the complete normalized SQLite catalog required by the benchmark, 60 deterministic task definitions, catalog summary, validation report, and readable oracle reports. Raw web pages and model outputs are not included. The archive is split into 65 assets; the public runner downloads, assembles, and validates them automatically when --tasks 60 or --tasks 260 is selected.

Data use terms: this data package is not covered by the repository MIT License. It may be downloaded and used for research reproduction and evaluation of ACWorld. No permission is granted to redistribute, mirror, rehost, sublicense, sell, or republish the package or its contents. See the repository README for the complete notice.