Benchmark references

Benchmark references

The dated comparison of 35 references covers every entry in terryyz/llm-benchmark, plus eight applied comparators and 15 additional high-visibility benchmark families. Sources were checked on 11 November 2026. It compares documented scope and evidence; the external suites were not executed for that review.

Reference family	What it contributes	PALO's proposed emphasis
BIG-bench, GLUE, SuperGLUE, CLUE	Task definitions and broad capability evaluation	A narrower business-specification fidelity study
HELM, LM Evaluation Harness, BigScience Evaluation	Reproducible scenarios, adapters and metrics	A domain-specific adjudication and evidence contract
BigBIO	Dataset provenance and harmonised schemas	Corpus governance for business-source families
BigCode, CodeXGLUE, SWE-bench	Code and software-agent evaluation	Checking source requirements before implementation
OfficeQA, DABstep, OmniDocBench, FACTS	Document reasoning, data tasks, parsing and grounding	Altered business rules and human repair effort
Arena, Artificial Analysis, Vals AI	Comparison methodology, uncertainty and applied measurements	Configuration-specific evidence with a scoped procurement study

Overlap is expected. Multi-dimensional evaluation and enterprise tasks are established prior art. PALO has not demonstrated methodological superiority.

Scholar Evals was unavailable at the directory's linked repository; its capabilities are not inferred. Some referenced implementations are historical or no longer maintained. Andon Labs' Blueprint-Bench 2 is a separate spatial floor-plan reconstruction benchmark with no affiliation.

CSV | Dated source records | Earlier literature review

Visibility extension

The 15 additions are HLE, ARC-AGI, GPQA, MMLU-Pro, AIME, FrontierMath, LiveCodeBench, Terminal-Bench, SWE-Bench Pro, BrowseComp, DeepResearch Bench, OSWorld, tau2-bench, BFCL and GDPval. Each has a source-backed visibility signal, version boundaries, limitations and a PALO transfer assessment. This is a purposive selection, not a measured ranking of global hype. Repository stars are dated attention proxies, not quality or usage measures.

The four proposed transfer priorities are expert gold and atomic rubrics; end-to-end workflow reliability; separate structural and semantic evidence; and freshness, custody and disclosure. No external suite was run and no independent qualification was granted.