Repository navigation
Releases
v0.4.3
InclusionBench v0.4.3
Current build: 1
Build 1 keeps software version 0.4.3 and raises the proof-export cap from 32 MiB to 256 MiB (268,435,456 serialized UTF-8 bytes). The auditor now parses proof exports inside its isolated container. Each isolated stage retains its 8 GiB memory limit, and audit reports have a separate 32 MiB cap. Source limits, declaration limits, proof checks, and the frozen benchmark are unchanged.
The current download is build 1: inclusion_bench-0.4.3-1-py3-none-any.whl. It was built from source commit 7d70e1e143407316196134ba6b295a502456910d. The v0.4.3 tag, its source archives, and the original unnumbered wheel remain pinned to the initial release. The added wheel build number distinguishes the follow-up while preserving package version 0.4.3.
- Wheel SHA-256:
a0552ede8c60ce6e4dd4ce3dda76ce1af1735c9fe753c8db0a0379605fbfe815. - Submission guide for build 1.
To install or replace an existing 0.4.3 installation with this build:
python3 -m pip install --force-reinstall "https://github.com/tkwa/inclusion-bench/releases/download/v0.4.3/inclusion_bench-0.4.3-1-py3-none-any.whl#sha256=a0552ede8c60ce6e4dd4ce3dda76ce1af1735c9fe753c8db0a0379605fbfe815"One representative 262,378,199-byte proof export passed the real isolated verifier with a 300-second timeout per stage and the existing 8 GiB memory limit. Other proofs remain subject to the time and memory limits.
Original release notes — tag v0.4.3
These notes describe the original tagged source and unnumbered wheel, which remain unchanged.
Agents can submit ordinary multi-file Lean projects and use uniform integer-array interfaces with explicit complexity and size guarantees.
- Multi-file proofs. A small
submission.jsonnames the entry module and source files. Normal imports connect helper modules; installed pinned Lean and Mathlib libraries are available. The verifier builds the module graph in its isolated container and exports all declarations needed by the target proofs. Legacyproof.leanbodies remain supported. - Complete submission evidence. Provider adapters retain the module tree, claim mapping, and literature requests. The runner seals every file. Proof review and scoring bind the report to the exact manifest and sources under one project directory, so a checked entrypoint cannot stand in for unchecked helpers.
- Uniform signed-integer arrays. The support library supplies binary encodings, explicit length and bit-size bounds, bounded tabulation, indexed reads and updates, maps, arithmetic, sums, dot products, and rectangular matrices. The same algorithm must work across all inputs. Generic operations need a uniform bit-time computation guarantee.
- Separate contracts for large implicit arrays. Efficient indexed access is distinguished from constructing an entire array. Materialization requires an additional polynomial bound on the number of entries; efficient access alone does not justify fast whole-array sums or dot products.
- Setup and examples. The starter project, submission guide, theorem catalog, and packaged two-file example support the new workflow. The Linux setup command can provision additional modules from the pinned Mathlib cache with
--mathlib-module.
The support catalog now contains 189 kernel-proved helpers and 31 explicitly cited premises. This release adds 77 helpers and 12 premises for standard binary arithmetic and array computation; the succinct-array layer adds no new premises.
Projects allow up to 128 Lean files and 16 MiB including the manifest; the manifest is capped at 64 KiB. Legacy source bodies keep their 2 MiB limit. Exported proof JSON remains capped at 32 MiB, and the existing isolated time and memory limits still apply. Candidate object files never enter fresh kernel replay. Newly submitted inductive types and structures remain outside the codec; imported library types are supported.
This release preserves the frozen v0.4.0 benchmark definitions, dataset, questions, historical judgments, and scoring rules. No new mathematical result or model score is claimed. See the submission guide for project and array examples.
InclusionBench v0.4.2
InclusionBench v0.4.2
Agents can now express proofs through textbook complexity-theory interfaces and reuse published results without formalizing their original proofs. The release addresses the cost of rebuilding standard mathematics after finding a new argument.
- Familiar targets and reusable theorems. All 61 class definitions have short aliases, and the existing benchmark facts and rules have readable
Known.*names. A searchable catalog gives statements and citations. - Textbook proof obligations. The support library supplies 112 proved interfaces over 19 explicitly cited standard premises. It covers polynomial functions and reductions, complete-problem arguments, P predicates, NP witness relations, coNP universal certificates, GapP arithmetic, counting-class characterizations, and oracle/hierarchy reasoning. Standard model bridges are explicit cited premises; the derived interfaces are kernel-checked.
- Additional literature dependencies. A submission can name a published theorem and state its exact Lean type. The verifier checks the new argument conditional on that theorem. A maintainer reviews the source, precise statement, and model alignment; re-proving the cited result is unnecessary. Pending or revoked dependencies cannot support an admitted proof.
- A smaller submission workflow.
submission-initcreates exact targets and an editable proof;theoremssearches the catalog;check-submissionchecks the proof and dependencies;review-literaturerecords a maintainer decision. Provider adapters preserve citations and the trusted support snapshot alongside run evidence. - Target-only proof export. Proof format 3 exports the declarations actually needed by the requested targets. It retains shared expression tables and accepts formats 1 and 2. Fresh kernel replay still checks every reconstructed declaration and the exact target.
Literature decisions bind the reconstructed declaration, its local defining context, citations, dataset, generated target baseline, and trusted library/checker versions. Changing only the novel proof body can reuse an approval. Changing what an approved statement means cannot. Decisions are append-only and can be revoked.
This is a software and support-library release over the frozen v0.4.0 benchmark. The 50 scored classes, 61 operational definitions, 1,324 questions, dataset, taskset, historical decisions, and scoring rules are unchanged. No new model score or solution to an open problem is claimed.
The support library is a starting collection, not a complete formalization of complexity theory. New results still require precise arguments. Submissions remain one Lean source body under fixed imports; new submitted inductive types and structures remain unsupported. The source/export size limits remain 2 MiB/32 MiB. See the agent submission guide for examples and the literature review workflow.
InclusionBench v0.4.1
InclusionBench v0.4.1
Proof exports now preserve shared subexpressions instead of duplicating them in nested JSON trees. Format version 2 stores names, universe levels, and expressions in indexed tables shared by each declaration's type and value. Repeated references reuse the reconstructed Lean object. This reduces export size for proofs with repeated terms and avoids exponential expansion caused by nested sharing.
A synthetic sharing regression shrank from 7,340,123 bytes to 358 bytes. An 80-level doubling expression uses 81 nodes and 1,102 bytes; these are codec fixtures, not measurements of typical proof files.
The decoder accepts legacy version 1 exports. Version 2 rejects malformed, forward, cyclic, and out-of-range references. Reconstructed declarations still undergo the same fresh Lean kernel checks, exact theorem-target checks, and axiom allowlist checks. The accepted declaration kinds and proof-size limits are unchanged.
This is a software patch to the frozen v0.4.0 benchmark. The 50 scored classes, 61 class definitions, 1,324 questions, scoring rules, historical decisions, dataset hash, taskset hash, and publication record are unchanged. Software packages and the Git tag use v0.4.1; benchmark metadata continues to identify v0.4.0. Proof-verifier reports record the new checker hash.
See the proof-format documentation for compatibility and limits, and the v0.4.0 release notes for the benchmark snapshot.
InclusionBench 0.4.0
v0.4.0 scores 50 complexity classes selected from 61, with 1,324 frozen questions at the September 1, 2026 cutoff. All endpoints are classes of total binary decision languages. Verified implications count toward the score.
- Added: BPL, UL, PL, BQL, uniform NC¹, ∃R, P^#P, CH, QMA(2), StoqMA, QSZK.
- Moved to unscored background: AC⁰, coUP, coMA, coAM, coQMA, FewP, LWPP, WPP, coRP, E, Π₂P.
Background classes retain their definitions and implication paths. Results about them can earn credit through scored consequences, but their own pairs earn no direct points. The roster decision explains the scientific tradeoffs and lost coverage; this is an editorial selection, not an objective ranking.
Scores are not directly comparable with v0.3.1. Publication preserves the reviewed dataset, taskset and definition identities. The separate publication record records the release state; provisional labels in frozen audit records remain historical.
The AI-assisted audit's subjective residual estimate is about three incorrectly classified scored pairs, not below two. This is neither a calibrated bound nor human expert certification. Existing cited literature results remain trusted premises; new ordinary claims require mathematical review and isolated Lean verification. A missed pre-cutoff result earns no point and requires a recorded correction.
See the immutable release documentation, audit assessment, and merged PR #1. The tkwa.me frontend is maintained separately.
InclusionBench 0.3.1
InclusionBench 0.3.1 admits independence conditional on arithmetic soundness of ZFC, alongside unconditional and Con(ZFC)-conditional metatheorems. Arithmetic soundness means that every first-order arithmetic sentence whose standard translation ZFC proves is true in standard N. The target theory remains ZFC itself.
The Lean library now defines first-order arithmetic syntax and standard-N semantics, arithmetic soundness, the three permitted premise categories, and conditional independence certificates. It proves that arithmetic soundness implies consistency with the proof system's required logical rule explicit, and that a consistency-conditional certificate can be reused under the soundness premise without changing its target. Full ZFC encoding and correspondence remain expert-review obligations.
Task prompts and provider instructions use the new policy. Every admitted independence pair requires its own encoded sentence and both nonderivability arguments; its premise remains in evaluated provenance and leaderboard data. Independence does not produce ordinary inclusion or separation consequences.
The 50 classes, 709 known inclusions, 413 known noninclusions and 1,378 questions are unchanged. Version 0.3.0's historical audit is preserved, with an explicit policy-adoption record and a bounded primary-source review of the expanded scope.
Validation: local Python tests, Lean core and axiom audits, all 1,122 baseline inference traces, the SAT audit, generated-data/freeze checks, and a fresh offline package installation passed. The GitHub verification workflow also passed the complete quantum-library build and all seven actual isolated proof-admission tests.
The public site is tkwa.me, developed separately. This release updates the benchmark repository and its Pages mirror.
Attached wheel SHA-256: cc721b392d1e1f6a81d92589ca69f0d6ba0a857ad61ed6e85c64dd5800b9acb5.
InclusionBench 0.3.0 — audited cutoff release
InclusionBench 0.3.0 records the extended September 1, 2026 cutoff audit and is ready for model runs.
- Corrects eight previously missed known directions: NE and NEXP are each not contained in Θ₂P, coNP, coRP and coUP.
- Freezes 709 inclusions, 413 noninclusions and 1,378 questions audited as open at the cutoff. Runs reuse the recorded historical decisions; new answers still require proof and run-integrity review.
- Adds 106 implication rules, for 249 total.
- Checks all 1,122 known-label inference traces in Lean and every candidate polarity in an independent finite-theory SAT encoding.
- Proves normalization of the actual quantum gates, circuits and register embedding, plus acceptance bounds between zero and one.
The final working estimate is 1.98 remaining incorrectly classified pairs (ledger sum1.9822), slightly below the requested stopping threshold. This is a subjective, revisable judgment with broad sensitivity, not a calibrated bound. No model result or score has been invented.
Audit report · Leaderboard and task explorer · Run instructions
InclusionBench 0.2.0 — open for model runs
InclusionBench 0.2.0 is ready for AI model evaluations.
- 50 canonical complexity classes and 1,386 frozen candidate inclusion questions.
- OpenAI and Anthropic runners with explicit model IDs, token/time budgets, checkpoints and sealed evidence.
- A cited mathematical baseline, implication scoring, and exact-target Lean proof verification in isolated containers.
- Review and publication commands for model runs, with historical eligibility checked for every positive point.
- Public leaderboard and task explorer. No synthetic model results are included.
Existing literature proofs are trusted inputs. New ordinary proofs use fresh-kernel verification; ZFC independence has an explicit expert-review lane.
Validation: 72 fast tests, 7 actual isolated proof tests, core and quantum Lean checks, and standalone wheel installation. Provider tests use fake credentials and local mock endpoints; no paid model run was made.
Start from the README quickstart. The attached Python wheel includes the class data, Lean sources, adapters, and verifier setup files. Python 3.11 or later is required.