Skip to content

InclusionBench 0.2.0 — open for model runs

Choose a tag to compare

@tkwa tkwa released this 19 Sep 18:29

InclusionBench 0.2.0 is ready for AI model evaluations.

  • 50 canonical complexity classes and 1,386 frozen candidate inclusion questions.
  • OpenAI and Anthropic runners with explicit model IDs, token/time budgets, checkpoints and sealed evidence.
  • A cited mathematical baseline, implication scoring, and exact-target Lean proof verification in isolated containers.
  • Review and publication commands for model runs, with historical eligibility checked for every positive point.
  • Public leaderboard and task explorer. No synthetic model results are included.

Existing literature proofs are trusted inputs. New ordinary proofs use fresh-kernel verification; ZFC independence has an explicit expert-review lane.

Validation: 72 fast tests, 7 actual isolated proof tests, core and quantum Lean checks, and standalone wheel installation. Provider tests use fake credentials and local mock endpoints; no paid model run was made.

Start from the README quickstart. The attached Python wheel includes the class data, Lean sources, adapters, and verifier setup files. Python 3.11 or later is required.