Boolean Benchmark Runner Repository

This repo contains small standalone runners that all support the same request/result protocol to run standardized benchmarks.

Documentation

• Runner Overview — concepts, folder layout, execution model
• Runner I/O Contract — request/result JSON, timing fields, multi-run protocol
• Runner Manifest Spec — runner.yaml schema and capabilities
• Build Script Spec — build.py responsibilities and build-info.json
• Adding a New Runner — step-by-step, checklist, starter templates

Project	Type	Variant	Version	Capabilities	License
Blender	Tool	Exact	5.1		GPL
Blender	Tool	Fast	5.1		GPL
Carve	Library	Default	2014-9	R	GPL
CGAL	Library	Corefine (EPEC)	6.1.1	E R I	GPL / Commercial
CGAL	Library	Nef (EH-EI)	6.1.1	E R I S	GPL / Commercial
CGAL	Library	Nef (EH-EI, Reg)	6.1.1	E R I S	GPL / Commercial
Geogram	Library	Boolean	1.10.0	E R S	BSD-3
Interactive and Robust Mesh Booleans	Research	Default	2024-6	R	MIT
Manifold	Library	Default	3.4.0	R I	Apache-2.0
mcut	Library	Default	1.3.0	R	LGPL / Commercial
Mesh Arrangements for Solid Geometry	Research	Default	2.6.0	E R S	GPL / Commercial (via CGAL)
MeshLib	Library	Default	3.1.1.211		Free (non-commercial) / Commercial
QuickCSG	Research	Default	2022-10		Free (non-commercial) / Commercial
Solidean	Library	Default	2026.1	E R I S	Proprietary (Free non-commercial)
Trueform	Library	Default	0.7.0	R I S	Free (non-commercial) / Commercial
VTK	Library	BoolOp	9.6.0		BSD-3
VTK	Library	LoopBool	9.6.0		BSD-3

Capability Legend

Capabilities are self-reported by the respective projects. They reflect what a method claims within its stated preconditions, not independent verification. A later benchmark may show issues even if a method claims robustness.

• [E] Exact arithmetic: exact constructions, not just predicates. Operations like (A \ B) u (A n B) == A hold exactly.
• [R] Robust: the method claims robustness within its stated preconditions.
• [I] Stable under repeated/iterative operations: the result is usable for additional booleans without loss of information. Methods that output indexed float/double triangle meshes generally cannot claim this unless specifically designed for it.
• [S] Supports self-intersections: the method must support some self-intersecting input meshes properly and by design.

Runner Folder Structure

Each runner lives under <family>/<version>/ (or <family>/<version>/<variant>/):

runner.yaml          # runner identity and runtime config (mandatory)
build.py             # build script — run with: uv run build.py
CMakeLists.txt       # CMake project (C++ runners)
vcpkg.json           # vcpkg manifest declaring C++ dependencies
src/main.cc          # adapter source code
.gitignore           # ignores auto-created bin/ and download/
bin/                 # auto-created: compiled executables, build-info.json
download/            # auto-created: downloaded/cloned upstream sources

Only the runner.yaml is mandatory, the rest depends on the runner type. A common C++ setup is shown.

Quickstart

• uv run build-all.py -y builds all runners, accepting all licenses (can take 30+ min cold) (-y means you accept all licenses, including the free non-commercial licenses of each runner; omit -y to be prompted interactively where required)
• uv run bootstrap-vcpkg.py (optional, pre-warms vcpkg packages for all runners, 60+ min cold)
• uv run run-smoke-test.py runs every built runner against the cube-minus-cube smoke case and prints a results table (-r <pattern> to filter by slug/name, repeatable)

TODOs

• Blender: needs an exact + self-intersection variant (the current Exact variant does not support self-intersecting input).
• Add N for non-manifold handling. This requires a properly thought out notion of what "correctness" means but is otherwise high-value.