v0.2-dev results

Development preview of CADClamp v0.2: 47 prompts (tiers 1 to 5, with a DfAM set) in seven CAD languages: build123d, OpenSCAD, CadQuery, FreeCAD, Blender, and Rhino 8 and Autodesk Fusion driven through their MCP servers, plus Track C (ten McMaster-Carr parts redesigned to print). The live leaderboard is at https://christiannhill.github.io/cadclamp/ and the findings are in the README.

Scores use harness 0.3, engine 0.2.2 and spec 0.2.3 (Track C trackc-0.1.3). Every prompt probes each stated feature, proved against mutants of its reference solution. Criterion checks count in the headline relative to the reference. Ten prompts form a held-out check. The top three models ran extra epochs: opus-5.5 and fable-5.1 three, astra two. No model was rerun for a spec change: the scores stored inside older logs predate the current checks, so score with scripts/leaderboard.py --regrade (leaderboard-latest.json is that result).

cadclamp-v0.2-dev-results.tar.gz (1.2 GB)

• 159 run logs in Inspect .eval format: 12 models in 7 languages, the extra epochs, the text and image repair rounds, the reference and 20 mm cube baselines, and the Track C runs. Each log holds every generation, its stderr, and its report card.
• 3,475 scored meshes (meshes/<sha1>.stl), so scripts/leaderboard.py <logs> --regrade re-scores every part with no model calls.
• 9,415 OrcaSlicer results (slicer/<machine>/<sha1>.json) from five printers with their makers' own profiles.
• leaderboard-latest.json, the spec audits (spec-flips-0.2.2.json, spec-flips-0.2.3.json, probe-audit.json), the parametric reruns (param-probe-cache.json), the slicer report, and MANIFEST.json with every version.

Home-directory paths in the logs are replaced with ~. Code is Apache-2.0 and the data is CDLA-Permissive-2.0.