Repository navigation
ParamCAD-AgentBench v2.2.0-preview
Pre-release
Pre-release
ParamCAD-AgentBench v2.2.0-preview
This preview publishes the benchmark protocol, public runtime, metrics, validation tools, and a leakage-aware data split covering 2,409 unique CAD models and 4,818 task records.
Included
- 800 development tasks over 400 models, with reference actions for reproducibility.
- 3,200 test tasks over 1,600 models, without reference actions.
- 818 long-horizon challenge tasks over 409 models, without reference actions.
- Abstract and detailed language tracks for every model.
- Tier-aware evaluation utilities, runtime checks, data manifests, checksums, and documentation.
Annotation status
The descriptions in this preview are the frozen v2.2 snapshot. A separate evidence-gated annotation pipeline is being completed. Its audited descriptions will be published as a follow-up release after every model reaches a successful final status.
Evaluation policy
Test and challenge reference programs and source geometry are intentionally withheld. This prevents direct oracle leakage and preserves the validity of future private evaluation.