v0.3+paper-repro - NeurIPS paper archive

The code and results behind the paper Mosaic: A Benchmark Suite for Differentiable Physics Solvers (NeurIPS 2026 Evaluations & Datasets Track, arXiv:2606.27895).

• The tag pins the harness, all 19 solver Tesseracts, every experiment definition, the paper's LaTeX source with the scripts that generate its figures and tables (paper/), and the solver-in-the-loop study (studies/kolmogorov-sil/). production.uv.lock is the harness environment of the paper's run.
• mosaic-paper-a100-results.tar.gz (1.2 GB) holds the results behind the paper's figures and tables, computed on one NVIDIA A100 80 GB. It also contains the mosaic status of the run, the package lists of every solver image, and the host's hardware details. The archive's README.md describes the setup and contents.

Regenerate the figures and tables

No Docker or GPU needed:

git clone --branch v0.3+paper-repro https://github.com/pasteurlabs/mosaic && cd mosaic
wget -qO- 'https://github.com/pasteurlabs/mosaic/releases/download/v0.3%2Bpaper-repro/mosaic-paper-a100-results.tar.gz' | tar xz
export MOSAIC_RESULTS_DIR=$PWD/mosaic-paper-a100-results/results

uv run paper/make_figures.py
uv run paper/make_tables.py --extra-results "$MOSAIC_RESULTS_DIR"
uv run studies/kolmogorov-sil/summarize.py "$MOSAIC_RESULTS_DIR" mosaic-paper-a100-results/results-sil-stability/*

To re-run the experiments from scratch (about two days on an A100), see Reproduce the paper in the README.

v0.2.0

Release v0.2.0

Action required

Thermal mesh problem rescaled — old results are no longer comparable (#140)

The thermal-mesh benchmark previously used a fixed slab thickness (Lz = 1.0), which produced extremely anisotropic elements at high refinement (e.g. ~500:1 aspect ratio at N=1024). That turned the benchmark into a test of how well a preconditioner tolerates elongated elements rather than a clean scaling test. The domain thickness is now auto-rescaled so elements stay roughly cubic across refinement levels.

What this means for you:

• Reference data has changed. The bundled forward_* references were regenerated. If you cache or compare against older thermal-mesh baselines, discard them and re-run.
• Boundary flux is now interpreted per unit area rather than as a total flux. Numerical results are otherwise unchanged in character, just better conditioned. Update any downstream analysis that assumed total-flux semantics.

Re-run your thermal-mesh solvers to get results comparable with this release.

Highlights

Richer, more trustworthy PR benchmark comments (#137)

The benchmark comment posted on pull requests now surfaces the signals contributors actually need:

• 📊 Metric changes reports numeric shifts (e.g. gradient rel-error, converged loss) even when a cell keeps the same coarse ok/anom/fail status — so a regression that doubles an error but stays "ok" no longer slips through silently.
• ⏱ Timing (indicative) is split out from correctness metrics, since shared-runner contention makes wall-clock noisier than the deterministic checks.
• Scoped attribution: a benchmark:solver run only attributes changes to the cells it actually re-measured, so baseline drift on untouched cells is no longer mis-reported as your regression.
• Every run now posts a verdict and a coverage banner stating what was measured versus carried over from baseline — even a skipped run gets a one-line message instead of silence.

No action needed. Note that the numeric metric/timing sections only start rendering once main has a metrics-aware baseline (the first publish-results.yml run after this release).

XLB checkpointing (#135)

XLB now supports gradient checkpointing, reducing VJP memory use. This fixes some of the observed VJP slowness in 2D and unlocks larger problem sizes in 3D.

Other changes

• INS.jl upgraded to v5.0.0 — native ν gradient, in-place forward, and float32 support for better performance (#103).
• mosaic run -s <solver> now validates solver names against all problems, not just the one passed via -p, giving clearer errors (#131).
• Harness: common Docker setup failures are now diagnosed with actionable messages instead of raw stderr dumps (#126).
• Harness: anomaly status is now reported from min_cosine, and the forward suite's status checks are applied correctly (#141, #148).
• Fixes to result merging/plotting and the 3D TGV reference (#151, #136).

See the full changelog below for the complete list of documentation and minor fixes.

What's Changed

Features

• Surface numeric metric regressions and scope PR benchmark diffs (#137)
• Implement checkpointing for XLB (#135)
• Implement auto-rescaling to fix anisotropy of thermal mesh (#140)
• Compare release PRs against the previous release (#153)

Bug Fixes

• Stop overlapping text in topopt_3d figure (#91)
• External contributor friendly doc builds (#111)
• Update installation instructions (#114)
• Validate -s solver names against all problems, not just -p (#131)
• Implement warp suggestions (#132)
• (harness) Diagnose common Docker setup failures instead of dumping stderr (#126)
• (harness) Report the anomaly status from min_cosine (#141)
• Use proper reference for 3D TGV (#136)
• Publish race (#146)
• Result merge and plotting (#151)
• (harness) Apply the forward suite's status checks (#148)
• Status-check overrides replace suite defaults (#152)

Documentation

• Link the Mosaic arXiv paper in the README (#88)
• Fix solver reference pages dropping all canonical IO fields (#84) (#90)
• Add which solver is best section to readme (#101)
• Surface numerical ranking for 3D NS optimization (#86) (#92)
• Correct INS.jl solver description and exclusion rationale (#102)
• Add disclaimer on llm usage (#134)

Performance

• Upgrade INS.jl tesseract to v5.0.0 (native ν gradient, in-place forward, float32) (#103)

Full diff: v0.1.1...v0.2.0

v0.1.1

Release v0.1.1

What's Changed

Bug Fixes

• Fix jax-fem; make sure full tracebacks get forwarded to PR status reports (#81)

Refactor

• Upgrade tesseract core to v1.10.0 (#75)

Documentation

• Better readme + docs (#74)

Testing

• Trigger jax-fem build (#82)

Full diff: v0.1.0...v0.1.1

v0.1.0

Release v0.1.0

Initial release of the Mosaic benchmark suite.

What's Changed

Features

• (run) Add --continue to resume after crashes (#25)
• (run) Result_partial.json checkpointing + fd_check PoC (#26)

Bug Fixes

• Cap drag_opt max_iters to keep optimization runner ≤ 6h (#56)
• Fix all tesseract builds (#62)

Refactor

• Delete code ratio logic and tesseracts command (#47)
• Keep only one copy of shared code (#44)
• Simpler result format + artifact-based result storage (#65)

Documentation

• Better docs rendering (#72)

Performance

• (lbfgs) Jit the L-BFGS update+linesearch step (#19)
• Port _project_divergence_free to jax.numpy (#20)
• (pict, drag_opt) Halve PISO correctors and drag_opt horizon (#57)

Testing

• Better test coverage (#52)

Full diff: v0.0.0...v0.1.0