Skip to content

v2.1.0 — Reproducible results

Latest

Choose a tag to compare

@Jamessfks Jamessfks released this 10 Aug 03:31
· 11 commits to main since this release

v2.0 made SimpleAtom stop returning wrong numbers. v2.1 makes the numbers it returns
verifiable by someone else, months later, without asking you.

Every result carries a reproducibility manifest

A model name is not a reproducible identifier. "MACE-OFF small" tells you nothing about
which weights ran. Each calculation now returns:

  • the SHA-256 of the checkpoint that was actually loaded
  • versions of mace-torch, torch, ase, numpy
  • a hash of the input structure, and of the input file
  • device, precision, and the MD seed
  • the source commit, with a dirty flag — a commit hash from a modified tree names code
    that never ran, so that distinction is recorded rather than implied

The checkpoint is identified by observing the load as it happens, not by re-deriving the
cache path, so it cannot silently drift out of step with upstream. It records how it
identified the file, and where something cannot be resolved it reports null with a reason
instead of guessing. No absolute paths are included — a manifest travels into shared links
and PDF exports, and a home directory does not belong in either.

Cost is ~0.06 s on a 134 MB checkpoint, cached so a replaced file cannot keep serving a
stale digest.

Advisory validation on every run

A validation pass now runs on every calculation and reports alongside the result. It is
advisory and never blocks: its thresholds are plausibility heuristics, and a single-point
on a strained or clashing geometry is a legitimate request — precisely the case where you
want the number. The checks that must block already run earlier, before any model loads.

Two places the interface stated things that were not true

  • The Run Configuration described a run that never happened. Client form state was merged
    underneath the backend's record of what was effective, so a static single-point displayed
    300 K · Δt 1 fs · friction 0.005 /fs · 100 MD steps · NVT. Eight molecular-dynamics and
    optimisation parameters for a calculation that had none — and it propagated into PDF
    exports, CSV exports and every shared link.
  • Reported timing was round-trip latency wearing a fabricated decimal. A run the backend
    timed at 0.132 s displayed "4.0 s": wall clock including model download, rounded to a whole
    second, then rendered to one decimal place. Compute time and round-trip are now separate,
    separately labelled, and displayed precision never exceeds what the source actually has.

3D viewer

  • Click atoms to measure. Two atoms give a bond length in Å, three an angle in degrees
    with the vertex named, four a signed dihedral — and selection order sets the vertex and the
    sign, as it should. Two atoms report Bond length only when a bond was actually
    perceived
    ; otherwise Distance, with the distinction stated rather than glossed.
  • Ambient occlusion had never once rendered. The WebGL2 capability check probed the
    visible canvas, which holds a blit target rather than a WebGL context whenever the viewer
    renders offscreen — so the check failed on every capable machine and the feature disabled
    itself silently on every load.
  • Structures were framed too small, and bulkier representations came out smaller than
    slimmer ones.
    The padding term added the largest radius of any element present to the
    outermost atom's position, so the error grew with the radii — spacefill, with the largest
    spheres, was framed smallest.
  • New settings menu: rotate, hide C–H bonds, clear selection, and PNG export.
  • The canvas is transparent, so the molecule sits on the page rather than in a white box.

Mobile

/calculate overflowed the viewport on a phone. A grid declared its columns only above the
lg breakpoint, so below it both panes fell into a single content-sized track and every card
inherited a width wider than the screen — which real browsers answer by zooming the whole
interface out.

Housekeeping

Removed a dead 417-line results component that still carried the mislabelled energy fixed
everywhere else — a trap for anyone who found it by search. Corrected every remaining
reference to a validator path that has never existed.

Verifying it yourself

export KMP_DUPLICATE_LIB_OK=TRUE
python3 -m unittest discover -s test_scripts -p "test_provenance.py"   # 40 tests
python3 test_scripts/validate_calculation.py --test

The provenance suite was mutation-tested: six deliberate defects were injected —
including a plausible hardcoded SHA-256 and a manifest built after the geometry had already
relaxed — and all six were caught. The strongest test re-finds the checkpoint by filename and
hashes it with plain hashlib, through no project code at all, so a stale or fabricated
digest fails it.

Known limitations in this release. The validator is not present in the hosted container
(its build context cannot reach the test directory), so hosted results report validation as
unavailable rather than running. Viewer framing still sits a few percent under target, and
force-vector arrows are included in the fit, which shrinks the molecule when they are shown.

Full documentation: https://mace-lake.vercel.app/docs