Skip to content

Parity benchmark: reproducible wasm→synth-AOT vs native-C→LLVM comparison — counter the 'wasm is ~40% slower' claim with measured cycles+bytes #735

Description

@avrabe

Motivation

A recurring industry claim: "WebAssembly is ~40% slower than native." That number comes from runtime/interpreter deployments (wasm3, interpreted sandboxes, JIT warm-up). It does not describe synth's model: AOT wasm→bare-metal native with no runtime — the output is Thumb-2/RV32 machine code, and the only question is compilation quality, which is measurable.

We have partial evidence both directions and should publish one honest, reproducible artifact instead of counter-slogans.

What we can already show (in-tree, CI-gated)

  • Semantics: bit-exact vs wasmtime across the differential suite (incl. the full f32 set, 156/156 + 84/84).
  • Structural wins native can't match: proof-carrying specialization (Beyond parity: proof-carrying specialization + optimal-alloc to BEAT LLVM (0.7× target) #494, SYNTH_FACT_SPEC) consumes loom wsc.facts value-range premises and ordeal-certifies guard/clamp elisions — gust_mix 50 B → 14 B on the branchless-select clamp, approaching LLVM's 12 B floor from a proven direction (LLVM cannot elide what it cannot prove; we carry the proof).
  • Allocator at optimum on key kernels: flat_flight at its Belady frame-traffic-0 optimum since v0.24.0.

What we must measure, not assert

Deliverable

scripts/repro/parity_benchmark/ + artifacts/parity-benchmark.md:

  1. Pinned toolchains, N kernels (gust_mix, gust_poll, flat_flight, a falcon f32 kernel), both compile paths, cycle+byte table.
  2. Falsification instructions (one command per row).
  3. Honest headline: where AOT-wasm ≥ native today (certified-elision shapes), where it isn't yet (general regalloc), and the trend per release.
  4. CI-pinned so the numbers can't drift from prose (claims.yaml).

Refs: #494 (beat-LLVM), #390 (size attribution), #209 (perf epic), #242 (North Star). gale owns the silicon rows.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions