Complete wasm-embedded-inference spike with wazero integration

[jeduden]

Summary

This PR completes the WASM-embedded-inference spike (plan 65), implementing a proof-of-concept that embeds a WebAssembly classifier artifact using go:embed and hosts it with the wazero runtime. The spike evaluates whether WASM can provide a viable inference path for mdsmith with no runtime dynamic library dependencies.

Key Changes

• WASM guest implementation (wasm/main.go): A minimal WASI reactor module that reuses the plan-64 linear classifier package verbatim, exposing three wasmexport functions (alloc, free, classify) for host-guest communication via linear memory.

• Host harness (main.go): A comprehensive benchmark and determinism-verification tool that:

  • Loads the embedded wasm artifact via go:embed
  • Instantiates it with wazero (v1.11.0, using the default wazevo optimizing compiler)
  • Measures cold-start compile time, per-call latency (avg/p50/p95/max), memory overhead, and determinism across process restarts
  • Compares against the MDS029 heuristic, pure-Go classifier (plan 64), and yzma embedded baselines

• Build infrastructure (run.sh): Automated script that:

  • Compiles the wasm guest with GOOS=wasip1 GOARCH=wasm -buildmode=c-shared
  • Builds the host harness and runs determinism checks across five process restarts
  • Measures binary-size delta by building mdsmith with and without the spike_wasm_classifier tag
  • Captures all metrics to bench.txt and size.txt

• Size-measurement hooks (internal/rules/concisenessscoring/wasmclassifier/embed.go, cmd/mdsmith/spike_wasm_classifier.go): Build-tag-gated stubs that force-link the embedded wasm artifact and wazero runtime so binary-size impact can be measured in isolation.

• Comprehensive documentation (updated README.md): Detailed findings including:

  • Determinism confirmation (5 in-process + 5 cross-process runs, all identical digest)
  • Latency metrics: ~2,022 us avg, 2,464 us p95 (vs. 3.38 us for pure-Go)
  • Memory: 109 MB RSS after bench (vs. 7.8 MB for pure-Go)
  • Binary cost: 4.17 MB delta (3.99 MB wasm artifact + 181 KB wazero/WASI)
  • Recommendation: reject this path — the pure-Go classifier already delivers deterministic output with 480 B binary cost and 3 us latency; wasm adds no capability while incurring 600x latency penalty and 14x memory overhead

• Plan status update (plan/65_spike-wasm-embedded-inference.md): Marked complete with all acceptance criteria met.

Notable Implementation Details

• Module loading strategy: One module instance is reused for all calls; re-instantiating per call would incur the ~1.7 s wazero compile cost repeatedly. The host calls _initialize explicitly after instantiation (reactor-module semantics).

• Memory safety: Host-guest communication uses linear memory with explicit alloc/free calls. The guest returns a packed int64 (outPtr<<32)|outLen pointing to a static 4 KB output buffer containing JSON results.

• Determinism: Outputs are byte-for-byte identical across in-process repeats and process restarts, with SHA256 digest pinning recommended for integrity checks.

• Fallback boundaries: Documented recommended guards (compile-time build tag, runtime instantiation with fallback to heuristic on error, per-call timeout wrapper, verbose diagnostics).

• Artifact update workflow: Safe path defined with SHA256 pinning, version tracking, and determinism validation before shipping.

Rationale for Rejection

The spike conclusively demonstrates that while wasm provides deterministic, sandboxed execution with no external dynamic

https://claude.ai/code/session_014iGcShL7oVcZVoYk4JSPpq

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.03%. Comparing base (143275b) to head (96af29e).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #164   +/-   ##
=======================================
  Coverage   88.03%   88.03%           
=======================================
  Files         110      110           
  Lines       14106    14106           
=======================================
  Hits        12418    12418           
  Misses       1228     1228           
  Partials      460      460           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

Completes plan 65’s WASM-embedded inference spike by adding a WASI/wasip1 guest module, a wazero-based host benchmark harness, build automation, and documentation of performance/determinism/size findings (with a recommendation to reject WASM for this use case).

Changes:

• Added a WASM guest (wasip1) exposing alloc, free, and classify, plus a host harness that embeds and executes the artifact via wazero for determinism/latency/memory measurement.
• Added a build+benchmark script and build-tag-gated linker hooks to measure mdsmith binary-size deltas when force-linking wazero + the embedded wasm artifact.
• Updated spike documentation and plan status to record results and mark the plan complete.

Reviewed changes

Copilot reviewed 9 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
plan/65_spike-wasm-embedded-inference.md	Marks tasks/acceptance criteria complete and links to spike findings.
internal/rules/concisenessscoring/wasmclassifier/embed.go	Build-tag-gated embed + force-link package for artifact + wazero/WASI size measurement.
cmd/mdsmith/spike_wasm_classifier.go	Build-tag-gated init hook to keep embedded wasm reachable for binary-size delta measurement.
docs/research/conciseness/spikes/wasm-embedded-inference/wasm/main.go	WASM guest implementation exporting alloc/free/classify over linear memory.
docs/research/conciseness/spikes/wasm-embedded-inference/main.go	Host harness that embeds the wasm artifact, instantiates via wazero, and benchmarks/digests outputs.
docs/research/conciseness/spikes/wasm-embedded-inference/run.sh	Automation to build the guest, run determinism/bench, and measure mdsmith size delta.
docs/research/conciseness/spikes/wasm-embedded-inference/README.md	Documents methodology, metrics, and recommendation to reject WASM path.
go.mod / go.sum	Adds wazero dependency for the spike artifacts/size measurement path.
PLAN.md	Updates plan table to show plan 65 complete.
.gitignore	Ignores generated wasm artifact copies and the .tmp/ output directory used by the spike.

[jeduden]

🟢 Merge Queue — picked up

This PR is in the queue and will be batched with other queue-labelled PRs.

Next: No action needed — you'll get another comment when CI starts on the batch. View merge queue run.

[jeduden]

🔵 Merge Queue — CI running

Merged into batch branch merge-queue/batch-164-1776885084. View CI run.

Next: No action needed — you'll be notified when CI completes.

[jeduden]

✅ Merge Queue — merged

This PR landed on main via commit d983848. CI run that validated the merge.

Next: Done — nothing more to do here.