Skip to content

Speed up interpreter hot paths - #31

Merged
IRus merged 7 commits into
mainfrom
perf/interpreter-hot-path
Aug 17, 2026
Merged

Speed up interpreter hot paths#31
IRus merged 7 commits into
mainfrom
perf/interpreter-hot-path

Conversation

@IRus

@IRus IRus commented Aug 17, 2026

Copy link
Copy Markdown
Member

Summary

  • eliminate boxed numeric Value allocations from interpreter result pushes by using the typed runtime-stack writers
  • cache packed opcode/immediate words for dense linear hot paths, with an identity-based MRU lookup
  • bound the new packed representation to 512 KiB per body, 2 MiB and 4,096 arrays per Store; oversized, sparse, or over-budget bodies fall back to the original Instr objects
  • add focused JVM/Native benchmarks and invariant tests for signed immediates, raw floating-point bits, cache identity, reuse, density, budget exhaustion, and both checkpoint fallback loops

Why

The interpreter's hot path paid for two avoidable costs:

  1. numeric results were wrapped in Value.I32/I64/F32/F64 objects before being unpacked into the runtime stack
  2. linear dispatch repeatedly decoded polymorphic instruction objects even though validated function bodies are immutable

The typed stack path removes the first source of allocation. The packed representation removes repeated object decoding while retaining a bounded-memory fallback for cold, sparse, and very large bodies.

Performance

Focused before/after runs used the same benchmark sources and JDK 17.

Benchmark Before After Change
JVM mixed numeric 1.465 ms/op 0.927 ms/op -36.7%
JVM allocation, mixed numeric 427,400 B/op 1,186 B/op -99.72%
JVM local transfer, checkpoints compiled out 5.652 ms/op 4.313 ms/op -23.7%
JVM local transfer, checkpoints enabled 5.825 ms/op 4.468 ms/op -23.3%
JVM first invoke 3.070 us/op 3.307 us/op +7.7%
macOS ARM64 Native mixed numeric 4.553 ms/op 2.538 ms/op -44.3%
macOS ARM64 Native local transfer, compiled out 17.371 ms/op 13.327 ms/op -23.3%
macOS ARM64 Native local transfer, enabled 17.589 ms/op 13.779 ms/op -21.7%
macOS ARM64 Native first invoke 0.01416 ms/op 0.01254 ms/op -11.4%

The JVM first-invoke tradeoff is a one-time 0.237 us and 2.36 KiB per packed body. The iOS paired stripped-binary gate also improved from 690,168 to 673,784 incremental bytes, below the 1,500,000-byte limit.

Validation

  • ./gradlew --no-watch-fs --rerun-tasks :core:jvmTest :core:macosArm64Test
  • snapshot, WASI, bindgen-runtime, and TCK JVM tests
  • snapshot, WASI, and TCK macOS ARM64 tests
  • all TCK/WABT/WASI exclusion validators
  • :samples-cli:run
  • :benchmarks:performanceGateToolTest :benchmarks:jvmSmokeBenchmark
  • scripts/verify-ios-core-footprint.sh
  • focused JMH and Kotlin/Native before/after runs

External spec and WASI corpora were not configured locally and remain covered by CI.

@IRus
IRus marked this pull request as ready for review August 17, 2026 07:24
@IRus
IRus merged commit 8f3d079 into main Aug 17, 2026
14 of 15 checks passed
@IRus
IRus deleted the perf/interpreter-hot-path branch August 17, 2026 07:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant