Performance en

中文 | English

Performance

Methodology (read this before the numbers)

• Micro-benchmarks: PerfReportTest runs 5 rounds of independent fixtures per JVM (each round truly cold, no shared caches), pools all samples, and reports median and minimum; serious conclusions re-run on 2+ JVMs and take the minimum (hotMin is most robust against scheduler noise);
• Capability face: the lightning suite runs 36 scenarios × 3 stock modes (one execution each; a family's first scenario includes first compilation + JIT warm-up — eliminated in production by the pre-compilation mixin, a measurement artifact) plus the 38-case boundary suite;
• Informational, always green: performance tests assert correctness invariants only; timings print as [perf] lines and are never asserted (CI machines vary in speed);
• Per-release benchmarks: every Release, CI builds automatically and writes the latest numbers into the block below (GitHub runner environment — for cross-release trend comparison only).

Latest release benchmarks

v1.1.0 · 2026-09-12 · GitHub Actions runner (ubuntu-latest, JDK 25) — values are for cross-release trend comparison only

Tests: 209 (OK)

Suite	Result
Lightning benchmark	39/39 SUPPORTED, falsePositive=0, engineError=0, timeout=0, total 118.4 ms
Boundary benchmark	37/37 ok (feasible=36)

Micro-benchmarks

[perf] case=fib24x1e9empty rounds=5 coldMedian=0.912ms coldMin=0.687ms hotMedian=0.475ms hotMin=0.233ms (150 hot samples)
[perf] case=conversionRing cold=1.298ms hotMedian=0.273ms hotMin=0.243ms (25 hot samples)
[perf] case=solverFib12Minimum cold=6.577ms hotMedian=1.946ms hotMin=1.078ms (15 hot samples)
[perf] case=fib32x1e9 rounds=5 coldMedian=1.053ms coldMin=0.873ms hotMedian=0.440ms hotMin=0.288ms (100 hot samples)
[perf] case=deadCycleGuard rounds=5 ringMedian=5.12us ringMin=2.85us chain24Median=12.70us chain24Min=6.97us (10000 samples each)

Reference data (2026-09-17, manual runs, Windows build machine / JDK 25, feature/trace-replay @ 1a96dab)

The lightning suite (36 cases) after the faithful CPU replica switched to per-pass batched firing (verdict-identical, see the 1.2.0 changelog):

Suite total 173–194 ms (three forced reruns; every case includes the faithful-runtime
gate, task-order construction and a cold compile of all its patterns)
Adjudication: 36/36 clean — supported=36, falsePositive=0, engineError=0, timeout=0 (initial run: 34 + 2 host-variant-route FALSE_POSITIVEs, since fixed by translating the fixture into its real 1.12 shape — a wildcard-durability crafting pattern; see Architecture §3)
Compare: v1.1.0 same-machine baseline 195.6 ms / 39 cases (no gate at all); 3303–3455 ms
after the gate landed but before batching

Highlights:

• The three single-dag/fibonacci cases (depth 32, 3,524,577 planned crafts each): 1036–1119 ms before batching (the gate's per-craft replica was 92%+ of it), 8.1 / 11.9 / 8.2 ms after — the replica dropped from O(total crafts) to O(tasks x passes), replaying 3.5M crafts in ~0.1 ms;
• The other 33 cases are within ±2 ms (the gate/self-check's fixed price on small plans; several are faster than v1.1.0);
• The first-executed case (single-dag/dispersed/missing, ~56 ms) is one-off JVM class-load warmup;
• Micro-benchmarks (VM calc path, never touches the replica) sit in the historical band: hotMedian fib24x1e9empty 0.75 ms, solverFib12 full pipeline 3.5 ms;
• The two ENGINE_TIMEOUT / NON_COOPERATIVE statuses are gone — all 36 cases now adjudicate cleanly.

Reference data (2026-09-11, manual runs, Windows build machine / JDK 25, master @ 0b06429~0beec0e)

Tests 190+ failed 0 | Lightning 39/39 SUPPORTED, 118.0–132.5 ms total | Boundary 37/37, 64 ms
1.21 baseline (2026-08-09): 38 SUPPORTED + 1 FALSE_POSITIVE (closed by this port)

Micro-benchmarks (two JVM runs, slash-separated):

Case	coldMedian	hotMedian	hotMin	Notes
fib32 × 10^9	1.6 / 1.7 ms	0.59 / 0.59 ms	0.46 / 0.33 ms	O(patterns) demand propagation + bundle replay
fib24 × 10^9, empty stock	1.4 / 1.4 ms	0.63 / 0.60 ms	0.48 / 0.40 ms	craftable intermediates never report missing
conversionRing	0.77 / 1.27 ms	0.39 / 0.31 ms	0.30 / 0.21 ms	conversion ring + stock awareness
solverFib12Minimum, full pipeline	3.4 / 3.8 ms	3.6 / 3.3 ms	3.1 / 2.7 ms	greedy + 2^10 enumeration + refinement + confirmation
Dead-cycle guard, single check	—	ring 9.7 / 8.8 μs; chain 23.6 / 22.5 μs	ring 4.8 / 4.6 μs	zero cost in single-candidate cases

A handful of the 36 scenarios exceed 5 ms: the first case carries JVM/class-load warm-up (~56–62 ms, one-off), the rest are per-scenario cold compilation of their pattern families — all eliminated in production by the pre-compilation mixin.

Optimization history (the post-port performance campaign)

Round	Change	Effect
Baseline	1.21 → 1.12 port complete, all capabilities aligned	feature-equivalent, perf pending
P0–P3	Solver fast path (long[] cascades + stock pre-read), plan memoization + version gating, resolverCache across passes	solver hot 8.9 → 4.3 ms; repeated requests milliseconds → microseconds
Confirmation guard + single-candidate early-out	Skip confirmation replay when the model shows no improvement; skip graph analysis on uncontended keys	Lightning total 196.1 → 132.5 ms (≈ -32%); solver cold 8.0 → 3.4 ms
Two-layer fuzzy-family cache	Sandbox family lists keyed by item + VM composed families keyed	fuzzy/missing 17.7 → 4.75 ms (-73%); lightning total → 118 ms
GAP-2 (T2.5)	Craftable-substitute resolution	correctness fix, no regression

Conclusion: the engine's regular paths and the solver are near the architecture's floor — engine-side optimization is closed.

Measurement pitfalls (notes to self)

1. BenchSimulationState simulated inserts persist across executes — use a fresh sim per request in multi-request tests;
2. Check test chains for orphan branches (once m5/m6 sat off-chain, the leaf was never consumed, and the "0 missing" assertion passed vacuously);
3. When comparing fuzzy timings, mind the suite's family cold-start scope (first scenario includes first compilation + JIT warm-up).