Repository navigation
Neural-inference cache (declined)
An upper-bound measurement of neural-inference repetition in chess self-play, used to decline
building an inference cache. Preserved here because it is not merged to master.
Decision
A neural-inference cache does not deserve implementation priority for this workload.
The measurement
The decision-relevant experiment reproduced the actual mixed search policy — 800-search full requests
at a deterministic 25% Bernoulli rate, 150-search fast requests otherwise, 60% retained root visits
before each full search, matching SelfPlayWorker.run_batch. 512 distinct random legal starting
positions over six measured moves, two matched seeds with reversed run order, production search
constants and root noise.
| Parallel searches | Inference positions | Potentially avoidable | Upper-bound repeat rate | Same-batch repeats | Searches/s |
|---|---|---|---|---|---|
| 1 | 1,824,486 | 64,741 | 3.5485% | 0 | 29,420.6 |
| 2 | 1,826,618 | 64,470 | 3.5295% | 1 | 38,222.0 (+29.9%) |
~3.5% gross opportunity, before paying any cache cost — finite capacity, lookup,
synchronization, output storage and transfer. Same-batch repetition was effectively zero: all repeats
were prior-batch. The result is insensitive to parallel-search count.
Why the earlier, more exciting numbers were thrown out
An earlier harness reported 37.6% repetition at high budget and 98.5% at low budget. Those are
rejected methodology evidence and must not drive the decision: that harness used up to 64
parallel searches, cycled a fixed 50-opening suite, used evaluation-like search settings, and
repeatedly restarted completed games from the same openings. It did not represent project self-play.
The correction is the main value of this branch — an appealing result that did not survive being
measured against the real workload.
Incidental finding that did survive
parallel_searches: 2 improved aggregate search throughput +29.9% over 1 in two matched seeds
with reversed run order (+33.2% and +26.6% individually), with identical instrumentation in both
arms. It created no meaningful cache opportunity, but it is a throughput result in its own right.
Full data, environment and checksums:
documentation/benchmarks/inference-cache-hit-rate-20260818/