+198 ±7 Elo over v1.0.52. A single defect, hiding behind a second one that cancelled it out.
What was wrong
evaluate() indexed the L1 weights with the side-to-move and not-side-to-move halves the wrong way round. Swapping the two perspectives of a dual-perspective network negates its output — and does so symmetrically, so a position and its exact colour mirror still agreed with each other. Nothing structural could catch it: the net loaded, the accumulator matched a from-scratch recompute bit-for-bit, and the evaluation was internally consistent. It was simply the wrong sign.
The reason the engine nonetheless played the right way round is that the training corpus had its game-result label inverted relative to white. Two errors, cancelling exactly, since v1.0.40.
That cancellation was found by feeding the engine's own self-play data through the training pipeline. Correctly labelled data trained a net that played to lose.
Why the cancellation still cost 198 Elo
bullet blends its training target 75% game result / 25% search score. Only the result half was inverted; the score half was correct throughout. So a quarter of every training target was pulling against the other three quarters, for every net trained since v1.0.40.
Correcting both ends together is what this release buys. The corpus was measured rather than assumed — across four sampled files the result field agreed with its own score 0.0–0.6% of the time, while the score itself matched white's material lead 94.5%.
The new network
512,363,260 Stockfish depth-9 positions with the result field corrected (label coherence 0.6% → 99.4%), retrained for 600 superbatches. Architecture, dataset and hyperparameters are otherwise identical to v1.0.52 — 256×2 with 8 output buckets — so the labelling fix is the only variable, and the measured gain is attributable to it alone.
Strength
| conditions | games | result | Elo |
|---|---|---|---|
| 10s + 0.1s | 3000 | +1887 −344 =769 (75.7%) | +198 ±7 |
| 30–45s + 0.5–1.0s | 400 | +212 −32 =156 (72.5%) | +168 ±19 |
The gain is smaller at the longer control, which is the usual direction for an evaluation improvement — more search time partially compensates for a weaker eval. Both are far outside noise. NPS was even between the two engines throughout both runs, so this is not a speed effect.
The fixed-depth bench signature moves from 9,771,005 to 3,310,543 nodes. That is a real change and an expected one — the evaluation is different, so the search tree is different. It is not a speed improvement and should not be read as one.
For context, this single fix is worth more than every other lever measured on this engine's evaluation combined: output buckets came in at +5.3 ±10.4, doubling network width at −22.5 ±10.3, and halving the training data at −24.1 ±10.2.
Known limitations
Every previous network file is incompatible with this build. They were all trained against the inverted label and evaluate backwards with the corrected indexing. Swapping the embedded net back to any of them produces an engine that plays to lose while passing every structural check — the sign checks in examples/check_net are the only thing that catches it. They are retained in the repository purely as a record.
The evaluation scale has shifted. The new net returns roughly twice the centipawn magnitude of its predecessor for the same material. The search margins — razoring, futility, SEE thresholds — are all tuned in centipawns against the old scale, and have not been retuned. They are therefore effectively tighter than intended. This release is stronger despite that, not because of it; retuning them is outstanding work and may be worth further Elo.
Two architecture results are now suspect. The output-bucket and 256→512 width experiments were both measured on networks whose training objective was fighting itself. Their conclusions — that the architecture is saturated — cannot be trusted and are worth revisiting.
Also in this release
Self-play data generation is now resumable. datagen writes zstd-compressed shards every 500 games, each committed by an atomic rename, so a shard on disk is proof its games are complete and durable — an interrupted multi-day run resumes from the first game not already written instead of starting over. Output compresses about 4.4×.
Game N now always produces the same game, whatever the thread count, which is what makes resuming sound. Getting there fixed two defects: an opening filter that consumed a game index without producing a game (a run of 1000 yielded 904, and the resume arithmetic no longer matched the data on disk), and a reproducibility hole where the countermove table — alone among the move-ordering tables — was never reset between games, so a game's result depended on which other games its worker happened to be handed. A run that does not end on a shard boundary now records that shard's game count in its filename, so extending a dataset later adds exactly the games requested rather than silently generating fewer.
None of this affects play. The countermove table is deliberately still not cleared by ucinewgame: clearing it costs 12% more nodes on bench, and a 400-game match at 30–45s put the change at −11 ±17 Elo, which does not justify paying that.
examples/mirror_eval, a colour-mirror consistency check for the evaluation. It needs no reference implementation: a position and its exact mirror must score identically. It passes here, and it is what ruled the engine's perspective handling out during the investigation — though note it cannot detect a global sign flip, which is precisely how the original defect survived.