peras-v4

Network for Peras v3.4.0 and later.

Architecture (768x16hm + threats + pawn pairs -> 1024)x2 -> (32 -> 32 -> 1) x 8. Each side's accumulator has its two halves clipped and multiplied together before anything else sees them, and the single output layer is replaced by three layers per material bucket, the second of which is fed both its inputs and their squares. King buckets go from 10 to 16.

Piece-square weights i16 and threat/pawn-pair weights i8, both quantised by 255; the first affine layer is i8 quantised by 128, and the two layers after it i32.

Trained from scratch over 1200 superbatches on about 30 billion positions, a third more data than peras-v3 had: 100 superbatches ramping the learning rate up and back down, then 1100 annealing it from 1e-3 to 1e-6 while the training target shifts from the search score towards the game result.

Worth +6.4 +/- 10.1 Elo over peras-v3 at 10s+0.1s on one thread, and +31.4 +/- 30.8 at a fixed 40000 nodes.

Score of peras-v4 vs peras-v3: 460 - 423 - 1117  [0.509] 2000
Elo: +6.4 +/- 10.1  (LOS 89.3%), 0 time losses
10s+0.1s, one thread, 64 MB hash