triple-activation-2

new additional activation top on the dual activation!

with the asymmetric clamp, we aim to enable richer data transmission by allowing negative values

triple-activation

new additional activation top on the dual activation!

with the asymmetric clamp, we aim to enable richer data transmission by allowing negative values

test net (beans ti)

threans net for debugging

second threat inputs network

additional 15m games from the very first threat inputs network, so at total like:

15m games from very first multilayer net
15m games from second multilayer net
15m games from fourth multilayer net
15m games from fifth multilayer net
15m games from very first threat inputs network

threat-inputs-3

this time trained on 60m games!

threat-inputs-2

another attempt but this time 2x data, 30m games soo 2.4B positions

threat-inputs

384hl + multilayer + Threat Inputs net attempt! Trained with the second multilayer data around 1.2B positions

second pawn pawn inputs network

Second pawn pawn inputs network! Trained on:

15m games from very first multilayer net
15m games from second multilayer net
15m games from fourth multilayer net
15m games from fifth multilayer net
15m games from very first threat inputs network

pawn-pawn-inputs

Threat Inputs + Pawn Pawn Inputs! L1 = 384 network.

multilayer-fine-tune

an extra stage that doing low lr, 100 super batches and constant 0.7 wdl