Skip to content

Commit

Permalink
Update smallnet to nn-baff1ede1f90.nnue with wider eval range
Browse files Browse the repository at this point in the history
Created by training an L1-128 net from scratch with a wider range of
evals in the training data and wld-fen-skipping disabled during
training. The differences in this training data compared to the first
dual nnue PR are:

- removal of all positions with 3 pieces
- when piece count >= 16, keep positions with simple eval above 750
- when piece count < 16, remove positions with simple eval above 3000

The asymmetric data filtering was meant to flatten the training data
piece count distribution, which was previously heavily skewed towards
positions with low piece counts.

Additionally, the simple eval range where the smallnet is used was
widened to cover more positions previously evaluated by the big net and
simple eval.

```yaml
experiment-name: 128--S1-hse-S7-v4-S3-v1-no-wld-skip

training-dataset:
  - /data/hse/S3/leela96-filt-v2.min.high-simple-eval-1k.binpack
  - /data/hse/S3/dfrc99-16tb7p-eval-filt-v2.min.high-simple-eval-1k.binpack
  - /data/hse/S3/test80-apr2022-16tb7p.min.high-simple-eval-1k.binpack

  - /data/hse/S7/test60-2020-2tb7p.v6-3072.high-simple-eval-v4.binpack
  - /data/hse/S7/test60-novdec2021-12tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack

  - /data/hse/S7/test77-nov2021-2tb7p.v6-3072.min.high-simple-eval-v4.binpack
  - /data/hse/S7/test77-dec2021-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack
  - /data/hse/S7/test77-jan2022-2tb7p.high-simple-eval-v4.binpack

  - /data/hse/S7/test78-jantomay2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack
  - /data/hse/S7/test78-juntosep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack

  - /data/hse/S7/test79-apr2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack
  - /data/hse/S7/test79-may2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack

  - /data/hse/S7/test80-may2022-16tb7p.high-simple-eval-v4.binpack
  - /data/hse/S7/test80-jun2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack
  - /data/hse/S7/test80-jul2022-16tb7p.v6-dd.min.high-simple-eval-v4.binpack
  - /data/hse/S7/test80-aug2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack
  - /data/hse/S7/test80-sep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack
  - /data/hse/S7/test80-oct2022-16tb7p.v6-dd.high-simple-eval-v4.binpack
  - /data/hse/S7/test80-nov2022-16tb7p-v6-dd.min.high-simple-eval-v4.binpack

  - /data/hse/S7/test80-jan2023-3of3-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack
  - /data/hse/S7/test80-feb2023-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack
  - /data/hse/S7/test80-mar2023-2tb7p.v6-sk16.min.high-simple-eval-v4.binpack
  - /data/hse/S7/test80-apr2023-2tb7p-filter-v6-sk16.min.high-simple-eval-v4.binpack
  - /data/hse/S7/test80-may2023-2tb7p.v6.min.high-simple-eval-v4.binpack
  - /data/hse/S7/test80-jun2023-2tb7p.v6-3072.min.high-simple-eval-v4.binpack
  - /data/hse/S7/test80-jul2023-2tb7p.v6-3072.min.high-simple-eval-v4.binpack
  - /data/hse/S7/test80-aug2023-2tb7p.v6.min.high-simple-eval-v4.binpack
  - /data/hse/S7/test80-sep2023-2tb7p.high-simple-eval-v4.binpack
  - /data/hse/S7/test80-oct2023-2tb7p.high-simple-eval-v4.binpack

wld-fen-skipping: False
start-from-engine-test-net: False

nnue-pytorch-branch: linrock/nnue-pytorch/L1-128
engine-test-branch: linrock/Stockfish/L1-128-nolazy
engine-base-branch: linrock/Stockfish/L1-128

num-epochs: 500
start-lambda: 1.0
end-lambda: 1.0
```

Experiment yaml configs converted to easy_train.sh commands with:
https://github.com/linrock/nnue-tools/blob/4339954/yaml_easy_train.py

Binpacks interleaved at training time with:
official-stockfish/nnue-pytorch#259

FT weights permuted with 10k positions from fishpack32.binpack with:
official-stockfish/nnue-pytorch#254

Data filtered for high simple eval positions (v4) with:
https://github.com/linrock/Stockfish/blob/b9c8440/src/tools/transform.cpp#L640-L675

Training data can be found at:
https://robotmoon.com/nnue-training-data/

Local elo at 25k nodes per move of
L1-128 smallnet (nnue-only eval) vs. L1-128 trained on standard S1 data:
nn-epoch319.nnue : -241.7 +/- 3.2

Passed STC vs. 36db936:
https://tests.stockfishchess.org/tests/view/6576b3484d789acf40aabbfe
LLR: 2.94 (-2.94,2.94) <0.00,2.00>
Total: 21920 W: 5680 L: 5381 D: 10859
Ptnml(0-2): 82, 2488, 5520, 2789, 81

Passed LTC vs. DualNNUE #4915:
https://tests.stockfishchess.org/tests/view/65775c034d789acf40aac7e3
LLR: 2.95 (-2.94,2.94) <0.50,2.50>
Total: 147606 W: 36619 L: 36063 D: 74924
Ptnml(0-2): 98, 16591, 39891, 17103, 120

closes #4919

Bench: 1438336
  • Loading branch information
linrock authored and Disservin committed Jan 7, 2024
1 parent 584d9ef commit f09adaa
Show file tree
Hide file tree
Showing 3 changed files with 4 additions and 4 deletions.
4 changes: 2 additions & 2 deletions src/evaluate.cpp
Expand Up @@ -185,12 +185,12 @@ Value Eval::evaluate(const Position& pos) {
int shuffling = pos.rule50_count();
int simpleEval = simple_eval(pos, stm);

bool lazy = std::abs(simpleEval) > 2300;
bool lazy = std::abs(simpleEval) > 2550;
if (lazy)
v = simpleEval;
else
{
bool smallNet = std::abs(simpleEval) > 1100;
bool smallNet = std::abs(simpleEval) > 1050;

int nnueComplexity;

Expand Down
2 changes: 1 addition & 1 deletion src/evaluate.h
Expand Up @@ -40,7 +40,7 @@ extern std::string currentEvalFileName[2];
// for the build process (profile-build and fishtest) to work. Do not change the
// name of the macro, as it is used in the Makefile.
#define EvalFileDefaultNameBig "nn-b1e55edbea57.nnue"
#define EvalFileDefaultNameSmall "nn-c01dc0ffeede.nnue"
#define EvalFileDefaultNameSmall "nn-baff1ede1f90.nnue"

namespace NNUE {

Expand Down
2 changes: 1 addition & 1 deletion src/nnue/evaluate_nnue.cpp
Expand Up @@ -178,7 +178,7 @@ static bool write_parameters(std::ostream& stream, NetSize netSize) {
void hint_common_parent_position(const Position& pos) {

int simpleEval = simple_eval(pos, pos.side_to_move());
if (abs(simpleEval) > 1100)
if (abs(simpleEval) > 1050)
featureTransformerSmall->hint_common_access(pos);
else
featureTransformerBig->hint_common_access(pos);
Expand Down

0 comments on commit f09adaa

Please sign in to comment.