Skip to content

Update default net to nn-c38c3d8d3920.nnue - #4666

Closed
linrock wants to merge 1 commit into
official-stockfish:masterfrom
linrock:nn-c38c3d8d3920.nnue
Closed

linrock wants to merge 1 commit into
official-stockfish:masterfrom
linrock:nn-c38c3d8d3920.nnue

Conversation

@linrock

@linrock linrock commented Jul 6, 2023

Copy link
Copy Markdown
Contributor

This was a later epoch from the same experiment that led to the previous master net.
After training, it was prepared the same way:

  1. greedy permuting L1 weights with Permute master net weights to increase sparsity #4620
  2. leb128 compression with Support for LEB128 compression of feature transformer parameters. nnue-pytorch#251
  3. greedy 2- and 3- cycle permuting with Update default data to nn-a3d1bfca1672.nnue #4640

Local elo at 25k nodes per move (vs. L1-1536 nn-fdc1d0fe6455.nnue):
nn-epoch739.nnue : 20.2 +/- 1.7

Passed STC:
https://tests.stockfishchess.org/tests/view/64a050b33ee09aa549c4e4c8
LLR: 2.95 (-2.94,2.94) <0.00,2.00>
Total: 195552 W: 49977 L: 49430 D: 96145
Ptnml(0-2): 556, 22775, 50607, 23242, 596

Passed LTC:
https://tests.stockfishchess.org/tests/view/64a127bd3ee09aa549c4f60c
LLR: 2.94 (-2.94,2.94) <0.50,2.50>
Total: 235452 W: 60327 L: 59609 D: 115516
Ptnml(0-2): 119, 25173, 66426, 25887, 121

bench 2427629

This was a later epoch from the same experiment that led to the
previous master net. After training, it was prepared the same way:

1. greedy permuting L1 weights with official-stockfish#4620
2. leb128 compression with official-stockfish/nnue-pytorch#251
3. greedy 2- and 3- cycle permuting with official-stockfish#4640

Local elo at 25k nodes per move (vs. L1-1536 nn-fdc1d0fe6455.nnue):
nn-epoch739.nnue : 20.2 +/- 1.7

Passed STC:
https://tests.stockfishchess.org/tests/view/64a050b33ee09aa549c4e4c8
LLR: 2.95 (-2.94,2.94) <0.00,2.00>
Total: 195552 W: 49977 L: 49430 D: 96145
Ptnml(0-2): 556, 22775, 50607, 23242, 596

Passed LTC:
https://tests.stockfishchess.org/tests/view/64a127bd3ee09aa549c4f60c
LLR: 2.94 (-2.94,2.94) <0.50,2.50>
Total: 235452 W: 60327 L: 59609 D: 115516
Ptnml(0-2): 119, 25173, 66426, 25887, 121

bench 2427629
@vondele

vondele commented Jul 6, 2023

Copy link
Copy Markdown
Member

hmm for whatever reason the CI is failing again on MacOS.

I wonder if something is non-standard on our regexp that it fails to match bench 2427629 on MacOS ?

@Sopel97

Sopel97 commented Jul 6, 2023 •

Copy link
Copy Markdown
Member

have you tried skipping the first step and only relying on "greedy 2- and 3- cycle permuting with #4640" ?

official-stockfish/nnue-pytorch#254

@linrock

linrock commented Jul 6, 2023

Copy link
Copy Markdown
Contributor Author

have you tried skipping the first step and only relying on "greedy 2- and 3- cycle permuting with #4640" ?

i have, it completely froze up on a few occasions and i don't know why. meanwhile, i haven't run into issues running the 3 steps sequentially other than it being resource intensive and taking about an hour per net.

i can give official-stockfish/nnue-pytorch#254 a try later.

@ppigazzini

Copy link
Copy Markdown
Contributor

hmm for whatever reason the CI is failing again on MacOS.

I wonder if something is non-standard on our regexp that it fails to match bench 2427629 on MacOS ?

Ubuntu extract the same bench

@vondele

vondele commented Jul 6, 2023

Copy link
Copy Markdown
Member

yeah, we discussing on discord.
It is the order of commits as extracted with the loop that is unexpected

@ppigazzini

Copy link
Copy Markdown
Contributor

In local test with this branch the script works fine:
bash --noprofile --norc -eo pipefail ../bench.sh
benchref=2427629
From commit: 9f4cd3b
Reference bench: 2427629

@vondele

vondele commented Jul 6, 2023

Copy link
Copy Markdown
Member

it might be related to how this branch is merged in master during the CI ?

@vondele

vondele commented Jul 6, 2023

Copy link
Copy Markdown
Member

look at the additional output in https://github.com/official-stockfish/Stockfish/actions/runs/5478194778/jobs/9978344031
(under Extract bench .... step)

@vondele vondele added the to be merged Will be merged shortly label Jul 6, 2023
@vondele vondele closed this in e699fee Jul 6, 2023
ppigazzini pushed a commit to ppigazzini/zfish that referenced this pull request Jul 12, 2026
This was a later epoch from the same experiment that led to the
previous master net. After training, it was prepared the same way:

1. greedy permuting L1 weights with official-stockfish/Stockfish#4620
2. leb128 compression with official-stockfish/nnue-pytorch#251
3. greedy 2- and 3- cycle permuting with official-stockfish/Stockfish#4640

Local elo at 25k nodes per move (vs. L1-1536 nn-fdc1d0fe6455.nnue):
nn-epoch739.nnue : 20.2 +/- 1.7

Passed STC:
https://tests.stockfishchess.org/tests/view/64a050b33ee09aa549c4e4c8
LLR: 2.95 (-2.94,2.94) <0.00,2.00>
Total: 195552 W: 49977 L: 49430 D: 96145
Ptnml(0-2): 556, 22775, 50607, 23242, 596

Passed LTC:
https://tests.stockfishchess.org/tests/view/64a127bd3ee09aa549c4f60c
LLR: 2.94 (-2.94,2.94) <0.50,2.50>
Total: 235452 W: 60327 L: 59609 D: 115516
Ptnml(0-2): 119, 25173, 66426, 25887, 121

closes official-stockfish/Stockfish#4666

bench 2427629
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

to be merged Will be merged shortly

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants