Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

eval command breaks after AccumulatorRefreshTable merged #5190

Closed
PikaCat-OuO opened this issue Apr 25, 2024 · 0 comments
Closed

eval command breaks after AccumulatorRefreshTable merged #5190

PikaCat-OuO opened this issue Apr 25, 2024 · 0 comments

Comments

@PikaCat-OuO
Copy link
Contributor

PikaCat-OuO commented Apr 25, 2024

Describe the issue

The code does not call caches.clear(networks) when running trace. Result in the wrong result for command eval.

Stockfish/src/evaluate.cpp

Lines 100 to 105 in 49ef4c9

std::string Eval::trace(Position& pos, const Eval::NNUE::Networks& networks) {
auto caches = std::make_unique<Eval::NNUE::AccumulatorCaches>();
if (pos.checkers())
return "Final evaluation: none (in check)";

Expected behavior

output the correct result with command eval

Steps to reproduce

eval

before 49ef4c9:

Stockfish dev-20240422-c2615e30 by the Stockfish developers (see AUTHORS file)
eval
info string NNUE evaluation using nn-ae6a388e4a1a.nnue
info string NNUE evaluation using nn-baff1ede1f90.nnue


 NNUE derived piece values:
+-------+-------+-------+-------+-------+-------+-------+-------+
|   r   |   n   |   b   |   q   |   k   |   b   |   n   |   r   |
| -6.44 | -4.93 | -5.55 | -11.7 |       | -5.48 | -5.05 | -6.64 |
+-------+-------+-------+-------+-------+-------+-------+-------+
|   p   |   p   |   p   |   p   |   p   |   p   |   p   |   p   |
| -0.72 | -1.40 | -1.44 | -1.64 | -1.61 | -1.78 | -1.69 | -0.73 |
+-------+-------+-------+-------+-------+-------+-------+-------+
|       |       |       |       |       |       |       |       |
|       |       |       |       |       |       |       |       |
+-------+-------+-------+-------+-------+-------+-------+-------+
|       |       |       |       |       |       |       |       |
|       |       |       |       |       |       |       |       |
+-------+-------+-------+-------+-------+-------+-------+-------+
|       |       |       |       |       |       |       |       |
|       |       |       |       |       |       |       |       |
+-------+-------+-------+-------+-------+-------+-------+-------+
|       |       |       |       |       |       |       |       |
|       |       |       |       |       |       |       |       |
+-------+-------+-------+-------+-------+-------+-------+-------+
|   P   |   P   |   P   |   P   |   P   |   P   |   P   |   P   |
| +0.47 | +1.04 | +1.11 | +1.31 | +1.29 | +1.41 | +1.26 | +0.36 |
+-------+-------+-------+-------+-------+-------+-------+-------+
|   R   |   N   |   B   |   Q   |   K   |   B   |   N   |   R   |
| +6.39 | +4.45 | +5.17 | +9.56 |       | +4.87 | +4.61 | +6.52 |
+-------+-------+-------+-------+-------+-------+-------+-------+

 NNUE network contributions (White to move)
+------------+------------+------------+------------+
|   Bucket   |  Material  | Positional |   Total    |
|            |   (PSQT)   |  (Layers)  |            |
+------------+------------+------------+------------+
|  0         |     0.00   |  -  1.55   |  -  1.55   |
|  1         |     0.00   |  +  0.68   |  +  0.68   |
|  2         |     0.00   |  +  0.29   |  +  0.29   |
|  3         |     0.00   |  +  0.29   |  +  0.29   |
|  4         |     0.00   |  +  0.16   |  +  0.16   |
|  5         |     0.00   |  +  0.31   |  +  0.31   |
|  6         |     0.00   |  +  0.12   |  +  0.12   |
|  7         |     0.00   |  +  0.12   |  +  0.12   | <-- this bucket is used
+------------+------------+------------+------------+

NNUE evaluation        +0.12 (white side)
Final evaluation       +0.13 (white side) [with scaled NNUE, ...]

after:

Stockfish dev-20240424-49ef4c93 by the Stockfish developers (see AUTHORS file)
eval
info string NNUE evaluation using nn-ae6a388e4a1a.nnue
info string NNUE evaluation using nn-baff1ede1f90.nnue


 NNUE derived piece values:
+-------+-------+-------+-------+-------+-------+-------+-------+
|   r   |   n   |   b   |   q   |   k   |   b   |   n   |   r   |
| -5.04 | -4.21 | -4.66 | -10.3 |       | -5.07 | -4.15 | -4.86 |
+-------+-------+-------+-------+-------+-------+-------+-------+
|   p   |   p   |   p   |   p   |   p   |   p   |   p   |   p   |
| +0.08 | -0.15 | -0.27 | -0.07 | -0.22 | -1.40 | -0.70 | -0.18 |
+-------+-------+-------+-------+-------+-------+-------+-------+
|       |       |       |       |       |       |       |       |
|       |       |       |       |       |       |       |       |
+-------+-------+-------+-------+-------+-------+-------+-------+
|       |       |       |       |       |       |       |       |
|       |       |       |       |       |       |       |       |
+-------+-------+-------+-------+-------+-------+-------+-------+
|       |       |       |       |       |       |       |       |
|       |       |       |       |       |       |       |       |
+-------+-------+-------+-------+-------+-------+-------+-------+
|       |       |       |       |       |       |       |       |
|       |       |       |       |       |       |       |       |
+-------+-------+-------+-------+-------+-------+-------+-------+
|   P   |   P   |   P   |   P   |   P   |   P   |   P   |   P   |
| +0.33 | +0.29 | +0.10 | +0.14 | +0.49 | +0.74 | +0.50 | +0.22 |
+-------+-------+-------+-------+-------+-------+-------+-------+
|   R   |   N   |   B   |   Q   |   K   |   B   |   N   |   R   |
| +4.46 | +4.06 | +4.59 | +6.79 |       | +4.51 | +4.26 | +4.48 |
+-------+-------+-------+-------+-------+-------+-------+-------+

 NNUE network contributions (White to move)
+------------+------------+------------+------------+
|   Bucket   |  Material  | Positional |   Total    |
|            |   (PSQT)   |  (Layers)  |            |
+------------+------------+------------+------------+
|  0         |     0.00   |  -  8.75   |  -  8.75   |
|  1         |     0.00   |  +  0.22   |  +  0.22   |
|  2         |     0.00   |  -  0.93   |  -  0.93   |
|  3         |     0.00   |  -  0.09   |  -  0.09   |
|  4         |     0.00   |  -  2.28   |  -  2.28   |
|  5         |     0.00   |  +  1.20   |  +  1.20   |
|  6         |     0.00   |  -  0.61   |  -  0.61   |
|  7         |     0.00   |  -  0.11   |  -  0.11   | <-- this bucket is used
+------------+------------+------------+------------+

NNUE evaluation        -0.11 (white side)
Final evaluation       -0.12 (white side) [with scaled NNUE, ...]

Anything else?

No response

Operating system

All

Stockfish version

49ef4c9

peregrineshahin added a commit to peregrineshahin/Stockfish that referenced this issue Apr 28, 2024
commit 0fe6428
Author: Stefan Geschwentner <stgeschwentner@gmail.com>
Date:   Sun Apr 28 16:53:47 2024 +0200

    More reduction at cut nodes which are not a former PV node

    But the tt move and first killer are excluded.

    This idea is based on following LMR condition tuning
    https://tests.stockfishchess.org/tests/view/66228bed3fe04ce4cefc0c71 by
    using only the two largest terms P[0] and P[1].

    Passed STC:
    LLR: 2.93 (-2.94,2.94) <0.00,2.00>
    Total: 173248 W: 45091 L: 44565 D: 83592
    Ptnml(0-2): 693, 20534, 43673, 21002, 722
    https://tests.stockfishchess.org/tests/view/6629603b3fe04ce4cefc7d37

    Passed LTC:
    LLR: 2.94 (-2.94,2.94) <0.50,2.50>
    Total: 722394 W: 183231 L: 181487 D: 357676
    Ptnml(0-2): 462, 80650, 197252, 82348, 485
    https://tests.stockfishchess.org/tests/view/662cbe45d46f72253dcff7bf

    closes official-stockfish#5199

    Bench: 1619613

commit 48a3b7c
Author: Stefan Geschwentner <stgeschwentner@gmail.com>
Date:   Sun Apr 28 16:04:28 2024 +0200

    Simplify non-pawn material divisor to a constant

    Passed STC:
    https://tests.stockfishchess.org/tests/view/662942603fe04ce4cefc7aba
    LLR: 2.93 (-2.94,2.94) <-1.75,0.25>
    Total: 272832 W: 70456 L: 70497 D: 131879
    Ptnml(0-2): 1020, 32619, 69154, 32628, 995

    Passed LTC:
    https://tests.stockfishchess.org/tests/view/662dfe3b6115ff6764c829eb
    LLR: 2.94 (-2.94,2.94) <-1.75,0.25>
    Total: 100254 W: 25446 L: 25303 D: 49505
    Ptnml(0-2): 121, 11292, 27166, 11419, 129

    closes official-stockfish#5198

    Bench: 1544645

commit 834e8ff
Author: cj5716 <125858804+cj5716@users.noreply.github.com>
Date:   Sun Apr 28 08:53:28 2024 +0800

    Penalise the TT move in multicut

    Passed STC:
    LLR: 2.99 (-2.94,2.94) <0.00,2.00>
    Total: 185504 W: 48079 L: 47533 D: 89892
    Ptnml(0-2): 716, 21866, 46988, 22520, 662
    https://tests.stockfishchess.org/tests/view/662d9e1d6115ff6764c7f83d

    Passed LTC:
    LLR: 2.94 (-2.94,2.94) <0.50,2.50>
    Total: 75612 W: 19351 L: 18948 D: 37313
    Ptnml(0-2): 46, 8363, 20592, 8752, 53
    https://tests.stockfishchess.org/tests/view/662dc9dc6115ff6764c80fea

    closes official-stockfish#5195

    Bench: 1415435

commit a129c06
Author: mstembera <m_stembera@yahoo.com>
Date:   Sun Apr 28 10:28:25 2024 -0700

    Combine remove and add in update_accumulator_refresh_cache()

    Combine remove and add in update_accumulator_refresh_cache().
    Move remove before add to match other parts of the code.

    STC:
    https://tests.stockfishchess.org/tests/view/662d96dc6115ff6764c7f4ca
    LLR: 2.95 (-2.94,2.94) <0.00,2.00>
    Total: 364032 W: 94421 L: 93624 D: 175987
    Ptnml(0-2): 1261, 41983, 94811, 42620, 1341

    closes official-stockfish#5194

    Bench: 1836777

commit 940a3a7
Author: mstembera <m_stembera@yahoo.com>
Date:   Thu Apr 25 18:20:08 2024 -0700

    Cache small net w/ psqtOnly support

    Caching the small net in the same way as the big net allows them to
    share the same code path and completely removes
    update_accumulator_refresh().

    STC:
    https://tests.stockfishchess.org/tests/view/662bfb5ed46f72253dcfed85
    LLR: 2.94 (-2.94,2.94) <-1.75,0.25>
    Total: 151712 W: 39252 L: 39158 D: 73302
    Ptnml(0-2): 565, 17474, 39683, 17570, 564

    closes official-stockfish#5194

    Bench: 1836777

commit bc45cbc
Author: Joost VandeVondele <Joost.VandeVondele@gmail.com>
Date:   Sat Apr 27 18:09:45 2024 +0200

    Output some basic info about the used networks

    Adds size in memory as well as layer sizes as in

    info string NNUE evaluation using nn-ae6a388e4a1a.nnue (132MiB, (22528, 3072, 15, 32, 1))
    info string NNUE evaluation using nn-baff1ede1f90.nnue (6MiB, (22528, 128, 15, 32, 1))

    For example, the size in MiB is useful to keep the fishtest memory sizes up-to-date,
    the L1-L3 sizes give a useful hint about the architecture used.

    closes official-stockfish#5193

    No functional change

commit 3502c8a
Author: Disservin <disservin.social@gmail.com>
Date:   Thu Apr 25 19:20:57 2024 +0200

    Fix missing initialization of AccumulatorCaches in Eval::trace

    Add a constructor to `AccumulatorCaches` instead of just calling
    `clear(networks)` to prevent similar issues from appearing in the
    future.

    fixes official-stockfish#5190

    closes official-stockfish#5191

    No functional change

commit 886ed90
Author: xoto10 <23479932+xoto10@users.noreply.github.com>
Date:   Sun Apr 28 16:27:40 2024 +0100

    Use less time on recaptures

    Credit for the idea goes to peregrine on discord.

    Passed STC 10+0.1:
    https://tests.stockfishchess.org/tests/view/662652623fe04ce4cefc48cf
    LLR: 2.95 (-2.94,2.94) <0.00,2.00>
    Total: 75712 W: 19793 L: 19423 D: 36496
    Ptnml(0-2): 258, 8487, 20023, 8803, 285

    Passed LTC 60+0.6:
    https://tests.stockfishchess.org/tests/view/6627495e3fe04ce4cefc59b6
    LLR: 2.94 (-2.94,2.94) <0.50,2.50>
    Total: 49788 W: 12743 L: 12404 D: 24641
    Ptnml(0-2): 29, 5141, 14215, 5480, 29

    The code was updated slightly and tested for non-regression against the
    original code at STC:

    LLR: 2.94 (-2.94,2.94) <-1.75,0.25>
    Total: 41952 W: 10912 L: 10698 D: 20342
    Ptnml(0-2): 133, 4825, 10835, 5061, 122
    https://tests.stockfishchess.org/tests/view/662d84f56115ff6764c7e438

    closes official-stockfish#5189

    Bench: 1836777
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant