Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sf nnue update #45

Merged
merged 19 commits into from Jul 11, 2020
Merged

Sf nnue update #45

merged 19 commits into from Jul 11, 2020

Commits on Jun 28, 2020

  1. Fix fragile code to use proper random 64 bit keys.

    This fixes an old issue where we want to make a position unique but only
    change a small number of bits in the key instead of all 64 of them randomly.
    This is fragile and can lead to non uniqueness issues in the TT.
    
    Key make_key(uint64_t seed) takes any integer and produces a unique random 64 bit key.
    It is computationally efficient and is based on a congruential pseudo random number
    generator using well tested constants by Donald Knuth
    (see https://en.wikipedia.org/wiki/Linear_congruential_generator)
    
    STC https://tests.stockfishchess.org/tests/view/5ef6c78f761b685b4c724bb6
    LLR: 2.95 (-2.94,2.94) {-1.50,0.50}
    Total: 154320 W: 29343 L: 29376 D: 95601
    Ptnml(0-2): 2543, 18170, 35891, 17889, 2667
    
    LTC https://tests.stockfishchess.org/tests/view/5ef7d1a9020eec13834a940e
    LLR: 2.95 (-2.94,2.94) {-1.50,0.50}
    Total: 53488 W: 6629 L: 6584 D: 40275
    Ptnml(0-2): 372, 4878, 16183, 4955, 356
    
    closes official-stockfish/Stockfish#2773
    
    bench: 4626776
    mstembera authored and vondele committed Jun 28, 2020
    Copy the full SHA
    de24fce View commit details
    Browse the repository at this point in the history
  2. Remove old zobrist trick for castling rights

    Removes an 8 year old micro optimization aimed at 32-bit architectures
    because back then doing an xor of a Key could not be done in one instruction.
    See original commit here 821e1c7
    
    STC https://tests.stockfishchess.org/tests/view/5ef5833dde213bf647527d0c
    LLR: 2.94 (-2.94,2.94) {-1.50,0.50}
    Total: 162648 W: 31053 L: 31097 D: 100498
    Ptnml(0-2): 2841, 18966, 37715, 19000, 2802
    
    LTC https://tests.stockfishchess.org/tests/view/5ef7b1bbf993893290cc1489
    LLR: 2.93 (-2.94,2.94) {-1.50,0.50}
    Total: 62360 W: 7617 L: 7586 D: 47157
    Ptnml(0-2): 423, 5662, 18994, 5663, 438
    
    closes official-stockfish/Stockfish#2775
    
    bench: 4591425
    mstembera authored and vondele committed Jun 28, 2020
    Copy the full SHA
    547c4a2 View commit details
    Browse the repository at this point in the history
  3. Increase value of pawns on fifth rank

    This patch increases the endgame value of pawns on the fifth rank.
    The increase is very small (+1 evaluation point, about 0.005 pawn)
    for the pawns on external columns (a-b-c-f-g-h) and a bit bigger
    (+7 evaluation points, about 0.033 pawn) for the pawns on d5/e5.
    
    STC:
    LLR: 2.93 (-2.94,2.94) {-0.50,1.50}
    Total: 79864 W: 15331 L: 15027 D: 49506
    Ptnml(0-2): 1336, 9284, 18433, 9498, 1381
    https://tests.stockfishchess.org/tests/view/5ef73e2ef993893290cc0c47
    
    LTC:
    LLR: 2.94 (-2.94,2.94) {0.25,1.75}
    Total: 47240 W: 5927 L: 5630 D: 35683
    Ptnml(0-2): 320, 4133, 14440, 4384, 343
    https://tests.stockfishchess.org/tests/view/5ef7c0c4f993893290cc14b7
    
    closes official-stockfish/Stockfish#2776
    
    Bench: 4794633
    snicolet authored and vondele committed Jun 28, 2020
    Copy the full SHA
    2810a1e View commit details
    Browse the repository at this point in the history
  4. Scale down eval for drawish rook endgames.

    STC:
    LLR: 2.96 (-2.94,2.94) {-0.50,1.50}
    Total: 82136 W: 15694 L: 15407 D: 51035
    Ptnml(0-2): 1076, 8960, 20767, 9131, 1134
    https://tests.stockfishchess.org/tests/view/5ef86cf8020eec13834a94dd
    
    LTC:
    LLR: 2.93 (-2.94,2.94) {0.25,1.75}
    Total: 70200 W: 8787 L: 8440 D: 52973
    Ptnml(0-2): 325, 5983, 22170, 6264, 358
    https://tests.stockfishchess.org/tests/view/5ef88225020eec13834a950a
    
    closes official-stockfish/Stockfish#2780
    
    Bench: 4478869
    locutus2 authored and vondele committed Jun 28, 2020
    Copy the full SHA
    16836f3 View commit details
    Browse the repository at this point in the history
  5. Scale down eval for queen imbalance

    We lower the endgame value of the evaluation when we detect that there
    is only one queen left on the board (more precisely, we use a scale
    factor of 37/64, or about 0.58, for the endgame part of the evaluation).
    Hopefully this helps a little bit for the assessment of positions with
    queen imbalance, which are one of the well-known Stockfish weaknesses.
    
    STC:
    LLR: 2.94 (-2.94,2.94) {-0.50,1.50}
    Total: 21600 W: 4176 L: 3955 D: 13469
    Ptnml(0-2): 351, 2457, 5003, 2598, 391
    https://tests.stockfishchess.org/tests/view/5ef871b6020eec13834a94e8
    
    LTC:
    LLR: 2.97 (-2.94,2.94) {0.25,1.75}
    Total: 248328 W: 30596 L: 29720 D: 188012
    Ptnml(0-2): 1544, 22345, 75665, 22911, 1699
    https://tests.stockfishchess.org/tests/view/5ef87aec020eec13834a94fe
    
    Closes official-stockfish/Stockfish#2781
    
    Bench: 4441323
    snicolet committed Jun 28, 2020
    Copy the full SHA
    c7194bd View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2020

  1. Tweak single queen endgame scaling.

    Increase scaling factor for each minor of the opponent side of the queen.
    
    STC:
    LLR: 2.94 (-2.94,2.94) {-0.50,1.50}
    Total: 14528 W: 2860 L: 2653 D: 9015
    Ptnml(0-2): 217, 1632, 3408, 1741, 266
    https://tests.stockfishchess.org/tests/view/5ef98384020eec13834a96a0
    
    LTC:
    LLR: 2.95 (-2.94,2.94) {0.25,1.75}
    Total: 34584 W: 4371 L: 4111 D: 26102
    Ptnml(0-2): 205, 3080, 10501, 3262, 244
    https://tests.stockfishchess.org/tests/view/5ef99972020eec13834a96c9
    
    closes official-stockfish/Stockfish#2782
    
    Bench: 4523573
    locutus2 authored and vondele committed Jun 29, 2020
    Copy the full SHA
    69d3be4 View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2020

  1. Provide WDL statistics

    A number of engines, GUIs and tournaments start to report WDL estimates
    along or instead of scores. This patch enables reporting of those stats
    in a more or less standard way (http://www.talkchess.com/forum3/viewtopic.php?t=72140)
    
    The model this reporting uses is based on data derived from a few million fishtest LTC games,
    given a score and a game ply, a win rate is provided that matches rather closely,
    especially in the intermediate range [0.05, 0.95] that data. Some data is shown at
    https://github.com/glinscott/fishtest/wiki/UsefulData#win-loss-draw-statistics-of-ltc-games-on-fishtest
    Making the conversion game ply dependent is important for a good fit, and is in line
    with experience that a +1 score in the early midgame is more likely a win than in the late endgame.
    
    Even when enabled, the printing of the info causes no significant overhead.
    
    Passed STC:
    LLR: 2.94 (-2.94,2.94) {-1.50,0.50}
    Total: 197112 W: 37226 L: 37347 D: 122539
    Ptnml(0-2): 2591, 21025, 51464, 20866, 2610
    https://tests.stockfishchess.org/tests/view/5ef79ef4f993893290cc146b
    
    closes official-stockfish/Stockfish#2778
    
    No functional change
    vondele committed Jul 1, 2020
    Copy the full SHA
    1100688 View commit details
    Browse the repository at this point in the history

Commits on Jul 3, 2020

  1. Use arrays

    for safe checks, outposts and king protectors in evaluate.cpp
    
    Tested for non regression on the safe checks
    https://tests.stockfishchess.org/tests/view/5ef8b75c020eec13834a9596
    LLR: 2.95 (-2.94,2.94) {-1.50,0.50}
    Total: 22256 W: 4283 L: 4143 D: 13830
    Ptnml(0-2): 291, 2439, 5588, 2459, 351
    
    Tested for non regression on the safe checks, outposts and king protectors
    https://tests.stockfishchess.org/tests/view/5ef8e543020eec13834a95e7
    LLR: 2.95 (-2.94,2.94) {-1.50,0.50}
    Total: 28400 W: 5382 L: 5253 D: 17765
    Ptnml(0-2): 394, 3078, 7119, 3223, 386
    
    closes official-stockfish/Stockfish#2785
    
    No functional change
    Rocky640 authored and vondele committed Jul 3, 2020
    Copy the full SHA
    268c00b View commit details
    Browse the repository at this point in the history
  2. Set UCI_ShowWDL by default to false

    UCI_ShowWDL might not be shown by GUIs that don't know the option,
    but crash on the WDL output, effectively making it hard for users to
    turn it off and run the engine. This sets it by default to false.
    
    fixes official-stockfish/Stockfish#2787
    
    closes official-stockfish/Stockfish#2788
    
    No functional change.
    vondele committed Jul 3, 2020
    Copy the full SHA
    fb83da0 View commit details
    Browse the repository at this point in the history
  3. Remove passed pawn condition.

    This will help scale down relatively high eval in drawish rook endgames with passed pawn like in TCEC S18 Superfinal Game 90.
    
    Passed STC
    LLR: 2.94 (-2.94,2.94) {-1.50,0.50}
    Total: 50456 W: 9644 L: 9540 D: 31272
    Ptnml(0-2): 760, 5637, 12332, 5737, 762
    https://tests.stockfishchess.org/tests/view/5efcb76e59f6f035328940ed
    
    Passed LTC
    LLR: 2.94 (-2.94,2.94) {-1.50,0.50}
    Total: 77264 W: 9518 L: 9518 D: 58228
    Ptnml(0-2): 402, 6766, 24321, 6716, 427
    https://tests.stockfishchess.org/tests/view/5efd2ad759f6f03532894143
    
    closes official-stockfish/Stockfish#2792
    
    Bench: 4431626
    SFisGOD authored and vondele committed Jul 3, 2020
    Copy the full SHA
    67818ee View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2020

  1. denormalize KRKP.

    a non-functional code style change that denormalizes the KRKP endgame,
    making it somewhat easier to read.
    
    closes official-stockfish/Stockfish#2786
    
    No functional change
    protonspring authored and vondele committed Jul 6, 2020
    Copy the full SHA
    c5b2a92 View commit details
    Browse the repository at this point in the history
  2. Add a rank based bonus for blocked pawns.

    Fix for overevaluated blocked pawns on the 5th and 6th rank.
    This is a rewrite of the original idea that uses only two parameters.
    Thanks to rocky640 for pointing this out.
    
    STC:
    LLR: 2.94 (-2.94,2.94) {-0.50,1.50}
    Total: 50800 W: 9707 L: 9446 D: 31647
    Ptnml(0-2): 831, 5851, 11822, 6018, 878
    https://tests.stockfishchess.org/tests/view/5f00b4f359f6f03532894304
    
    LTC:
    LLR: 2.93 (-2.94,2.94) {0.25,1.75}
    Total: 52064 W: 6477 L: 6167 D: 39420
    Ptnml(0-2): 331, 4628, 15834, 4878, 361
    https://tests.stockfishchess.org/tests/view/5f0115fe59f6f03532894345
    
    closes official-stockfish/Stockfish#2794
    
    Bench: 4882833
    locutus2 authored and vondele committed Jul 6, 2020
    Copy the full SHA
    7225d25 View commit details
    Browse the repository at this point in the history

Commits on Jul 9, 2020

  1. Clean-up en passant processing

    the goal of this PR is to better document how we process
    the ep square (if any) given position fen command, and to
    output more meaningful (and consistent) debug fen on the "d"
    command. The implementation follows
    https://en.wikipedia.org/wiki/X-FEN#Encoding_en-passant
    following x-fen, it is "valid" to record ep even if ep would put king en prise.
    
    fixes #2784
    
    closes official-stockfish/Stockfish#2797
    
    No functional change
    Rocky640 authored and vondele committed Jul 9, 2020
    Copy the full SHA
    76a0390 View commit details
    Browse the repository at this point in the history
  2. Connected / blocked pawns simplification

    There is no need to score blocked pawns at many places.
    The idea originated from: Rocky
    Tuning and testing by: Fauzi
    
    Passed STC:
    https://tests.stockfishchess.org/tests/view/5f04f8fd59f6f035328945d4
    LLR: 2.94 (-2.94,2.94) {-1.50,0.50}
    Total: 6352 W: 1299 L: 1118 D: 3935
    Ptnml(0-2): 89, 695, 1469, 792, 131
    
    Passed LTC:
    https://tests.stockfishchess.org/tests/view/5f0527bd59f6f035328945e3
    LLR: 2.94 (-2.94,2.94) {-1.50,0.50}
    Total: 27648 W: 3517 L: 3433 D: 20698
    Ptnml(0-2): 177, 2561, 8301, 2571, 214
    
    closes official-stockfish/Stockfish#2799
    
    Bench: 4734746
    FauziAkram authored and vondele committed Jul 9, 2020
    Copy the full SHA
    804a29c View commit details
    Browse the repository at this point in the history
  3. Simplify make_promotions()

    Remove special case handling of QUIET_CHECKS in make_promotions()
    
    STC https://tests.stockfishchess.org/tests/view/5f055dbb59f6f035328945fb
    LLR: 2.98 (-2.94,2.94) {-1.50,0.50}
    Total: 42808 W: 8177 L: 8054 D: 26577
    Ptnml(0-2): 665, 4890, 10201, 4953, 695
    
    LTC https://tests.stockfishchess.org/tests/view/5f06231a59f6f03532894661
    LLR: 2.96 (-2.94,2.94) {-1.50,0.50}
    Total: 9616 W: 1214 L: 1111 D: 7291
    Ptnml(0-2): 53, 821, 2965, 908, 61
    
    closes official-stockfish/Stockfish#2800
    
    Bench: 4576410
    mstembera authored and vondele committed Jul 9, 2020
    Copy the full SHA
    bf5ce1c View commit details
    Browse the repository at this point in the history
  4. Small cleanups

    closes official-stockfish/Stockfish#2772
    
    No functional change
    vondele committed Jul 9, 2020
    Copy the full SHA
    4006f2c View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2020

  1. Maximize usage of transposition table in probcut

    Probcut is a heuristic that wasn't changed a lot in past years,
    all attempts to change it using information / writing info to transposition table failed.
    
    This patch has a number of differences that can be summarized as follows:
    
    * For TT write/read we use depth - 3. Because probcut search is depth - 4 but we actually do the move prior to it so effectively we do depth - 3 search;
    * In any case of depth of eval from transposition table being >= depth - 3 we either produce cutoff or refuse to even do probcut search, this is allowing us to write info of probcut to transposition table because we know that we wouldn't be overwriting some deeper data with our depth - 3 search - this is an important aspect of this patch;
    * For some not really known reason this patch completely ignores tte->bound() - which was the case for previous patch that made probcut interact with TT, maybe 2) is the reason, although it's unproven.
    
    A first version of this patch passed STC and LTC
    
    passed STC
    https://tests.stockfishchess.org/tests/view/5f05908a59f6f03532894613
    LLR: 2.95 (-2.94,2.94) {-0.50,1.50}
    Total: 95776 W: 18300 L: 17973 D: 59503
    Ptnml(0-2): 1646, 10944, 22377, 11279, 1642
    
    passed LTC
    https://tests.stockfishchess.org/tests/view/5f06b54059f6f035328946bb
    LLR: 2.94 (-2.94,2.94) {0.25,1.75}
    Total: 57128 W: 7266 L: 6938 D: 42924
    Ptnml(0-2): 372, 5163, 17217, 5389, 423
    
    However, an additional bugfix was needed to avoid checking a condition on ttMove if was not available. This passed non-regression bounds on top of the first version:
    
    at STC
    https://tests.stockfishchess.org/tests/view/5f080e5059f6f03532894766
    LLR: 2.94 (-2.94,2.94) {-1.50,0.50}
    Total: 14096 W: 2800 L: 2628 D: 8668
    Ptnml(0-2): 225, 1620, 3238, 1688, 277
    
    at LTC
    https://tests.stockfishchess.org/tests/view/5f0836a559f6f0353289479c
    LLR: 2.95 (-2.94,2.94) {-1.50,0.50}
    Total: 25352 W: 3228 L: 3139 D: 18985
    Ptnml(0-2): 175, 2350, 7549, 2415, 187
    
    closes official-stockfish/Stockfish#2804
    
    Bench 4540940
    Vizvezdenec authored and vondele committed Jul 11, 2020
    Copy the full SHA
    5e91c5d View commit details
    Browse the repository at this point in the history
  2. Introduce bad outpost penalty

    In some French games, Stockfish likes to bring the Knight to a bad outpost spot. This is evident in TCEC S18 Superfinal Game 63, where there is a Knight outpost on the queenside but is actually useless. Stockfish is effectively playing a piece down while holding ground against Leela's break on the kingside.
    
    This patch turns the +56 mg bonus for a Knight outpost into a -7 mg penalty if it satisfies the following conditions:
    
    * The outpost square is not on the CenterFiles (i.e. not on files C,D,E and F)
    * The knight is not attacking non pawn enemies.
    * The side where the outpost is located contains only few enemies, with a particular conditional_more_than_two() implementation
    
    Thank you to apospa...@gmail.com for bringing this to our attention and for providing insights.
    See https://groups.google.com/forum/?fromgroups=#!topic/fishcooking/dEXNzSIBgZU
    Reference game: https://tcec-chess.com/#div=sf&game=63&season=18
    
    Passed STC:
    LLR: 2.93 (-2.94,2.94) {-0.50,1.50}
    Total: 6960 W: 1454 L: 1247 D: 4259
    Ptnml(0-2): 115, 739, 1610, 856, 160
    https://tests.stockfishchess.org/tests/view/5f08221059f6f0353289477e
    
    Passed LTC:
    LLR: 2.98 (-2.94,2.94) {0.25,1.75}
    Total: 21440 W: 2767 L: 2543 D: 16130
    Ptnml(0-2): 122, 1904, 6462, 2092, 140
    https://tests.stockfishchess.org/tests/view/5f0838ed59f6f035328947a2
    
    various related tests show strong test results, but so far no generalizations or simplifications of conditional_more_than_two() are found. See PR for details.
    
    closes official-stockfish/Stockfish#2803
    
    Bench: 4366686
    SFisGOD authored and vondele committed Jul 11, 2020
    Copy the full SHA
    1f3bd96 View commit details
    Browse the repository at this point in the history
  3. Copy the full SHA
    db0615e View commit details
    Browse the repository at this point in the history