-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TCEC Season 8, game 22: Eval dropped From +26.13 to 0.00 #501
Comments
This PGN posted by amhijo does show something not quite right with the handling of the hash entries. I'll reproduce it below together with my description of what is going on.
Using latest master, single-threaded. Select move 69 and engage infinite analysis. Immediately evaluated as 0.00 to depth 30+ since Black can repeat the position after move 67 with 69... Rb4. I don't think this is a true 3-fold yet though. Now, the effect of that is to load the hash with an 0.00 eval for that position with a depth of 30+. If I now click on the position after 63...Qf6, leaving the analysis running (so no hash clear) Bc2 is now discounted as a candidate move because of this deep 0.00 stored in the hash. Given enough time, so that Bc2 is evaluated to a high enough depth (bearing in mind its now way down the move order, so will have reductions), it will recover, but that would take a long time. So the 0.00 repetition based eval is being stored in the hash and then being applied to a position where that move would not be a repetition. Obviously this particular sequence is not an actual game - it's another case of something that interferes with analysis - but is it possible that something similar can happen wherein one thread analyses a line, marks a position (with a rep) as 0.00, and then another thread incorrectly applies that TT score to the non-repped position elsewhere in the tree? |
@cuddlestmonkey: This is a very old known problem: Bad interaction between Transposition table and 2-fold repetition. It can't be solved without slowing down the engine massively. |
@zamar Maybe, but it's worth noted that I don't get the same issue using Komodo (another 2-fold rep engine). Komodo 9 evaluates move 69 as 0.00, but evidently isn't using that 0.00 to short-circuit the eval of Bc2 when I switch to move 64. |
I strongly suspect the graph-history interaction problem also to be the cause of the game 22 problems. Although I have reproduced the problem with YBW Stockfish (assuming I am correct in thinking that the SF binary from abrok of Thu Oct 15 21:27:52, timestamp 1444969672 is still YBW!!), it would not surprise me if the problem occurs more often with lazy smp. This is because, as I understand, lazy smp lets some threads search deeper than other threads. This could result in one thread searching a position X deeper in the tree (with some key positions for the position X already flagged in history) with relatively high depth, resulting in that position being stored in hash with relatively high depth and a "too low" score (due to the flagged key positions being scored as draw when encountered below in the search of X). When X is then encountered closer to the root by another thread searching less deeply, that thread will accept the "too low" score even though that is wrong. To reduce the bad effects of the graph-history interaction problem, it seems important to not let threads search at different depths in the endgame. A paper from 1985 on this problem: From the conclusion: "The key in avoiding most occurrences of GHI appears to be iterative deepening". If a position occurs multiple times in the search tree, it should be attempted to first search the occurrence of it that is closest to the root. |
Another paper: "The Graph History Interaction (GHI) Problem occurs when the same game position behaves differently when reached via different paths. For example, after following one path a move m may be legal in position p, while after following another path the same move is illegal in p. |
Unfortunately that "general" solution is only general in a very specific sense. Basically it is of no value for a game-playing engine, but only for game solvers. See http://www.open-chess.org/viewtopic.php?p=17480#p17480 Before I place all the blame on lazy smp threads that search at different depths (even though the problem can be reproduced with YBW versions), the main trigger of the problem might be a particular combination of reductions and extensions that may result in, say, a position P being searched at depth N at a node further away from the root (with a larger position history) before that same position P is searched at depth N at a node closer to the root (with a smaller position history). |
Based on the linked talkchess thread (link given by Vince in the forum), So far, I was not able to get a draw score for the position at move 64. |
@syzygy1 Shame. In regard of reductions and extensions, there was a position given by Uli that exhibited a similar strange "reset to zero" behaviour, and lowering the amount of reductions in that case removed the problem, so you may well be right. |
Just one example with my patch.
It looks like as soon as the fail-low cycle begins, my patch breaks this and SF starts to fail-high again. I don't pretend my patch 'solves' anything, but it really seems to help. The other thing to consider is the search instability. I think it would also help to open the aspiration window a bit faster, not allowing so many fail-lows in sequence. |
This kind of issue is not very useful. After almost one year, it still can't be be reproduced. If it can't be reproduced, it can't be understood or fixed. |
@lucasart I agree. Closing. |
Mostly skip passed pawn bonus for grid chess
Stockfish lost track of the winning line.
Eval dropped from +26.13 to 0.00.
Potentially this is caused by the combination of the following:
The text was updated successfully, but these errors were encountered: