-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove the Contempt UCI option #3575
Conversation
This patch removes the UCI option for setting Contempt in classical evaluation. It is exactly equivalent to using Contempt=0 for the UCI contempt value and keeping the dynamic part in the algo (renaming this dynamic part `trend` to better describe what it does). We have tried quite hard to implement a working Contempt feature for NNUE but nothing really worked, so it is probably time to give up. Interested chess fans wishing to keep playing with the UCI option for Contempt and use it with the classical eval are urged to download the version tagged "SF_Classical" of Stockfish (dated 31 July 2020), as it was the last version where our search algorithm was tuned for the classical eval and is probably our strongest classical player ever: https://github.com/official-stockfish/Stockfish/tags Passed STC: LLR: 2.95 (-2.94,2.94) <-2.50,0.50> Total: 72904 W: 6228 L: 6175 D: 60501 Ptnml(0-2): 221, 5006, 25971, 5007, 247 https://tests.stockfishchess.org/tests/view/60c98bf9457376eb8bcab18d Passed LTC: LLR: 2.93 (-2.94,2.94) <-2.50,0.50> Total: 45168 W: 1601 L: 1547 D: 42020 Ptnml(0-2): 38, 1331, 19786, 1397, 32 https://tests.stockfishchess.org/tests/view/60c9c7fa457376eb8bcab1bb closes official-stockfish/Stockfish#3575 Bench: 4947716
0866636
to
6146cfe
Compare
I'm very much in favour of this change. Most users get a bit confused what contempt actually does - at least judging by the conversations I've seen in our discord. For now, we don't need this UCI option at all nor is it needed for non nnue analysis. |
Zactly! |
Thanks! I've been in favour of this for a while. |
Because this effects mainly classic eval this should also be a non-regression for USE NNUE=false IMO. Else we should ditch the current policy for using Use NNUE=false for classic eval changes. For me personally its not important if Contempt in or not but a consequent and not contradicting testing regime is important. |
If think this is slightly different from other changes to classical eval. Having said this, IIRC, we know this is not regressive from older tests. |
Just to clarify, the default contempt is currently 24, and this is used during classical evals when playing with use_nnue=true, is that right? |
yes |
Following up xoto10's question, it is useful to be able to do analysis with contempt=0 (while NNUE=true) when studying a line of play from the perspective of both white and black, without needing to add or subtract 24. Would this change mean that contempt=24 (while NNUE=true) would be unchangeable for the user? |
I think it will be Contempt=0 all the time when this patch is merged. |
@xoto10 Thanks, then I'm fine with the change. |
+1 Very nice, elimiates confusion and is no longer needed with NNUE. The value for analysis was always suspect anyway. It did have a place and time where it made sense to have it, but only aganist weaker engines. |
Although I understand that this is really hard (possibly impossible, due to the NNUE parametrization?), I'm still a bit sad to see contempt go. Not because of playing strength, but to be able to analyze human armageddon games or other “must-win” situations. You end up too easily in 0.00 land with a perpetual line that the player would just never accept. Maybe we can revisit this in ten years' time! :-) |
Contempt just didn't work with NNUE, like at all. It only impacted classical eval which created discrepancies between classical and NNUE. I'm working on nets trained on data scored with different draw score, to approach the possibility of having contempt in a different way, but rescoring will take some time. |
I understand; I'm not saying the PR is stupid and should be reverted. :-) Just lamenting a bit. (And I suppose scoring three-fold repetitions or draw by material as something nonzero will cause incredibly weird discontinuities in the eval.) |
This patch removes the UCI option for setting Contempt in classical evaluation. It is exactly equivalent to using Contempt=0 for the UCI contempt value and keeping the dynamic part in the algo (renaming this dynamic part `trend` to better describe what it does). We have tried quite hard to implement a working Contempt feature for NNUE but nothing really worked, so it is probably time to give up. Interested chess fans wishing to keep playing with the UCI option for Contempt and use it with the classical eval are urged to download the version tagged "SF_Classical" of Stockfish (dated 31 July 2020), as it was the last version where our search algorithm was tuned for the classical eval and is probably our strongest classical player ever: https://github.com/official-stockfish/Stockfish/tags Passed STC: LLR: 2.95 (-2.94,2.94) <-2.50,0.50> Total: 72904 W: 6228 L: 6175 D: 60501 Ptnml(0-2): 221, 5006, 25971, 5007, 247 https://tests.stockfishchess.org/tests/view/60c98bf9457376eb8bcab18d Passed LTC: LLR: 2.93 (-2.94,2.94) <-2.50,0.50> Total: 45168 W: 1601 L: 1547 D: 42020 Ptnml(0-2): 38, 1331, 19786, 1397, 32 https://tests.stockfishchess.org/tests/view/60c9c7fa457376eb8bcab1bb closes official-stockfish#3575 Bench: 4947716
as said contempt wasn't really functional with NNUE. Armageddon would probably need a completely new scoring to do properly, not just scoring draw positions, but even the evaluation. That would actually be a feasible, albeit big, project to start training from scratch. I.e. adjust eval of draw positions, rescore NNUE training data, train NNUE, etc... maybe some fork could be created to do that. |
This patch removes the UCI option for setting Contempt in classical evaluation.
It is exactly equivalent to using Contempt=0 for the UCI contempt value and keeping
the dynamic part in the algo (renaming this dynamic part
trend
to better describewhat it does). We have tried quite hard to implement a working Contempt feature for
NNUE but nothing really worked, so it is probably time to give up.
Interested chess fans wishing to keep playing with the UCI option for Contempt and
use it with the classical eval are urged to download the version tagged "SF_Classical"
of Stockfish (dated 31 July 2020), as it was the last version where our search
algorithm was tuned for the classical eval and is probably our strongest classical
player ever: https://github.com/official-stockfish/Stockfish/tags
Passed STC:
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 72904 W: 6228 L: 6175 D: 60501
Ptnml(0-2): 221, 5006, 25971, 5007, 247
https://tests.stockfishchess.org/tests/view/60c98bf9457376eb8bcab18d
Passed LTC:
LLR: 2.93 (-2.94,2.94) <-2.50,0.50>
Total: 45168 W: 1601 L: 1547 D: 42020
Ptnml(0-2): 38, 1331, 19786, 1397, 32
https://tests.stockfishchess.org/tests/view/60c9c7fa457376eb8bcab1bb
closes #3575
Bench: 4947716