Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove the Contempt UCI option #3575

Closed

Conversation

snicolet
Copy link
Member

@snicolet snicolet commented Jun 21, 2021

This patch removes the UCI option for setting Contempt in classical evaluation.

It is exactly equivalent to using Contempt=0 for the UCI contempt value and keeping
the dynamic part in the algo (renaming this dynamic part trend to better describe
what it does). We have tried quite hard to implement a working Contempt feature for
NNUE but nothing really worked, so it is probably time to give up.

Interested chess fans wishing to keep playing with the UCI option for Contempt and
use it with the classical eval are urged to download the version tagged "SF_Classical"
of Stockfish (dated 31 July 2020), as it was the last version where our search
algorithm was tuned for the classical eval and is probably our strongest classical
player ever: https://github.com/official-stockfish/Stockfish/tags

Passed STC:
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 72904 W: 6228 L: 6175 D: 60501
Ptnml(0-2): 221, 5006, 25971, 5007, 247
https://tests.stockfishchess.org/tests/view/60c98bf9457376eb8bcab18d

Passed LTC:
LLR: 2.93 (-2.94,2.94) <-2.50,0.50>
Total: 45168 W: 1601 L: 1547 D: 42020
Ptnml(0-2): 38, 1331, 19786, 1397, 32
https://tests.stockfishchess.org/tests/view/60c9c7fa457376eb8bcab1bb

closes #3575

Bench: 4947716

This patch removes the UCI option for setting Contempt in classical evaluation.

It is exactly equivalent to using Contempt=0 for the UCI contempt value and keeping
the dynamic part in the algo (renaming this dynamic part `trend` to better describe
what it does). We have tried quite hard to implement a working Contempt feature for
NNUE but nothing really worked, so it is probably time to give up.

Interested chess fans wishing to keep playing with the UCI option for Contempt and
use it with the classical eval are urged to download the version tagged "SF_Classical"
of Stockfish (dated 31 July 2020), as it was the last version where our search
algorithm was tuned for the classical eval and is probably our strongest classical
player ever: https://github.com/official-stockfish/Stockfish/tags

Passed STC:
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 72904 W: 6228 L: 6175 D: 60501
Ptnml(0-2): 221, 5006, 25971, 5007, 247
https://tests.stockfishchess.org/tests/view/60c98bf9457376eb8bcab18d

Passed LTC:
LLR: 2.93 (-2.94,2.94) <-2.50,0.50>
Total: 45168 W: 1601 L: 1547 D: 42020
Ptnml(0-2): 38, 1331, 19786, 1397, 32
https://tests.stockfishchess.org/tests/view/60c9c7fa457376eb8bcab1bb

closes official-stockfish/Stockfish#3575

Bench: 4947716
@NightlyKing
Copy link
Contributor

I'm very much in favour of this change. Most users get a bit confused what contempt actually does - at least judging by the conversations I've seen in our discord. For now, we don't need this UCI option at all nor is it needed for non nnue analysis.
If at some point in time somebody finds a contempt solution for NNUE, we're able to get that UCI option back no problem.

@snicolet
Copy link
Member Author

Zactly!

@ssj100
Copy link

ssj100 commented Jun 21, 2021

Thanks! I've been in favour of this for a while.

@locutus2
Copy link
Member

Because this effects mainly classic eval this should also be a non-regression for USE NNUE=false IMO. Else we should ditch the current policy for using Use NNUE=false for classic eval changes. For me personally its not important if Contempt in or not but a consequent and not contradicting testing regime is important.

@vondele
Copy link
Member

vondele commented Jun 21, 2021

If think this is slightly different from other changes to classical eval.
This is basically removing an option that is really confusing for users, as it has no impact for how the engine is used predominately.

Having said this, IIRC, we know this is not regressive from older tests.

@xoto10
Copy link
Contributor

xoto10 commented Jun 21, 2021

Just to clarify, the default contempt is currently 24, and this is used during classical evals when playing with use_nnue=true, is that right?
(Just looked at the code, I thought we'd set the default to 0)

@vondele
Copy link
Member

vondele commented Jun 21, 2021

yes

@crossbr
Copy link

crossbr commented Jun 21, 2021

Following up xoto10's question, it is useful to be able to do analysis with contempt=0 (while NNUE=true) when studying a line of play from the perspective of both white and black, without needing to add or subtract 24. Would this change mean that contempt=24 (while NNUE=true) would be unchangeable for the user?

@xoto10
Copy link
Contributor

xoto10 commented Jun 21, 2021

I think it will be Contempt=0 all the time when this patch is merged.

@crossbr
Copy link

crossbr commented Jun 21, 2021

@xoto10 Thanks, then I'm fine with the change.

@MichaelB7
Copy link
Contributor

+1 Very nice, elimiates confusion and is no longer needed with NNUE. The value for analysis was always suspect anyway. It did have a place and time where it made sense to have it, but only aganist weaker engines.

@vondele vondele added cluster to be merged Will be merged shortly and removed cluster labels Jun 21, 2021
@vondele vondele closed this in ed436a3 Jun 21, 2021
@sesse
Copy link

sesse commented Jul 5, 2021

Although I understand that this is really hard (possibly impossible, due to the NNUE parametrization?), I'm still a bit sad to see contempt go. Not because of playing strength, but to be able to analyze human armageddon games or other “must-win” situations. You end up too easily in 0.00 land with a perpetual line that the player would just never accept.

Maybe we can revisit this in ten years' time! :-)

@Sopel97
Copy link
Member

Sopel97 commented Jul 5, 2021

Contempt just didn't work with NNUE, like at all. It only impacted classical eval which created discrepancies between classical and NNUE. I'm working on nets trained on data scored with different draw score, to approach the possibility of having contempt in a different way, but rescoring will take some time.

@sesse
Copy link

sesse commented Jul 5, 2021

I understand; I'm not saying the PR is stupid and should be reverted. :-) Just lamenting a bit. (And I suppose scoring three-fold repetitions or draw by material as something nonzero will cause incredibly weird discontinuities in the eval.)

MichaelB7 pushed a commit to MichaelB7/Stockfish that referenced this pull request Jul 6, 2021
This patch removes the UCI option for setting Contempt in classical evaluation.

It is exactly equivalent to using Contempt=0 for the UCI contempt value and keeping
the dynamic part in the algo (renaming this dynamic part `trend` to better describe
what it does). We have tried quite hard to implement a working Contempt feature for
NNUE but nothing really worked, so it is probably time to give up.

Interested chess fans wishing to keep playing with the UCI option for Contempt and
use it with the classical eval are urged to download the version tagged "SF_Classical"
of Stockfish (dated 31 July 2020), as it was the last version where our search
algorithm was tuned for the classical eval and is probably our strongest classical
player ever: https://github.com/official-stockfish/Stockfish/tags

Passed STC:
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 72904 W: 6228 L: 6175 D: 60501
Ptnml(0-2): 221, 5006, 25971, 5007, 247
https://tests.stockfishchess.org/tests/view/60c98bf9457376eb8bcab18d

Passed LTC:
LLR: 2.93 (-2.94,2.94) <-2.50,0.50>
Total: 45168 W: 1601 L: 1547 D: 42020
Ptnml(0-2): 38, 1331, 19786, 1397, 32
https://tests.stockfishchess.org/tests/view/60c9c7fa457376eb8bcab1bb

closes official-stockfish#3575

Bench: 4947716
@vondele
Copy link
Member

vondele commented Jul 6, 2021

as said contempt wasn't really functional with NNUE. Armageddon would probably need a completely new scoring to do properly, not just scoring draw positions, but even the evaluation. That would actually be a feasible, albeit big, project to start training from scratch. I.e. adjust eval of draw positions, rescore NNUE training data, train NNUE, etc... maybe some fork could be created to do that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
to be merged Will be merged shortly
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

10 participants