Skip to content

Commit

Permalink
Introduce dynamic contempt
Browse files Browse the repository at this point in the history
Make contempt dependent on the current score of the root position.

The idea is that we now use a linear formula like the following to decide
on the contempt to use during a search :

    contempt = x + y * eval

where x is the base contempt set by the user in the "Contempt" UCI option,
and y * eval is the dynamic part which adapts itself to the estimation of
the evaluation of the root position returned by the search. In this patch,
we use x = 18 centipawns by default, and the y * eval correction can go
from -20 centipawns if the root eval is less than -2.0 pawns, up to +20
centipawns when the root eval is more than 2.0 pawns.

To summarize, the new contempt goes from -0.02 to 0.38 pawns, depending if
Stockfish is losing or winning, with an average value of 0.18 pawns by default.

STC:
LLR: 2.95 (-2.94,2.94) [0.00,5.00]
Total: 110052 W: 24614 L: 23938 D: 61500
http://tests.stockfishchess.org/tests/view/5a72e6020ebc590f2c86ea20

LTC:
LLR: 2.97 (-2.94,2.94) [0.00,5.00]
Total: 16470 W: 2896 L: 2705 D: 10869
http://tests.stockfishchess.org/tests/view/5a76c5b90ebc5902971a9830

A second match at LTC was organised against the current master:

ELO: 1.45 +-2.9 (95%) LOS: 84.0%
Total: 19369 W: 3350 L: 3269 D: 12750
http://tests.stockfishchess.org/tests/view/5a7acf980ebc5902971a9a2e

Finally, we checked that there is no apparent problem with multithreading,
despite the fact that some threads might have a slightly different contempt
level that the main thread.

Match of this version against master, both using 5 threads, time control 30+0.3:
ELO: 2.18 +-3.2 (95%) LOS: 90.8%
Total: 14840 W: 2502 L: 2409 D: 9929
http://tests.stockfishchess.org/tests/view/5a7bf3e80ebc5902971a9aa2

Include suggestions from Marco Costalba, Aram Tumanian, Ronald de Man, etc.

Bench: 5207156
  • Loading branch information
Stefano80 authored and snicolet committed Feb 9, 2018
1 parent d71adc5 commit cb13243
Show file tree
Hide file tree
Showing 4 changed files with 18 additions and 8 deletions.
2 changes: 1 addition & 1 deletion src/evaluate.cpp
Expand Up @@ -905,7 +905,7 @@ namespace {

} // namespace

Score Eval::Contempt = SCORE_ZERO;
std::atomic<Score> Eval::Contempt;

/// evaluate() is the evaluator for the outer world. It returns a static evaluation
/// of the position from the point of view of the side to move.
Expand Down
3 changes: 2 additions & 1 deletion src/evaluate.h
Expand Up @@ -21,6 +21,7 @@
#ifndef EVALUATE_H_INCLUDED
#define EVALUATE_H_INCLUDED

#include <atomic>
#include <string>

#include "types.h"
Expand All @@ -31,7 +32,7 @@ namespace Eval {

const Value Tempo = Value(20); // Must be visible to search

extern Score Contempt;
extern std::atomic<Score> Contempt;

std::string trace(const Position& pos);

Expand Down
19 changes: 14 additions & 5 deletions src/search.cpp
Expand Up @@ -191,11 +191,6 @@ void MainThread::search() {
Time.init(Limits, us, rootPos.game_ply());
TT.new_search();

int contempt = Options["Contempt"] * PawnValueEg / 100; // From centipawns

Eval::Contempt = (us == WHITE ? make_score(contempt, contempt / 2)
: -make_score(contempt, contempt / 2));

if (rootMoves.empty())
{
rootMoves.emplace_back(MOVE_NONE);
Expand Down Expand Up @@ -282,6 +277,7 @@ void Thread::search() {
Depth lastBestMoveDepth = DEPTH_ZERO;
MainThread* mainThread = (this == Threads.main() ? Threads.main() : nullptr);
double timeReduction = 1.0;
Color us = rootPos.side_to_move();

std::memset(ss-4, 0, 7 * sizeof(Stack));
for (int i = 4; i > 0; i--)
Expand All @@ -306,6 +302,10 @@ void Thread::search() {

multiPV = std::min(multiPV, rootMoves.size());

int contempt = Options["Contempt"] * PawnValueEg / 100; // From centipawns
Eval::Contempt = (us == WHITE ? make_score(contempt, contempt / 2)
: -make_score(contempt, contempt / 2));

// Iterative deepening loop until requested to stop or the target depth is reached
while ( (rootDepth += ONE_PLY) < DEPTH_MAX
&& !Threads.stop
Expand Down Expand Up @@ -340,6 +340,15 @@ void Thread::search() {
delta = Value(18);
alpha = std::max(rootMoves[PVIdx].previousScore - delta,-VALUE_INFINITE);
beta = std::min(rootMoves[PVIdx].previousScore + delta, VALUE_INFINITE);

// Adjust contempt based on current situation
contempt = Options["Contempt"] * PawnValueEg / 100; // From centipawns
contempt += bestValue > 500 ? 50: // Dynamic contempt
bestValue < -500 ? -50:
bestValue / 10;

Eval::Contempt = (us == WHITE ? make_score(contempt, contempt / 2)
: -make_score(contempt, contempt / 2));
}

// Start with a small aspiration window and, in the case of a fail
Expand Down
2 changes: 1 addition & 1 deletion src/ucioption.cpp
Expand Up @@ -59,7 +59,7 @@ void init(OptionsMap& o) {
const int MaxHashMB = Is64Bit ? 131072 : 2048;

o["Debug Log File"] << Option("", on_logger);
o["Contempt"] << Option(20, -100, 100);
o["Contempt"] << Option(18, -100, 100);
o["Threads"] << Option(1, 1, 512, on_threads);
o["Hash"] << Option(16, 1, MaxHashMB, on_hash_size);
o["Clear Hash"] << Option(on_clear_hash);
Expand Down

3 comments on commit cb13243

@FauziAkram
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this test was supposed to be 40k games, but it has already surpassed that

@snicolet
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

http://tests.stockfishchess.org/tests/view/5a7defef0ebc5902971a9c00

Indeed, I suppose that it ran 40000 games, then the first worker result was purged (idx 0 was red and is now missing), and now it runs over 40000 games, this is funny :-)

What's more, the machine mgrabiak in slot 45 is posting a result which means that SF9 is losing Elo compared to SF8, this is funny too.

mgrabiak

I propose to let it run to see how this will develop...

@snicolet
Copy link
Owner

@snicolet snicolet commented on cb13243 Feb 12, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moreover, when mgrabiak-3cores-15 is busy running in this regression test, it does not pollute the other tests. Time to have some green patches :-)

Please sign in to comment.