Skip to content

Commit

Permalink
Normalize evaluation
Browse files Browse the repository at this point in the history
With NNUE, our evaluation is no longer related to the classical parameter PawnValueEg (=208).

This leads to the current evaluation changing quite a bit from release to release,
for example, the eval needed to have 50% win rate at fishtest LTC (in cp and internal Value):

June 2020  :   113cp (237)
June 2021  :   115cp (240)
April 2022 :   134cp (279)
July 2022  :   167cp (348)

This inflation can be fixed if one fixes 100cp to mean 50% win chance,
and decouple this conversion from PawnValueEg. This conversion is somewhat arbitrary,
only the relative ranking of positions is important for an engine, which is designed the find the best move.

While there is no simple 1-to-1 relation between the internally used Value, and the win rate,
we can base this on the win_rate_model. The 'a' parameter of this model, gives 50%, and by picking
this value at move 32, this is just the sum of the parameters of the model for a (i.e. the 'as' array).

This patch introduces Internal2Pawn to convert the internal units to cp,
and converts to win_rate_model to internal units.

Generally, it might be better to directly use the wdl values (available with the option UCI_ShowWDL) in analysis,
or focus directly on the bestmove and PV lines provided.

No functional change
  • Loading branch information
vondele committed Nov 1, 2022
1 parent d09653d commit 7c72f3f
Show file tree
Hide file tree
Showing 3 changed files with 14 additions and 6 deletions.
4 changes: 2 additions & 2 deletions src/nnue/evaluate_nnue.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,7 @@ namespace Stockfish::Eval::NNUE {

buffer[0] = (v < 0 ? '-' : v > 0 ? '+' : ' ');

int cp = std::abs(100 * v / PawnValueEg);
int cp = std::abs(100 * v / UCI::Internal2Pawn);
if (cp >= 10000)
{
buffer[1] = '0' + cp / 10000; cp %= 10000;
Expand Down Expand Up @@ -251,7 +251,7 @@ namespace Stockfish::Eval::NNUE {

buffer[0] = (v < 0 ? '-' : v > 0 ? '+' : ' ');

double cp = 1.0 * std::abs(int(v)) / PawnValueEg;
double cp = 1.0 * std::abs(int(v)) / UCI::Internal2Pawn;
sprintf(&buffer[1], "%6.2f", cp);
}

Expand Down
12 changes: 8 additions & 4 deletions src/uci.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -207,13 +207,17 @@ namespace {
// The coefficients of a third-order polynomial fit is based on the fishtest data
// for two parameters that need to transform eval to the argument of a logistic
// function.
double as[] = { 0.50379905, -4.12755858, 18.95487051, 152.00733652};
double bs[] = {-1.71790378, 10.71543602, -17.05515898, 41.15680404};
constexpr double as[] = { 1.04790516, -8.58534089, 39.42615625, 316.17524816};
constexpr double bs[] = { -3.57324784, 22.28816201, -35.47480551, 85.60617701 };

// Enforce that Internal2Pawn corresponds to a 50% win rate at ply 64
static_assert(UCI::Internal2Pawn == int(as[0] + as[1] + as[2] + as[3]));

double a = (((as[0] * m + as[1]) * m + as[2]) * m) + as[3];
double b = (((bs[0] * m + bs[1]) * m + bs[2]) * m) + bs[3];

// Transform the eval to centipawns with limited range
double x = std::clamp(double(100 * v) / PawnValueEg, -2000.0, 2000.0);
double x = std::clamp(double(v), -4000.0, 4000.0);

// Return the win rate in per mille units rounded to the nearest value
return int(0.5 + 1000 / (1 + std::exp((a - x) / b)));
Expand Down Expand Up @@ -312,7 +316,7 @@ string UCI::value(Value v) {
stringstream ss;

if (abs(v) < VALUE_MATE_IN_MAX_PLY)
ss << "cp " << v * 100 / PawnValueEg;
ss << "cp " << v * 100 / Internal2Pawn;
else
ss << "mate " << (v > 0 ? VALUE_MATE - v + 1 : -VALUE_MATE - v) / 2;

Expand Down
4 changes: 4 additions & 0 deletions src/uci.h
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,10 @@ class Position;

namespace UCI {

// Formerly PawnValueEg. This value currently results in 100cp having a 50% win rate.
// This is not too different from the historical value (113cp was 50% win rate in 2020).
const int Internal2Pawn = 348;

class Option;

/// Define a custom comparator, because the UCI options should be case-insensitive
Expand Down

0 comments on commit 7c72f3f

Please sign in to comment.