Quantize eval to multiples of 16.

[vondele]

remove some excess precision, helps searchs.

Effectively reintroduces 45dbd9c,
with a slightly different context.

passed STC
LLR: 2.97 (-2.94,2.94) {-0.50,1.50}
Total: 197032 W: 37938 L: 37462 D: 121632
Ptnml(0-2): 3359, 22994, 45446, 23246, 3471
https://tests.stockfishchess.org/tests/view/5ee0c228f29b40b0fc95ae53

passed LTC
LLR: 2.94 (-2.94,2.94) {0.25,1.75}
Total: 77696 W: 9970 L: 9581 D: 58145
Ptnml(0-2): 530, 7075, 23311, 7340, 592
https://tests.stockfishchess.org/tests/view/5ee21426f29b40b0fc95af43

passed LTC SMP
LLR: 2.96 (-2.94,2.94) {0.25,1.75}
Total: 64136 W: 7425 L: 7091 D: 49620
Ptnml(0-2): 345, 5416, 20228, 5718, 361
https://tests.stockfishchess.org/tests/view/5ee387bbf29b40b0fc95b04c

Bench: 4562134

[ssj100]

Correct link for the "running LTC SMP":
https://tests.stockfishchess.org/tests/view/5ee387bbf29b40b0fc95b04c

[jhellis3]

I think the question is: are the consequences worth 1.5 Elo?

[adentong]

@jhellis3 Technically speaking, every single patch has consequences. Curious why you think the consequences of this are severe enough to overlook the apparent elo gain?

[jhellis3]

Well, anyone who gives it some genuine thought should to be able to come up with at least a few legitimate concerns....

[vondele]

feel free to share your concerns, especially with some data helps us understand what the problem is.

[jhellis3]

I'm not the one who makes the decision on what gets merged.

[jhellis3]

And I also can't provide data from the future.... What I can say is Stockfish has gained considerable Elo in the last 6+ years.

[AlexandreMasta]

And I also can provide data from the future.... What I can say is Stockfish has gained considerable Elo in the last 6+ years.

I think that what Jhellis wants to say is that obfuscating eval by rounding it can damage search gains in the future. In the long run the more accurate eval is the more search speed ups and improvements are more impactful. BTW...what is the final goal of an engine? Isn´t it to achieve a probably perfect evaluation? How will you achieve this goal by adding noise to it? In some point this "trick" will have to be removed to achieve new limits.

Maybe this is the rationale. Maybe I´m totally wrong. But whatever. I just tried to understand what he was saying.

[ddugovic]

For candidate move-ordering purposes, this complication sounds similar in effect to adding a small pseudo-random number to each evaluation, but without the advantage of being able to seed the PRNG (in order to expose butterfly effects).

[adentong]

@AlexandreMasta I can't say I agree. An engine should be able to play the perfect GAME, regardless of what the eval looks like.

[adentong]

Even the SMP test passed now. Solid elogainer this is.

[snicolet]

Merged via 4d65761, congrats!

[vdbergh]

For candidate move-ordering purposes, this complication sounds similar in effect to adding a small pseudo-random number to each evaluation,

@ddugovic Yes that's right. Quantization noise is usually modeled as white noise. I wonder though if 16 is already so large that there might be a non-negligible correlation between the signal and the noise.

[Rocky640]

I think that future SPSA tuning about evaluate.cpp, pawns.cpp or material.cpp will have to first disable that line.
In general it is hard to imagine any new small eval bonus or tweak that will make it. 16 seems quite large.

[vondele]

@Rocky640 I actually don't think so, but it would be worth testing. The reason is that these small terms will still contribute, i.e. trip quantization to jump one way or another. Said differently small terms increase the accuracy of the eval function (i.e. move it in the right way), but don't improve the precision of the val function (which is mostly the largest error). useful picture. So, I think small terms will pass equally well. Actually the experiment is easy, let's try to remove a bunch of eval terms with simplification bounds... I don't think we'll be able to remove any.

[vondele]

So, as a test, I started 6 tests to remove small eval times as simplifications:

  constexpr Score BishopPawns         = S(  3,  7);
  constexpr Score BishopXRayPawns     = S(  4,  5);
  constexpr Score FlankAttacks        = S(  8,  0);
  constexpr Score BishopKingProtector = S(  6,  9);
  constexpr Score KnightKingProtector = S(  8,  9);
  constexpr Score RestrictedPiece     = S(  7,  7);
  constexpr Score RookOnQueenFile     = S(  5,  9);

[vondele]

None of the terms could be removed (5 failed at STC, RookOnQueenFile at LTC).

[Rocky640]

That was quick ! And thank you for the explanation. (useful picture with text here https://chemistrygod.com/accuracy-and-precision-in-chemistry)

The bonus you tested, are usually multiplied by some factor
RookOnQueenFile will use a factor 2 at most, the other often more.

I'm still concerned about features which are scored usually only once.
For example psqt, or some material imbalance.

I'm not against the quantization idea, time will tell if 16 i the best value.
Just curious... Was 8 tried ? Was 4 tried ? or more than 16 ?

[vondele]

yes, other values were tried
8: https://tests.stockfishchess.org/tests/view/5ee106aef29b40b0fc95ae9c
32: https://tests.stockfishchess.org/tests/view/5ee106aaf29b40b0fc95ae9a
as well as other locations (wrt to tempo and 50mrc scaling).

yes, I'm aware these bonus terms are often scaled, but typically the smallest change is still on the order of the bonus. However, feel free to test other terms. Things would be very different if we would round after each eval terms is added, but that's not what we do.

[Rocky640]

Another result
At LTC Quanta 8 was not better than master Quanta 16
https://tests.stockfishchess.org/tests/view/5ee6aab587586124bc2c109c