Correction for each nominal game stage, depending on the total quantity of material

[GuardianRM]

Correction for each nominal game stage, depending on the total quantity of material -
The system of bonuses and penalties, for the position estimate correction, determined by the formula

Bonus = (k1 - k2) * k3.

Where:
k1 - is a configurable tabular value, that depends on the current set of pieces.
k2 - the index of the nominal current game stage, determined by the total conventional weight of the pieces (the values ​​are taken: pawn = 1 knight = 3 bishop = 3 rook = 5 queen = 10).
k3 - the coefficient of significance.

STC: [Yellow]
LLR: -2.95 (-2.94,2.94) [0.00,5.00]
Total: 88523 W: 19994 L: 19683 D: 48846
http://tests.stockfishchess.org/tests/view/5b6b33d50ebc5902bdb9e0e1

LTC: [Green]
LLR: 2.95 (-2.94,2.94) [0.00,5.00]
Total: 14179 W: 2507 L: 2326 D: 9346
http://tests.stockfishchess.org/tests/view/5b6ba46f0ebc5902bdb9e6c6

Bench: 4645525

[GuardianRM]

It is possible that there may be some more elo here. But as it turned out it looks quite difficult to configure. If to leave "k2" as it is, probably can get 0-4 patch. When setting up k2 - there may be difficulties:
http://tests.stockfishchess.org/tests/view/5b6b4c1e0ebc5902bdb9e233
http://tests.stockfishchess.org/tests/view/5b6b9f240ebc5902bdb9e5fd

[Stefano80]

Hi @GuardianRM , congratulation to make this pass LTC!

I find the patch currently still a bit invasive. Here my remarks:

1. Would it be possible to try and replace k2by something proportional to pos.non_pawn_material?
2. Also I find GameStage not idiomatic: do you mean GamePhase?
3. Finally, looking at the non-pawn coefficients, they look to me quite quadratic. Could it be possible to use some simpler formula and get rid of some magic numbers?

[Vizvezdenec]

I think that all this suggestions @Stefano80 can be tried in future so this patch will look more pretty and compact. Usually a good way to do such stuff is to commit what we have and then try to simplify to pinpoint where exactly the elo of this idea is.
Also side note @GuardianRM - I think you should include 1st set of patches in description - I mean this one http://tests.stockfishchess.org/tests/view/5b6a54a20ebc5902bdb9d27d and this one http://tests.stockfishchess.org/tests/view/5b6a9eaf0ebc5902bdb9d5db since this patch is basically a slightly slower but bugfixed version of what already passed STC and LTC so confidence of this patch gaining elo is even higher than usual 99,75% :)
With latest not-so-good results of regression tests this is a good thing to do, IMHO :)

[NKONSTANTAKIS]

I consider this great, since the trade of speed for accuracy is almost always scaling well, both in practice and in theory. The sprts are never a proof for this, but its nice to show in the same direction.

Accurate (and more detailed/sophisticated) game phase has very strong and positive interaction with contempt, as eval (ct,ct/2) affects very much the search tree direction. Further testing could be tried for even more accuracy, and maybe long LTC tuning together with the ct/2 gradient.

ct/2 gradient so far showed to be close to optimal and very sensitive. 25%, 75% failed hard, and my suggestion of 60% also failed.
http://tests.stockfishchess.org/tests/view/5b1fb5ad0ebc5902ab9c8dd8

So imo this field offers great potential in general. If this can be done in less invasive way is something technical. For me I think its very fruitful to explore subtleties of the gamephase conditioning even further, because always some "endgames" will be better classified as middlegames, and the reverse.
By being generic, we lose this opportunity.

[Rocky640]

I think that since the formula for the k2 is additive, it can be simplified away.

For example, assuming

int bonus = (k1 - (k2 * 4)) * 16;
k2(knight) = 3;
constexpr int GameStageKnights[] = { -6, 22, 42,  8 };

we can simply have

int bonus = k1;
constexpr int GameStageKnights[] = { (-6-3*0*4)*16, (22-3*1*4)*16, (42-3*2*4)*16,  (8-3*3*4)*16 };

and similar adjustment for the other pieces.

bench should be the same

[ceebo]

@Rocky640 wrote:

I think that since the formula for the k2 is additive, it can be simplified away.

Amen to that. The coefficients in this patch are hugely redundant against the coefficients we already have.

and similar adjustment for the other pieces.
bench should be the same

Bench might not be exactly the same because computation of k1 clips when there are more than 3 knights, bishops or rooks. I refuse to believe this is significant though.

Here are more steps that could be taken to simplify. I'm not going to code or test this. Note after stage 2 my Knight values are { -6, 10, 18 } in agreement with @Rocky640. All the other arithmetic was done by hand. Expect mistakes.

Original values:
  constexpr int GameStagePawns[]   = { 20, 43, 37, 12, 1, -4, 2, 19, 32 };
  constexpr int GameStageKnights[] = { -6, 22, 42,  8 };
  constexpr int GameStageBishops[] = { -2, 26, 40, -8 };
  constexpr int GameStageRooks[]   = {  4, 24, 35, -7 };
  constexpr int GameStageQueens[]  = {-13, 54,-28 };

Step 1: Refuse to believe promoted material is significant:
  constexpr int GameStagePawns[]   = { 20, 43, 37, 12, 1, -4, 2, 19, 32 };
  constexpr int GameStageKnights[] = { -6, 22, 42 };
  constexpr int GameStageBishops[] = { -2, 26, 40 };
  constexpr int GameStageRooks[]   = {  4, 24, 35 };
  constexpr int GameStageQueens[]  = {-13, 54 };

Step 2: Move k2 contribution inside k1 as suggested by Rock640
  constexpr int GameStagePawns[]   = { 20, 39, 29, 0, -15, -24, -22, -7, 0 };
  constexpr int GameStageKnights[] = { -6, 10, 18 };
  constexpr int GameStageBishops[] = { -2, 14, 16 };
  constexpr int GameStageRooks[]   = {  4, 4, -5 };
  constexpr int GameStageQueens[]  = {-13, 14 };

Step 3: Shift so that first coeff is 0
  constexpr int GameStagePawns[]   = { 0, 19, 9, -20, -35, -44, -42, -27, -20 };
  constexpr int GameStageKnights[] = { 0, 16, 24 };
  constexpr int GameStageBishops[] = { 0, 16, 18 };
  constexpr int GameStageRooks[]   = { 0, 0, -9 };
  constexpr int GameStageQueens[]  = { 0, 27 };

Step 4: Reduce Bishop Pair bonus by 14 and compensate by adding 14 to
GameStageBishops[2]:
  constexpr int GameStagePawns[]   = { 0, 19, 9, -20, -35, -44, -42, -27, -20 };
  constexpr int GameStageKnights[] = { 0, 16, 24 };
  constexpr int GameStageBishops[] = { 0, 16, 32 };
  constexpr int GameStageRooks[]   = { 0, 0, -9 };
  constexpr int GameStageQueens[]  = { 0, 27 };

Step 5: Take 4 from QuadraticOurs[KNIGHT][KNIGHT] and compensate by
adding { 0^2 * 4, 1^2 * 4, 2^2 * 4} == { 0, 4, 16 } to GameStageKnights:
  constexpr int GameStagePawns[]   = { 0, 19, 9, -20, -35, -44, -42, -27, -20 };
  constexpr int GameStageKnights[] = { 0, 20, 40 };
  constexpr int GameStageBishops[] = { 0, 16, 32 };
  constexpr int GameStageRooks[]   = { 0, 0, -9 };
  constexpr int GameStageQueens[]  = { 0, 27 };

Step 6: Take 4 from QuadraticOurs[ROOK][ROOK] and compensate by
adding {0^2 * 4, 1^2 * 4, 2^2 * 4} == { 0, 4, 16 } to GameStageRooks:
  constexpr int GameStagePawns[]   = { 0, 19, 9, -20, -35, -44, -42, -27, -20 };
  constexpr int GameStageKnights[] = { 0, 20, 40 };
  constexpr int GameStageBishops[] = { 0, 16, 32 };
  constexpr int GameStageRooks[]   = { 0, 4, 7 };
  constexpr int GameStageQueens[]  = { 0, 27 };


Step 7: Now that the arrays for KNIGHT/BISHOP/ROOK/QUEEN are (very nearly)
linear get rid of them and compensate by adding 20/16, 16/16, 4/16
and 27/16 to the piece values in types.h

In summary it looks like we can already express all of this patch in terms of existing coefficients except for the GameStagePawns array.

[GuardianRM]

Thanks for the comments.

I agree that the construction looks cumbersome and is not against simplification.

Apparently so far it is possible to find better coefficients k1, k3, since these are the coefficients after the 1st setting. After this, it is possible to find better coefficients k2.

But it is possible that the linearity of the coefficients after the transformations may not be sufficiently linear.

[snicolet]

@Rocky640 @ceebo Nice analysis, nice work for the derivation of values.

My current plan is to try some tests with the suggested pieces values in types.h and/or the GameStagePawns[] array before deciding how to continue with the pull request.

[ceebo]

@snicolet @Rocky640 Oh, hang on... I forgot about the k3 multiplier. I think this means that my suggested changes to QuadraticOurs[X][X] should all be 16 times bigger and there should be no denominator of 16 in the changes to the piece values. Sound reasonable?

[snicolet]

what is the k3 multiplier?

[ceebo]

k3 was used by @GuardianRM in the original post:

Bonus = (k1 - k2) * k3

In the code k3 is 16 on line 108 but I forgot about it. It seems that @Rocky640 and I agree on how to get rid of the k2 part of the bonus but I forgot about k3 in my extra steps.

[ceebo]

Time for some code instead of just hand-waving. On top of this PR I made a crude debugging commit ceebo/Stockfish@263a8c2 and a commit ceebo/Stockfish@caf0e0a which I believe is very close to a non-functional change.

Evidence for this is:
a) the debugging information written by commit 1 is unchanged when commit 2 is applied.
b) the node counts from the start position in this PR and after commit 2 are very similar. They agree up to depth 13 with differences appearing at depth 14 and beyond. Hopefully this is only because abundant material has appeared in the search tree.

Further debugging idea: if any material counts are outside the normal range then use the exact same method as in this PR, otherwise use the method in commit 2. Fingers crossed benches would be identical then.

[Rocky640]

Under the (unproved) assumption that the adjustments of the values for the 2nd Q or 3rd N, B, or R are useless, I confirm that @ceebo latest values are good.

For my proof, I kept the PR values for the 2nd Q or 3rd N, B or R,
and I carried the adjustments to them as well.

a) I based GuardianRM patch on top of "Remove Condition For Passed Pawns"
https://github.com/Rocky640/Stockfish/tree/ImbGuardian
Established bench for depth 13, 20 and 24

b) https://github.com/Rocky640/Stockfish/tree/Imb1
You can follow the different steps to get to ceebo's values while preserving
the PR values for 2nd Q or 3rd N, B or R
Respective bench for depth 13, 20 and 24 were identical to a) until the very last step,
where bench for depth 13, 20 were almost identical

So I pushed two NumGames 50000 tests against master "Remove Condition For Passed Pawns"
hoping that they help put some light on the subject.

STC
http://tests.stockfishchess.org/tests/view/5b6f86a40ebc5902bdba1b02
discard the adjustments for the 2nd Q, etc
move extras to types.h

http://tests.stockfishchess.org/tests/view/5b6f879d0ebc5902bdba1b05
same, but also discard the pawn imbalances.

[locutus2]

This PR #1733 overlaps with this one (changed piece values). So one of them should be retested one top of the other.

[snicolet]

I will commit #1733 and #1732, then retest with normal SPRT runs the versions derived by Chris and Alain:

• only the new pieces values
• only the new pawn imbalances
• combinaison of pieces values and pawn imbalances

[snicolet]

And one of these versions passed STC and LTC:
http://tests.stockfishchess.org/tests/view/5b6ff6e60ebc5902bdba1d87
http://tests.stockfishchess.org/tests/view/5b702dbd0ebc5902bdba216b

Congrats to everyone involved (I will open a pull request with GuardianRM as patch author, but with full credit to Alain and Chris for the help! )

[snicolet]

The pull request #1734 having been merged, I now close this one.