Remove nmpColor

[dubslow]

STC: https://tests.stockfishchess.org/tests/view/642e293977ff3301150e9b55
LLR: 2.94 (-2.94,2.94) <-1.75,0.25>
Total: 161848 W: 43332 L: 43254 D: 75262
Ptnml(0-2): 418, 16987, 46058, 17021, 440

LTC: https://tests.stockfishchess.org/tests/view/642fea9420eb941419bde296
LLR: 2.95 (-2.94,2.94) <-1.75,0.25>
Total: 120208 W: 32529 L: 32418 D: 55261
Ptnml(0-2): 35, 11424, 37080, 11525, 40

bench 3672914

[peregrineshahin]

I think we need to know if this is even a functional change then if it is test some zugzwang positions probably.

[dubslow]

I assume that it's functional, just not in the bench positions because bench depth is low and because bench doesn't have positions which contain zugzwangs, which are quite rare indeed from a practical point of view. Obviously, if it isn't functional, then this is dead code anyways, so let's assume it is functional.

The effect of this patch is to do stricter verification searches. In master, when performing a verification search, we actually do allow the opposing player to do null move pruning, i.e. we assume that if one side is in zugzwang in a given position, then the opponent surely cannot be in zugzwang a few ply down the road. Removing this code removes this assumption, and thus actually catches more zugzwangs than master.

On that basis, I think it's fine to merge as-is, tho if someone were to bother to test the patch on a few zugzwangs, I certainly wouldn't mind seeing the data.

[peregrineshahin]

Okay makes sense, I think now this going in the correct direction, less nmp (if possible) is indeed a preferable outcome in general.

[vondele]

There is no need to assume it is functional. You can run a deep bench, and see that it is functional.

As others have expressed, there is some reluctance to remove NMP verification code, this was added for a reason. I think the old PRs that introduced this code had example FENs. It would be interesting to see how it performs.

[dubslow]

As others have expressed, there is some reluctance to remove NMP verification code

Well then it's a good thing this patch does the opposite and does more verification

(indeed removing verification altogether also passed nonreg, but the consensus is that that's not worth submitting)

[vondele]

Ah right, mixed the two patches

[pb00068]

I can do some verify with the extended Zugzwang-Allgeuer.epd which I used when introducing nmpColor
#1338
However in my opinion the patch is OK, because I think the change it's completely neutral in detecting zugzwang
for following reasons:

• before PR1338 verification search allowed null moves on both sides,
  so it solved basic zugwang motivs but failed to resolve situatiations where the zugwang consists in a serie of worsening moves.
  A classical situation is where one side is forced to push his pawns before getting forced to do the decisive bad move.
• the reason I introduced nmpColor was just to limit the growth of the verification search tree.
  Since PR1338 does forbid null moves for several plies the verification search tree became bigger.
• Here to say are also 2 things:
  over the time tuning patches shifted the minimal depth 12 for verification up to 14,
  this is a reason why nowadays a change in verification hardly shows any effect.
  The nmp conditions habe been strictened over time, so the occurences where SF would put a null move on nmpColor are rare nowadays.
• reciprocal zugzwang situations are usually solved easily because the forced bad move occurs immediately.
• I don't know a single position where one side is in zugzwang (not basic) then the other is in zugzwang a few plies down the road.
• to me it just seems that the overall branching cost for nmpColor (was executed in almost each node) is bigger than searching a slightly bigger verification tree from time to time.

[Vizvezdenec]

Intuitively this should make verification search better. And if it costs no elo it's all good.

[pb00068]

Intuitively this should make verification search better. And if it costs no elo it's all good.

As I expected master and patch do perform equally on Zugzwang-Allgeuer.epd (singlecore, 256m hash, max. 25 seconds per problem)
Master resolved 35/42
Patch resolved 35/42

Interesting is that patch solves systematically problem nr. 23 (8/8/8/1B6/6p1/8/3K1Ppp/3N2kr w - - bm f4)
in 2 seconds while master requires about 20 seconds.

Here the report of the analysys mady using Arena 3.5.1:

Analysis of C:\Users\pb00067\Documents\Zugzwang-Allgeuer.epd   
Analyzing engine: Stockfishmaster
4/11/2023 11:36:05 AM Level: 25 Seconds
   Source                    : EPD-Positions
   Colours                   : White moves=True, Black moves=True
   Direction                 : forward
   Positions (EPD)           : From 1 to 999999 
   Use Engine(s)             : List
      Stockfishmaster
      Stockfish
   Use seconds per move      : 25 Seconds per move
   Analysis Lines            : Minimum search depth=2
   Activate abort analysis   : True. Minimal analysis depth 9. Abort, if the correct move is held 3 additional ply
  1) Rc5-c1, Rc5-c2, Rc5-c3, Rc5-c4, Rc5-c6  Rc5-c1  * 0 Seconds
  2) Rh7-h3, Rh7-h8, Rh7-e7, Rh7-f7, Rh7-g7, Rh7-h1  Rh7-h3  * 0 Seconds
  3) Ke4-d5             Ke4-d5  * 0 Seconds
  4) Kb6-c6             Kb6-c6  * 0 Seconds
  5) Ne5-g4             Ne5-g4  * 0 Seconds
  6) a5-a6              a5-a6   * 0 Seconds
  7) Kc3-b2, Kc3-b2     Kc3-b2  * 0 Seconds
  8) Ne7-c6             Ne7-c6  * 0 Seconds
  9) Rc5-c4             Rc5-c4  * 0 Seconds
 10) g6-g7              g6-g7   * 6 Seconds
 11) Qd4-d3             Qd4-d3  * 0 Seconds
 12) Re6-e1             Re6-e1  * 0 Seconds
 13) Bf3-d1, Bf3-e2     Bf3-g2   
 14) Qg3xg2             Qg3xg2  * 0 Seconds
 15) .. Nc7-e8          Nc7-e8  * 0 Seconds
 16) Rf4-f7             Rf4-f7  * 0 Seconds
 17) .. Nf4xd5          Nf4xd5  * 0 Seconds
 18) Qh3xh4             Qh3xh4  * 2 Seconds
 19) Qa6-c8             Qa6-d6   
 20) Bd8-c7             g6xh7    
 21) .. Kg8-h7          Kg8-h7  * 0 Seconds
 22) Ke2-d3, Ke2-e3     Ke2-e3  * 0 Seconds
 23) f2-f4              f2-f4   * 20 Seconds
 24) Be8-a4             Be8-a4  * 0 Seconds
 25) Qg8-g7             Qg8-g7  * 0 Seconds
 26) h2-h3              h2-h3   * 0 Seconds
 27) Qf2-a7             Qf2-a7  * 5 Seconds
 28) Nd1-c3             Nd1-c3  * 2 Seconds
 29) Bg2-e4             Bg2-e4  * 0 Seconds
 30) Nh2-f3             Nh2-f3  * 0 Seconds
 31) Rc1xc3             Rc1xc3  * 0 Seconds
 32) Kb7-a8             Kb7-a8  * 0 Seconds
 33) Kc6-d5, Bc4-b5     Bc4-b5  * 3 Seconds
 34) .. d6-d5           d6-d5   * 0 Seconds
 35) Kf6-e6, Rb5-g5     Kf6-e6  * 0 Seconds
 36) Kc2-b3, Rf1-f5     Kc2-b3  * 0 Seconds
 37) Kh7-g8             Kh7-g8  * 1 Second
 38) Na5-b3             Nd4xc2   
 39) Be2-h5             Nh3xf2   
 40) Be1-h4             Be1-h4  * 4 Seconds
 41) b2-b4, b2-b4, c2-b3  Ra2xa5   
 42) Bd4-h8, Bd4-h8, Ka1-d1  Bd4-g7   
35 of 42 matching moves
4/11/2023 11:41:49 AM, Total time: 12:05:44 AM Rated time: 03:38 = 218 Seconds

--------------------------------------------------------------------------------

Analysis of C:\Users\pb00067\Documents\Zugzwang-Allgeuer.epd   
Analyzing engine: Stockfish
4/11/2023 11:36:05 AM Level: Tournament Game in 5 Minutes
   Source                    : EPD-Positions
   Colours                   : White moves=True, Black moves=True
   Direction                 : forward
   Positions (EPD)           : From 1 to 999999 
   Use Engine(s)             : List
      Stockfishmaster
      Stockfish
   Use seconds per move      : 25 Seconds per move
   Analysis Lines            : Minimum search depth=2
   Activate abort analysis   : True. Minimal analysis depth 9. Abort, if the correct move is held 3 additional ply
  1) Rc5-c1, Rc5-c2, Rc5-c3, Rc5-c4, Rc5-c6  Rc5-c1  * 0 Seconds
  2) Rh7-h3, Rh7-h8, Rh7-e7, Rh7-f7, Rh7-g7, Rh7-h1  Rh7-h3  * 0 Seconds
  3) Ke4-d5             Ke4-d5  * 0 Seconds
  4) Kb6-c6             Kb6-c6  * 0 Seconds
  5) Ne5-g4             Ne5-g4  * 0 Seconds
  6) a5-a6              a5-a6   * 0 Seconds
  7) Kc3-b2, Kc3-b2     Kc3-b2  * 0 Seconds
  8) Ne7-c6             Ne7-c6  * 0 Seconds
  9) Rc5-c4             Rc5-c4  * 0 Seconds
 10) g6-g7              g6-g7   * 1 Second
 11) Qd4-d3             Qd4-d3  * 0 Seconds
 12) Re6-e1             Re6-e1  * 0 Seconds
 13) Bf3-d1, Bf3-e2     Bf3-g2   
 14) Qg3xg2             Qg3xg2  * 0 Seconds
 15) .. Nc7-e8          Nc7-e8  * 0 Seconds
 16) Rf4-f7             Rf4-f7  * 0 Seconds
 17) .. Nf4xd5          Nf4xd5  * 9 Seconds
 18) Qh3xh4             Qh3xh4  * 2 Seconds
 19) Qa6-c8             Qa6-d6   
 20) Bd8-c7             g6xh7    
 21) .. Kg8-h7          Kg8-h7  * 0 Seconds
 22) Ke2-d3, Ke2-e3     Ke2-e3  * 0 Seconds
 23) f2-f4              f2-f4   * 2 Seconds
 24) Be8-a4             Be8-a4  * 0 Seconds
 25) Qg8-g7             Qg8-g7  * 0 Seconds
 26) h2-h3              h2-h3   * 0 Seconds
 27) Qf2-a7             Qf2-a7  * 6 Seconds
 28) Nd1-c3             Nd1-c3  * 3 Seconds
 29) Bg2-e4             Bg2-e4  * 0 Seconds
 30) Nh2-f3             Nh2-f3  * 0 Seconds
 31) Rc1xc3             Rc1xc3  * 0 Seconds
 32) Kb7-a8             Kb7-a8  * 0 Seconds
 33) Kc6-d5, Bc4-b5     Bc4-b5  * 3 Seconds
 34) .. d6-d5           d6-d5   * 0 Seconds
 35) Kf6-e6, Rb5-g5     Kf6-e6  * 0 Seconds
 36) Kc2-b3, Rf1-f5     Rf1-f5  * 0 Seconds
 37) Kh7-g8             Kh7-g8  * 1 Second
 38) Na5-b3             Nd4xc2   
 39) Be2-h5             Nh3xf2   
 40) Be1-h4             Be1-h4  * 2 Seconds
 41) b2-b4, b2-b4, c2-b3  Ra2xa5   
 42) Bd4-h8, Bd4-h8, Ka1-d1  Bd4-g7   
35 of 42 matching moves
4/11/2023 11:47:12 AM, Total time: 12:11:07 AM Rated time: 03:48 = 228 Seconds

[pb00068]

I made a deep bench ( bench 256 1 24) to measure how often a null-move now will get allowed for the other side to move in regard to current master.
This by removing the condition on line 805 and adding
if (ss->ply < thisThread->nmpMinPly ) dbg_hit_on(us != thisThread->nmpColor); on line 808

Result:
Total 46115 Hits 44 Hit Rate (%) 0.0954136

So only in 0.09% of verification nodes we will now insert a null-move where current master does not.
Thus nowadays, if the nmp conditions fit for one side to move, they are already really prohibitive for the other side to move.
Patch is good.