Fix dynamic contempt for MultiPV

[WOnder93]

Use rootMoves[PVIdx].previousScore instead of bestValue for
dynamic contempt. This is equivalent for MultiPV=1 (bench remained the
same, even for higher depths), but more correct for MultiPV.

STC (MultiPV=3):
LLR: 2.95 (-2.94,2.94) [0.00,5.00]
Total: 2657 W: 1079 L: 898 D: 680
http://tests.stockfishchess.org/tests/view/5aaa47cb0ebc590297330403

LTC (MultiPV=3):
LLR: 2.95 (-2.94,2.94) [0.00,5.00]
Total: 2390 W: 874 L: 706 D: 810
http://tests.stockfishchess.org/tests/view/5aaa593a0ebc59029733040b

VLTC 240+2.4 (MultiPV=3):
LLR: 2.96 (-2.94,2.94) [0.00,5.00]
Total: 2399 W: 861 L: 694 D: 844
http://tests.stockfishchess.org/tests/view/5aaf983e0ebc5902a182131f

LTC (MultiPV=4, Skill Level=17):
LLR: 2.95 (-2.94,2.94) [0.00,5.00]
Total: 747 W: 333 L: 175 D: 239
http://tests.stockfishchess.org/tests/view/5aabccee0ebc5902997ff006

No functional change for MultiPV=1

[Hanamuke]

Well, that was fast !

[Kingdefender]

Aha, I had no idea dyn. contempt might not be working correctly with Multi PV even though I have been using it for a while. Seems like a good catch, thanks!

[snicolet]

Approved, under the condition that we do extra verification steps to catch any decrease of quality in the multiPV moves and MultiPV lines.

See for instance this thread in early February about a MultiPV bug not caught by a standard test in the framework: #1392

So we have to be extra careful

[Hanamuke]

I will repeat what I said last time : I think the easiest way to do that is to run MultiPV tests at a reduced ["Skill"] setting, for example 17 , so that the secondary lines are sometimes played.

[WOnder93]

@snicolet I agree. I know very little about MultiPV search (and actually chess programming in general :) but since I didn't change any inner logic of the search, just tweaked the evaluation, I think SPRT tests should cover it.

Either way, I welcome anyone to do/suggest any tests that seem relevant (@syzygy1, perhaps?). Due to the extremely low draw rate with MultiPV, I was wondering if I should also test at higher TC (say, 180+1.8) to verify that the test conclusiveness wasn't influenced by the low draw rate...

@Hanamuke Good idea, I'll try that at LTC.

[Hanamuke]

@WOnder93 Just so you know, enabling skill sets MultiPV at 4 if it was lower, so your test actually runs at MultiPV=4. Of course it's not a problem since it's the same for base and test.

[WOnder93]

@Hanamuke Oh, you're right! I resubmitted the test to avoid confusion (since the test wasn't running yet).

[Hanamuke]

@WOnder93 joergoser is probably right, it would explain the observation. Also it's normal that skill would reduce the observed Elo worth of a patch, maybe skill=19 would bring enough diversity i don't know.

[WOnder93]

Ah, so it is not just 'not needed', but 'not working', I didn't realize that... I will decrease the priority of the running test (for now) and start a new one.

[WOnder93]

OK, so once I fixed the misquoted option, the test passed really fast:

LLR: 2.95 (-2.94,2.94) [0.00,5.00]
Total: 747 W: 333 L: 175 D: 239

http://tests.stockfishchess.org/tests/view/5aabccee0ebc5902997ff006

[WOnder93]

To be on the safe side, I ran a VLTC test with MultiPV=3 at 240+2.4" and it also got green quickly:

LLR: 2.96 (-2.94,2.94) [0.00,5.00]
Total: 2399 W: 861 L: 694 D: 844

http://tests.stockfishchess.org/tests/view/5aaf983e0ebc5902a182131f

As for the concerns about breaking the PV output, I will just repeat that this patch only tweaks the dynamic contempt value, and thus any regression should be revealed by the usual SPRT tests.

@snicolet I am now waiting for your verdict :)

[snicolet]

@WOnder93
Thanks, I still would like that we do a visual inspection of the lines for MultiPV=10, comparing master and your patch at depth 12 and 24:

• from the starting position for White
• from the position after 1.e4 for Black

This is what Ronald did in #1392 to catch the bug of the previous MultiPV improvement :-)

[WOnder93]

@snicolet I don't know what exactly I should look for but at least I don't see any obvious problem in the output:

Start position, depth 12, master:

info depth 12 seldepth 17 multipv 1 score cp 47 nodes 709643 nps 984248 tbhits 0 time 721 pv e2e4 e7e5 b1c3 b8c6 g1f3 g8f6 f1b5 f8b4 e1g1 e8g8 d2d3 b4c3 b5c6 d7c6 b2c3
info depth 12 seldepth 16 multipv 2 score cp 43 nodes 709643 nps 984248 tbhits 0 time 721 pv b1c3 d7d5 d2d4 c7c5 c1f4 g8f6 e2e3 d8a5 f1b5 c8d7 b5d7 b8d7 g1f3 f6e4 e1g1
info depth 12 seldepth 16 multipv 3 score cp 30 nodes 709643 nps 984248 tbhits 0 time 721 pv d2d4 d7d5 c1f4 e7e6 e2e3 g8f6 g1f3 c7c5 f1e2 d8b6 b1c3 b6b2
info depth 12 seldepth 15 multipv 4 score cp 25 nodes 709643 nps 984248 tbhits 0 time 721 pv g1f3 g8f6 d2d4 e7e6 e2e3 d7d5 b1c3 c7c5 f1e2 f8e7 d4c5 e8g8 e1g1 b8c6
info depth 12 seldepth 17 multipv 5 score cp 22 nodes 709643 nps 984248 tbhits 0 time 721 pv e2e3 d7d5 d2d4 e7e6 b1c3 g8f6 g1f3 c7c5 f1e2 f8e7 e1g1 e8g8
info depth 12 seldepth 18 multipv 6 score cp 8 nodes 709643 nps 984248 tbhits 0 time 721 pv c2c4 e7e5 b1c3 b8c6 g1f3 f8b4 e2e4 g8f6 f1d3 d7d6 e1g1 e8g8 c3d5
info depth 12 seldepth 14 multipv 7 score cp 5 nodes 709643 nps 984248 tbhits 0 time 721 pv h2h3 e7e5 e2e4 b8c6 b1c3 g8f6 g1f3 d7d5 e4d5 f6d5 d2d4 d5c3 b2c3
info depth 12 seldepth 17 multipv 8 score cp -17 nodes 709643 nps 984248 tbhits 0 time 721 pv c2c3 e7e5 d2d4 e5e4 d4d5 c7c6 d5d6 d8f6 d1a4 f6d6 a4e4 d6e7 e4e7 g8e7 c1f4 e7d5 e2e3
info depth 12 seldepth 16 multipv 9 score cp -20 nodes 709643 nps 984248 tbhits 0 time 721 pv d2d3 d7d5 d3d4 c8f5 e2e3 e7e6 b1d2 c7c5 g2g4 f5g6 g1f3 b8c6 f1b5
info depth 12 seldepth 14 multipv 10 score cp -32 nodes 709643 nps 984248 tbhits 0 time 721 pv f2f4 g8f6 g1f3 d7d5 e2e3 e7e6 b1c3 f8e7 f1d3 e8g8 e1g1 b8c6 f3e5

Start position, depth 12, patch:

info depth 12 seldepth 18 multipv 1 score cp 58 nodes 699800 nps 961263 tbhits 0 time 728 pv e2e4 e7e5 b1c3 g8f6 g1f3 f8b4 f3e5 d7d6 e5f3 e8g8 f1c4 b4c3 d2c3 f6e4
info depth 12 seldepth 18 multipv 2 score cp 58 nodes 699800 nps 961263 tbhits 0 time 728 pv e2e3 e7e6 g1f3 d7d5 d2d4 c7c5 c2c4 g8f6 c4d5 e6d5 f1e2 c5d4 f3d4
info depth 12 seldepth 17 multipv 3 score cp 56 nodes 699800 nps 961263 tbhits 0 time 728 pv d2d4 d7d5 e2e3 e7e6 g1f3 c7c5 f1e2 b8c6 c1d2 c5c4 e1g1 g8f6 b1c3
info depth 12 seldepth 16 multipv 4 score cp 52 nodes 699800 nps 961263 tbhits 0 time 728 pv g1f3 d7d5 d2d4 g8f6 e2e3 e7e6 f1e2 c7c5 c2c4 d5c4 e1g1 c8d7 d4c5 f8c5 e2c4
info depth 12 seldepth 15 multipv 5 score cp 35 nodes 699800 nps 961263 tbhits 0 time 728 pv b1c3 d7d5 d2d4 g8f6 c1f4 c8d7 e2e3 e7e6 c3b5 d7b5 f1b5 c7c6 b5d3 f8d6 g1f3
info depth 12 seldepth 17 multipv 6 score cp -3 nodes 699800 nps 961263 tbhits 0 time 728 pv a2a3 g8f6 d2d4 d7d5 b1c3 e7e6 c1f4 f8d6 f4d6 c7d6 e2e3 e8g8
info depth 12 seldepth 17 multipv 7 score cp -3 nodes 699800 nps 961263 tbhits 0 time 728 pv c2c3 e7e5 e2e4 d7d5 e4d5 d8d5 d2d4 g8f6 d4e5 d5e5 f1e2 f8d6 g1f3 e5e7 e1g1 e8g8
info depth 12 seldepth 16 multipv 8 score cp -5 nodes 699800 nps 961263 tbhits 0 time 728 pv h2h3 e7e5 e2e4 b8c6 g1f3 g8f6 d1e2 d7d5 e4d5 d8d5 b1c3 d5c5 e2b5
info depth 12 seldepth 16 multipv 9 score cp -16 nodes 699800 nps 961263 tbhits 0 time 728 pv d2d3 d7d5 g1f3 b8c6 d3d4 c8f5 e2e3 c6b4 f1d3 b4d3 c2d3 g8f6 e1g1 e7e6
info depth 12 seldepth 17 multipv 10 score cp -18 nodes 699800 nps 961263 tbhits 0 time 728 pv c2c4 e7e5 b1c3 b8c6 g1f3 g8f6 d2d4 e5d4 f3d4 f8b4 a2a3 b4c3 b2c3 e8g8

1. e4 played, depth 12, master:

info depth 12 seldepth 17 multipv 1 score cp -53 nodes 886558 nps 894609 tbhits 0 time 991 pv e7e5 d2d4 e5d4 d1d4 b8c6 d4e3 g8f6 h2h3 d7d6 g1f3 c6e5 b1c3 c7c6 a2a3 c8d7
info depth 12 seldepth 15 multipv 2 score cp -58 nodes 886558 nps 894609 tbhits 0 time 991 pv d7d5 e4d5 c7c6 g1f3 c6d5 d2d4 g8f6 c2c3 b8c6 f1d3 e7e6 e1g1 d8b6
info depth 12 seldepth 18 multipv 3 score cp -59 nodes 886558 nps 894609 tbhits 0 time 991 pv e7e6 d2d4 d7d5 b1c3 f8b4 a2a3 b4c3 b2c3 g8e7 d1g4 e8g8 f1d3 b8c6 e4e5
info depth 12 seldepth 18 multipv 4 score cp -74 nodes 886558 nps 894609 tbhits 0 time 991 pv d7d6 d2d4 g8f6 b1c3 b8d7 g1f3 c7c6 f1d3 e7e5 e1g1 f8e7 c1e3 e8g8 d4d5 d7c5
info depth 12 seldepth 14 multipv 5 score cp -77 nodes 886558 nps 894609 tbhits 0 time 991 pv c7c6 d2d4 d7d5 e4e5 c6c5 g1f3 c5d4 c2c3 d4c3 f1b5 b8c6 b1c3 e7e6
info depth 12 seldepth 17 multipv 6 score cp -77 nodes 886558 nps 894609 tbhits 0 time 991 pv h7h6 d2d4 e7e6 b1c3 d7d5 g1f3 d5e4 c3e4 b8c6 f1b5 a7a6 b5c6 b7c6
info depth 12 seldepth 18 multipv 7 score cp -79 nodes 886558 nps 894609 tbhits 0 time 991 pv c7c5 g1f3 d7d6 d2d4 c5d4 f3d4 e7e5 d4f5 c8e6 b1c3 b8c6 f5e3
info depth 12 seldepth 20 multipv 8 score cp -92 nodes 886558 nps 894609 tbhits 0 time 991 pv h7h5 d2d4 e7e6 g1f3 d7d5 b1c3 f8b4 e4e5 c7c5 d4c5 b4c5 f1b5 c8d7 e1g1
info depth 12 seldepth 16 multipv 9 score cp -105 nodes 886558 nps 894609 tbhits 0 time 991 pv a7a6 b1c3 e7e5 g1f3 b8c6 d2d4 e5d4 f3d4 f8b4 a2a3 b4c3 b2c3 d8e7
info depth 12 seldepth 16 multipv 10 score cp -109 nodes 886558 nps 894609 tbhits 0 time 991 pv b7b6 d2d4 e7e6 g1f3 c8b7 b1c3 f8b4 f1d3 d7d5 e4e5 b4c3 b2c3 c7c5

1. e4 played, depth 12, patch:

info depth 12 seldepth 19 multipv 1 score cp -19 nodes 770298 nps 972598 tbhits 0 time 792 pv e7e5 g1f3 g8f6 d2d4 e5d4 e4e5 f6e4 f1d3 d7d5 e5d6 e4d6 d1e2 d8e7 f3d4
info depth 12 seldepth 17 multipv 2 score cp -46 nodes 770298 nps 972598 tbhits 0 time 792 pv e7e6 b1c3 g8e7 d2d4 d7d5 a2a3 d5e4 c3e4 c8d7 d1e2 d7c6 g1f3
info depth 12 seldepth 17 multipv 3 score cp -54 nodes 770298 nps 972598 tbhits 0 time 792 pv c7c5 g1f3 b8c6 f1b5 c6d4 f3d4 c5d4 e1g1 e7e5 b5a4 d8b6 d2d3 g8f6
info depth 12 seldepth 19 multipv 4 score cp -67 nodes 770298 nps 972598 tbhits 0 time 792 pv b8c6 d2d4 d7d5 b1c3 d5e4 d4d5 c6b8 c3e4 e7e6 f1b5 c7c6 d5c6 d8d1 e1d1 b7c6 b5e2
info depth 12 seldepth 17 multipv 5 score cp -70 nodes 770298 nps 972598 tbhits 0 time 792 pv d7d5 e4d5 g8f6 d2d4 c7c6 g1f3 c6d5 b1c3 e7e6 f1b5 c8d7 b5d7 f6d7 e1g1 b8c6 c1g5 d8b6
info depth 12 seldepth 17 multipv 6 score cp -78 nodes 770298 nps 972598 tbhits 0 time 792 pv c7c6 d2d4 d7d5 e4e5 e7e6 g1f3 c6c5 c2c3 b8c6 f1b5 c8d7 b5c6 d7c6 e1g1 c6b5
info depth 12 seldepth 18 multipv 7 score cp -85 nodes 770298 nps 972598 tbhits 0 time 792 pv d7d6 d2d4 g8f6 b1c3 b8d7 g1f3 e7e5 f1e2 f8e7 e1g1 e8g8 c1e3 e5d4
info depth 12 seldepth 16 multipv 8 score cp -92 nodes 770298 nps 972598 tbhits 0 time 792 pv h7h6 g1f3 e7e6 b1c3 c7c5 d2d4 c5d4 f3d4 b8c6 d4c6 b7c6 f1e2
info depth 12 seldepth 18 multipv 9 score cp -101 nodes 770298 nps 972598 tbhits 0 time 792 pv a7a6 d2d4 e7e6 d1e2 d7d6 g1f3 b8d7 c1e3 e6e5 c2c4 e5d4 f3d4 c7c5 d4f3
info depth 12 seldepth 14 multipv 10 score cp -112 nodes 770298 nps 972598 tbhits 0 time 792 pv h7h5 d2d4 e7e6 g1f3 d7d5 e4d5 e6d5 c1g5 g8f6 b1c3 d8e7 f1e2 e7b4 e1g1

Start position, depth 24, master:

info depth 24 seldepth 37 multipv 1 score cp 53 nodes 163548863 nps 823811 hashfull 999 tbhits 0 time 198527 pv d2d4 g8f6 g1f3 d7d5 c2c4 e7e6 c1g5 b8d7 e2e3 h7h6 g5f6 d7f6 b1c3 f8e7 f1e2 e8g8 e1g1 d5c4 f3e5 c7c5 e5c4 c5d4 e3d4 g8h8 e2f3
info depth 24 seldepth 32 multipv 2 score cp 38 nodes 163548863 nps 823811 hashfull 999 tbhits 0 time 198527 pv g1f3 d7d5 e2e3 e7e6 d2d4 g8f6 c2c4 f8d6 b1c3 e8g8 f1d3 d5c4 d3c4 c7c5 d4c5 d6c5 e1g1 b8c6 h2h3 d8d1 f1d1 e6e5 e3e4 c8e6 c4e6
info depth 24 seldepth 32 multipv 3 score cp 37 nodes 163548863 nps 823811 hashfull 999 tbhits 0 time 198527 pv e2e4 e7e6 d2d4 d7d5 b1c3 f8b4 e4d5 e6d5 g1f3 g8f6 f1d3 e8g8 e1g1 b8c6 a2a3 b4c3 b2c3 f8e8 h2h3 h7h6 c1b2 d8d6 f1e1 e8e1 d1e1 c8e6 f3e5 f6d7 e5c6
info depth 24 seldepth 35 multipv 4 score cp 32 nodes 163548863 nps 823811 hashfull 999 tbhits 0 time 198527 pv c2c4 e7e5 b1c3 f8b4 c3d5 b8a6 a2a3 b4a5 b2b4 c7c6 d5c3 a5c7 e2e4 g8f6 g1f3 e8g8 f1e2 h7h6 e1g1 d7d5 e4d5 c6d5 d2d4 e5e4 f3e5
info depth 24 seldepth 36 multipv 5 score cp 29 nodes 163548863 nps 823811 hashfull 999 tbhits 0 time 198527 pv h2h3 d7d5 d2d4 e7e6 e2e3 g8f6 g1f3 c7c5 c2c4 b8c6 b1c3 h7h6 a2a3 c5d4 e3d4 d5c4 f1c4 f8d6 c1e3 c8d7 c4d3 a7a6 e1g1 e8g8
info depth 24 seldepth 32 multipv 6 score cp 27 nodes 163548863 nps 823811 hashfull 999 tbhits 0 time 198527 pv e2e3 e7e5 d2d4 e5d4 e3d4 d7d5 b1c3 g8f6 g1f3 f8d6 f1d3 e8g8 e1g1 c7c6 f1e1 c8e6 f3e5 b8d7 d1f3 d8b6 f3g3 d7e5 d4e5 d6c7 g1h1
info depth 24 seldepth 31 multipv 7 score cp 20 nodes 163548863 nps 823811 hashfull 999 tbhits 0 time 198527 pv d2d3 d7d5 g1f3 e7e6 c1f4 f8d6 f4d6 d8d6 b1d2 c7c5 e2e4 g8e7 f1e2 e8g8 e1g1 c8d7 c2c3 d6b6 e4d5 e6d5 f1e1 b8c6 a1b1 a8d8
info depth 24 seldepth 32 multipv 8 score cp 12 nodes 163548863 nps 823811 hashfull 999 tbhits 0 time 198527 pv c2c3 d7d5 g1f3 g8f6 d2d4 c8f5 c1f4 e7e6 b1d2 b8d7 e2e3 f8e7 f1e2 h7h6 e1g1 e8g8 h2h3 c7c5 d1b3 d7b6 d4c5 e7c5 a2a4 a7a5 f4g3 f6d7 c3c4
info depth 24 seldepth 31 multipv 9 score cp 9 nodes 163548863 nps 823811 hashfull 999 tbhits 0 time 198527 pv a2a3 e7e5 c2c4 g8f6 b1c3 c7c6 d2d4 e5d4 d1d4 d7d5 e2e4 c8e6 c4d5 c6d5 f1b5 b8c6 c1g5 f8e7 e4e5 f6d7 b5c6 b7c6 g5e7 d8e7 g1f3
info depth 24 seldepth 37 multipv 10 score cp 8 nodes 163548863 nps 823811 hashfull 999 tbhits 0 time 198527 pv b1c3 d7d5 d2d4 g8f6 c1f4 e7e6 c3b5 f8b4 c2c3 b4a5 g1f3 a7a6 b5a3 c7c5 d4c5 e8g8 b2b4 a5c7 e2e3 b8c6 f4c7 d8c7 f1e2 e6e5 e1g1 c8e6 d1c2 h7h6 a1d1 a8d8 g1h1 f6e4 f3d2

Start position, depth 24, patch:

info depth 24 seldepth 30 multipv 1 score cp 45 nodes 144319363 nps 861530 hashfull 999 tbhits 0 time 167515 pv e2e3 e7e5 d2d4 e5d4 e3d4 d7d5 g1f3 g8f6 f1d3 f8d6 h2h3 d8e7 d3e2 c8e6 e1g1 e8g8 f1e1 e7d8 b1c3 a7a6 f3e5 c7c5 c1f4 b8c6 e5c6 b7c6 f4d6 d8d6 a2a3 c5d4
info depth 24 seldepth 34 multipv 2 score cp 45 nodes 144319363 nps 861530 hashfull 999 tbhits 0 time 167515 pv g1f3 d7d5 e2e3 c7c5 d2d4 g8f6 c2c4 e7e6 a2a3 b8c6 d4c5 f8c5 b2b4 c5d6 c1b2 d5c4 f1c4 e8g8 e1g1 a7a6 b1c3 b7b6 b4b5 c6a5 c4d3
info depth 24 seldepth 30 multipv 3 score cp 43 nodes 144319363 nps 861530 hashfull 999 tbhits 0 time 167515 pv e2e4 e7e6 d2d4 d7d5 e4d5 e6d5 g1f3 g8f6 f1d3 f8d6 h2h3 d8e7 d3e2 c8e6 e1g1 e8g8 f1e1 e7d8 b1c3 a7a6 f3e5 b8c6 c1f4 f8e8 a2a3 d6e5 d4e5 f6e4 c3e4 d5e4
info depth 24 seldepth 30 multipv 4 score cp 42 nodes 144319363 nps 861530 hashfull 999 tbhits 0 time 167515 pv d2d4 d7d5 c2c4 e7e6 g1f3 g8f6 b1c3 f8b4 c1d2 c7c5 a2a3 b4c3 d2c3 c5d4 c3d4 d5c4 e2e3 b8c6 f1c4 c6d4 e3d4 c8d7 e1g1 d7c6 f3e5 e8g8 e5c6 b7c6
info depth 24 seldepth 34 multipv 5 score cp 31 nodes 144319363 nps 861530 hashfull 999 tbhits 0 time 167515 pv c2c4 e7e5 b1c3 g8f6 g1f3 b8c6 e2e3 f8b4 d2d4 e5d4 e3d4 e8g8 f1e2 f8e8 e1g1 d7d5 a2a3 b4c3 b2c3 f6e4 e2d3 e4c3 d1c2 c3e4 f1e1 c8f5 c4d5 d8d5
info depth 24 seldepth 30 multipv 6 score cp 29 nodes 144319363 nps 861530 hashfull 999 tbhits 0 time 167515 pv b1c3 d7d5 d2d4 g8f6 c1f4 e7e6 c3b5 b8a6 e2e3 c7c6 b5c3 f8b4 f1a6 b7a6 g1e2 e8g8 e1g1 c8b7 f4g3 f6h5 g3e5 f7f6 a2a3 b4e7 e2g3 f6e5 g3h5
info depth 24 seldepth 32 multipv 7 score cp 16 nodes 144319363 nps 861530 hashfull 999 tbhits 0 time 167515 pv a2a3 e7e5 c2c4 g8f6 b1c3 b8c6 g1f3 d7d5 c4d5 f6d5 e2e4 d5f4 d2d4 e5d4 c1f4 d4c3 d1d8 c6d8 b2c3 d8e6 f4e3 f8e7 f1c4 e6c5 e3c5 e7c5 e1g1 c8g4 c4f7 e8f7 f3e5 f7f6
info depth 24 seldepth 33 multipv 8 score cp 15 nodes 144319363 nps 861530 hashfull 999 tbhits 0 time 167515 pv h2h3 e7e5 e2e3 d7d5 d2d4 b8d7 c2c4 g8f6 b1c3 e5d4 e3d4 f8b4 g1f3 e8g8 c1e3 d5c4 f1c4 d7b6 c4b3 b6d5 e1g1 b4c3 b2c3 d5c3 d1c2 f6e4 a1e1
info depth 24 seldepth 32 multipv 9 score cp 12 nodes 144319363 nps 861530 hashfull 999 tbhits 0 time 167515 pv d2d3 d7d5 g1f3 g8f6 c1f4 b8c6 d3d4 c8f5 e2e3 e7e6 f1b5 f8d6 f4d6 d8d6 b1c3 e8g8 e1g1 f5g4 h2h3 g4f3 d1f3 f8d8 b5c6 d6c6 g1h1
info depth 24 seldepth 28 multipv 10 score cp 9 nodes 144319363 nps 861530 hashfull 999 tbhits 0 time 167515 pv c2c3 e7e5 d2d4 e5e4 c3c4 c7c6 d4d5 g8f6 b1c3 f8b4 c1d2 e8g8 e2e3 b8a6 g1e2 d7d6 e2g3 c8g4 f1e2 g4e2 d1e2 f8e8 e1g1 a6c5 g3e4 c5e4 c3e4 e8e4

1. e4 played, depth 24, master:

info depth 24 seldepth 35 multipv 1 score cp -3 nodes 213388661 nps 820997 hashfull 999 tbhits 0 time 259914 pv e7e6 d2d4 d7d5 e4d5 e6d5 g1f3 g8f6 f1d3 f8d6 e1g1 e8g8 h2h3 b8c6 c2c4 d5c4 d3c4 a7a6 a2a3 b7b5 c4a2 c8b7 b1c3 f8e8 c1e3 c6a5 f3e5 d6e5 d4e5 e8e5
info depth 24 seldepth 39 multipv 2 score cp -16 nodes 213388661 nps 820997 hashfull 999 tbhits 0 time 259914 pv e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 d2d4 f8e7 d1e2 e4d6 b5c6 b7c6 d4e5 d6f5 b1c3 e8g8 f1d1 d7d5 g2g4 f5h4 f3h4 e7h4 f2f4 d5d4 f4f5
info depth 24 seldepth 36 multipv 3 score cp -27 nodes 213388661 nps 820997 hashfull 999 tbhits 0 time 259914 pv c7c5 g1f3 e7e6 b1c3 b8c6 d2d4 c5d4 f3d4 g8f6 f1e2 d7d5 e1g1 d5e4 d4c6 d8d1 f1d1 b7c6 c1g5 f8e7 a1b1 e6e5 g5h4 e8g8 h4g3 c8e6 g3e5 f8d8 a2a4 e7b4
info depth 24 seldepth 38 multipv 4 score cp -34 nodes 213388661 nps 820997 hashfull 999 tbhits 0 time 259914 pv b8c6 d2d4 d7d5 e4e5 c8f5 g1f3 e7e6 f1e2 g8e7 e1g1 e7g6 b1c3 f8b4 f3e1 g6e7 c1e3 f5g6 a2a3 b4c3 b2c3 e8g8 a1b1 b7b6 e1d3 e7f5 d3f4 f5e3 f2e3
info depth 24 seldepth 37 multipv 5 score cp -37 nodes 213388661 nps 820997 hashfull 999 tbhits 0 time 259914 pv d7d6 d2d4 g8f6 b1c3 e7e6 g1f3 f8e7 f1d3 c7c5 d4c5 d6c5 e1g1 b8c6 e4e5 f6d5 d1e2 d5c3 b2c3 d8c7 c1f4 c6a5 f3d2 b7b6 f1d1 c8b7 d2e4 e8g8 e4d6 g7g6 d6b7 a5b7 a1b1 a8d8 e2e4
info depth 24 seldepth 33 multipv 6 score cp -42 nodes 213388661 nps 820997 hashfull 999 tbhits 0 time 259914 pv a7a6 d2d4 e7e6 f1d3 d7d5 b1c3 f8b4 g1e2 d5e4 d3e4 g8f6 e4f3 e8g8 e1g1 b4d6 c1g5 b8d7 h2h3 h7h6 g5h4 e6e5 c3e4 g7g5 h4g3 f6e4 f3e4 d7f6 d4e5 f6e4 e5d6 c7d6
info depth 24 seldepth 35 multipv 7 score cp -50 nodes 213388661 nps 820997 hashfull 999 tbhits 0 time 259914 pv h7h6 d2d4 e7e6 c1e3 g8f6 b1c3 f8b4 e4e5 f6d5 e3d2 d5c3 b2c3 b4e7 d1g4 g7g6 g1f3 d7d5 f1b5 c7c6 b5d3 c6c5 d4c5 b8d7 d3b5 a7a6 b5a4 e8g8 e1g1 d7c5
info depth 24 seldepth 35 multipv 8 score cp -57 nodes 213388661 nps 820997 hashfull 999 tbhits 0 time 259914 pv g8f6 e4e5 f6d5 d2d4 d7d6 g1f3 c8f5 f3h4 f5c8 c2c4 d5b6 e5d6 e7d6 h4f3 f8e7 b1c3 e8g8 f1d3 b8c6 e1g1 e7f6 c1e3 f8e8 f1e1 c6b4 a2a3 b4d3 d1d3 g8h8 d4d5 b6d7 g1h1
info depth 24 seldepth 32 multipv 9 score cp -57 nodes 213388661 nps 820997 hashfull 999 tbhits 0 time 259914 pv c7c6 b1c3 d7d5 g1f3 d5e4 c3e4 g8f6 e4f6 e7f6 d2d4 c8g4 c2c3 f8d6 f1d3 e8g8 e1g1 b8d7 h2h3 g4h5 c1e3 h5g6 d3g6 h7g6 c3c4 f8e8 c4c5 d6b8
info depth 24 seldepth 38 multipv 10 score cp -58 nodes 213388661 nps 820997 hashfull 999 tbhits 0 time 259914 pv d7d5 e4d5 g8f6 g1f3 f6d5 d2d4 e7e6 c2c4 f8b4 c1d2 b4d2 d1d2 d5f6 b1c3 e8g8 f1e2 b7b6 e1g1 c8b7 a1d1 b8d7 f3e5 c7c5 d2e3 d8c7 e5d7 f6d7 d4d5 d7e5 a2a3 e6d5 c4d5

1. e4 played, depth 24, patch:

info depth 24 seldepth 35 multipv 1 score cp -2 nodes 167155042 nps 856524 hashfull 999 tbhits 0 time 195155 pv e7e5 g1f3 b8c6 f1b5 a7a6 b5c6 d7c6 e1g1 c8e6 c2c3 g8f6 d1e2 e6g4 d2d3 g4f3 e2f3 d8d7 b1d2 a8d8 f3g3 d7d3 g3e5 f8e7 e5c7 d3d7 c7d7 d8d7 a2a3 e8g8 f1d1
info depth 24 seldepth 37 multipv 2 score cp -12 nodes 167155042 nps 856524 hashfull 999 tbhits 0 time 195155 pv e7e6 d2d4 d7d5 b1c3 f8b4 f1d3 d5e4 d3e4 g8f6 e4f3 e8g8 g1e2 e6e5 c1g5 e5d4 d1d4 d8d4 e2d4 f8e8 e1f1 c7c6 g5f6 g7f6 c3e4 b4e7 c2c3 c8d7
info depth 24 seldepth 34 multipv 3 score cp -20 nodes 167155042 nps 856524 hashfull 999 tbhits 0 time 195155 pv c7c5 g1f3 d7d6 b1c3 g8f6 f1b5 c8d7 b5d7 b8d7 d2d4 e7e6 d4d5 f8e7 e1g1 e6d5 e4d5 e8g8 a2a4 a7a6 c1d2 b7b5 a4b5 a6b5 a1a8
info depth 24 seldepth 35 multipv 4 score cp -38 nodes 167155042 nps 856524 hashfull 999 tbhits 0 time 195155 pv b8c6 d2d4 d7d5 e4e5 c8f5 g1f3 e7e6 f1b5 g8e7 e1g1 a7a6 b5a4 b7b5 a4b3 c6a5 c2c3 a5b3 a2b3 h7h6 c1e3 e7c6 b3b4 f8e7 b1d2 e8g8 d2b3 f5g6 f1e1
info depth 24 seldepth 28 multipv 5 score cp -40 nodes 167155042 nps 856524 hashfull 999 tbhits 0 time 195155 pv d7d6 d2d4 g8f6 b1c3 b8d7 g1f3 e7e5 f1e2 f8e7 e1g1 e8g8 a2a4 e5d4 f3d4 d6d5 c3d5 f6e4 c1f4 e7d6 f4d6 e4d6 e2d3 d7f6 d5f6 d8f6
info depth 24 seldepth 31 multipv 6 score cp -41 nodes 167155042 nps 856524 hashfull 999 tbhits 0 time 195155 pv c7c6 d2d4 d7d5 b1c3 d5e4 c3e4 c8f5 e4g3 f5g6 g1f3 g8f6 f1d3 e7e6 e1g1 b8d7 c2c4 g6d3 d1d3 f8d6 f1e1 e8g8 c1e3 d6c7 e3d2 c6c5 d4d5
info depth 24 seldepth 37 multipv 7 score cp -50 nodes 167155042 nps 856524 hashfull 999 tbhits 0 time 195155 pv h7h6 d2d4 e7e6 b1c3 d7d5 f1d3 g8f6 e4e5 f6d7 c3e2 c7c5 c2c3 b8c6 g1f3 c5c4 d3c2 f8e7 e1g1 b7b5 c1e3 b5b4 e2f4 c8b7 f4h5 e8g8 a1c1 a7a5 h2h3 g8h8
info depth 24 seldepth 36 multipv 8 score cp -50 nodes 167155042 nps 856524 hashfull 999 tbhits 0 time 195155 pv a7a6 d2d4 e7e6 b1c3 d7d5 f1d3 g8f6 e4e5 f6d7 c3e2 c7c5 c2c3 h7h6 g1f3 b8c6 e1g1 b7b5 c1d2 c8b7 h2h3 f8e7 a1c1 b5b4 c3b4 c5c4 d3b1 c6b4 d1a4 b4c6
info depth 24 seldepth 35 multipv 9 score cp -64 nodes 167155042 nps 856524 hashfull 999 tbhits 0 time 195155 pv d7d5 e4d5 d8d5 g1f3 g8f6 d2d4 e7e5 f3e5 b8c6 e5f3 c8g4 f1e2 c6d4 d1d4 g4f3 d4e3 f3e4 b1c3 d5e6 c3e4 f6e4 f2f3 e4f6 e3e6 f7e6 c1e3 f8d6 e1g1 e8f7 e2d3
info depth 24 seldepth 44 multipv 10 score cp -64 nodes 167155042 nps 856524 hashfull 999 tbhits 0 time 195155 pv g8f6 e4e5 f6d5 d2d4 d7d6 c2c4 d5b6 g1f3 c8g4 e5d6 c7d6 f1e2 e7e6 h2h3 g4f3 e2f3 b8c6 d4d5 e6d5 c4d5 c6e5 f3e2 f8e7 e1g1 e8g8 b1c3 e7f6 f2f4

Full output attached for further inspection:
multipv-12-e4-master.log
multipv-12-e4-patch.log
multipv-12-sp-master.log
multipv-12-sp-patch.log
multipv-24-e4-master.log
multipv-24-e4-patch.log
multipv-24-sp-master.log
multipv-24-sp-patch.log

[snicolet]

@WOnder93 Thanks!

[WOnder93]

FYI, this fishcooking thread made me curious so I ran a few more tests around dynamic contempt & MultiPV.

It turns out that dynamic contempt (as implemented in master) is a significant regression for MultiPV (test). With the fix in this PR, it changes from regression to improvement (test).

The same holds for multithreaded MultiPV (test for master, test for this PR).

I found another significant improvement in multi-threaded MultiPV (per-thread dynamic contempt) – I will submit it in a separate PR after this one is merged.

Note: although the ELO differences seem huge, they are probably inflated by the nature of Skill Level / MultiPV search, so I don't think they can be reasonably compared with classic ELO strength.

[snicolet]

Merged via 367304e, thanks!