Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove KNPK endgame #2611

Closed
wants to merge 3 commits into from

Conversation

protonspring
Copy link

This is a functional simplification that removes the KNPK endgame.

In more than 100k local KNPK games, there is no discernible difference between master and master with this endgame removed: master:42971, patch:42973, draws: 3969. Removal does not seem to regress in normal games.

STC
LLR: 2.94 (-2.94,2.94) {-1.50,0.50}
Total: 46390 W: 8998 L: 8884 D: 28508
Ptnml(0-2): 707, 5274, 11163, 5300, 751
https://tests.stockfishchess.org/tests/view/5e83b18ee42a5c3b3ca2ef02

LTC
LLR: 2.94 (-2.94,2.94) {-1.50,0.50}
Total: 44768 W: 5863 L: 5814 D: 33091
Ptnml(0-2): 251, 3918, 14028, 3905, 282
https://tests.stockfishchess.org/tests/view/5e84a82a4411759d9d0984f4

Bench 5147355

protonspring added 3 commits March 31, 2020 15:08
@vondele
Copy link
Member

vondele commented Apr 2, 2020

No real difference testing locally either, for a set of endgames that might lead to KNPvK:

Score of patch vs master: 6131 - 6188 - 27681  [0.499] 40000
Elo difference: -0.5 +/- 1.9, LOS: 30.4 %, DrawRatio: 69.2 %

@protonspring
Copy link
Author

Hmmm. . so there is something. I assume these are KNPsvKPs positions?
Do you have your book that I can verify?

@vondele
Copy link
Member

vondele commented Apr 2, 2020

The result is quite consistent with 0 Elo... however, that's the book (basically everything that matched KN?PvK?.epd KN??PvK??.epd) https://www.dropbox.com/s/v81tb5ok1sr7f2q/book_KNPvK_andmore.epd.zip?dl=0 TC=1.0+0.01

@protonspring
Copy link
Author

protonspring commented Apr 2, 2020

This book is testing very poorly for this patch.
TC2+0.02: patch vs master = 98 - 114 (740).

Should I close this?

EDIT: I believe I had accidentally run patch in debug mode. Better results below.

@vondele
Copy link
Member

vondele commented Apr 2, 2020

I'd leave test running for a while... just to make sure this is not noise (cutechess-cli has these nice error bars on results, they are worth reporting...). I'm guessing this is 0 within error bars.

However, I think there might be an important point to make. The endgame code is not only about how one plays the endgame, but also what the balance is between going into an endgame or staying out of it. So, I think one needs to do endgame code testing with 'nearby' endgames.

@protonspring
Copy link
Author

protonspring commented Apr 2, 2020 via email

@protonspring
Copy link
Author

protonspring commented Apr 2, 2020

Sry. . I gotta use the computer for other things. Current results are:
patch vs master: 655-689-3505. It looks like this endgame does at least something. Closing.

EDIT: I wasn't running clean versions. better results below.

@vondele
Copy link
Member

vondele commented Apr 2, 2020

And here, using 2+0.2TC:
Score of patch vs master: 5740 - 5741 - 28519 [0.500] 40000
Elo difference: -0.0 +/- 1.8, LOS: 49.6 %, DrawRatio: 71.3 %

@protonspring
Copy link
Author

Hmm. . maybe I screwed something up. I will retry when I get another spare computer and let it run overnight.

@protonspring protonspring reopened this Apr 2, 2020
@protonspring
Copy link
Author

After about 5k games: master vs patch: 785 - 776 - 3657.

I guess it looks like this is safe to remove.

@vondele vondele closed this in 3cb1c6c Apr 5, 2020
@vondele
Copy link
Member

vondele commented Apr 5, 2020

Thanks!

ddugovic pushed a commit to ddugovic/Stockfish that referenced this pull request Apr 5, 2020
In more than 100k local KNPK games, there is no discernible difference between master and master with this endgame removed: master:42971, patch:42973, draws: 3969. Removal does not seem to regress in normal games.

STC
LLR: 2.94 (-2.94,2.94) {-1.50,0.50}
Total: 46390 W: 8998 L: 8884 D: 28508
Ptnml(0-2): 707, 5274, 11163, 5300, 751
https://tests.stockfishchess.org/tests/view/5e83b18ee42a5c3b3ca2ef02

LTC
LLR: 2.94 (-2.94,2.94) {-1.50,0.50}
Total: 44768 W: 5863 L: 5814 D: 33091
Ptnml(0-2): 251, 3918, 14028, 3905, 282
https://tests.stockfishchess.org/tests/view/5e84a82a4411759d9d0984f4

In tests with a book of endgames that can convert into KNPK, no significant difference can be seen either

```
TC 1.0+0.01
Score of patch vs master: 6131 - 6188 - 27681  [0.499] 40000
Elo difference: -0.5 +/- 1.9, LOS: 30.4 %, DrawRatio: 69.2 %

TC 2.0+0.02
Score of patch vs master: 5740 - 5741 - 28519 [0.500] 40000
Elo difference: -0.0 +/- 1.8, LOS: 49.6 %, DrawRatio: 71.3 %
``

closes official-stockfish#2611

Bench 4512059
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants