Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JannLee accepted the crazyhouse challenge ! #147

Closed
Vinvin20 opened this issue Dec 1, 2016 · 45 comments
Closed

JannLee accepted the crazyhouse challenge ! #147

Vinvin20 opened this issue Dec 1, 2016 · 45 comments

Comments

@Vinvin20
Copy link

Vinvin20 commented Dec 1, 2016

The match will be play on Lichess, Saturday 6pm GMT.
Conditions : 10 games, time control 3m+2s.

I'll manually operate the latest SF-zh on my computer (6*4GHz) inside Chrome, using a couple of moves from Lichess opening book.
I'll manage the time myself : around 10 seconds/move in the opening and around 5 sec/move in the middlegame.

JannLee account : https://fr.lichess.org/@/JannLee
My account : https://fr.lichess.org/@/Vinvin

For information , JannLee beat SF-zh (0.5 sec / 1 core) , score 8-2 , 3 weeks ago, video here : https://www.youtube.com/watch?v=VTNAHmlQDXA

@ianfab
Copy link
Collaborator

ianfab commented Dec 1, 2016

Thanks for arranging this, it will be very interesting to see how it will go. If there is no bug/regression, I think Stockfish will crush him, because the improvements since last time and the longer time control should amount to 400-600 Elo compared to the 8-2 defeat. Unfortunately, I won't be able to watch it live (ironically, because I will be playing chess), but I am looking forward to watching the games on lichess or youtube afterwards.

How many nodes per second do you get with your machine (e.g. in the starting position)?

@Vinvin20
Copy link
Author

Vinvin20 commented Dec 1, 2016

How many nodes per second do you get with your machine (e.g. in the starting position)?

Around 4900-5000 knodes/s

@ianfab
Copy link
Collaborator

ianfab commented Dec 1, 2016

Thank you. Well, this should be more than enough, but let's see...

@isaacl
Copy link

isaacl commented Dec 2, 2016

pnacl performance is significantly lower than a native build. Why not type the moves or use a uci board to run stockfish natively?

But if you insist on chrome, I'd suggest Chrome Canary. There's a performance improvement that hasn't reached stable channel yet.

@Vinvin20
Copy link
Author

Vinvin20 commented Dec 2, 2016

pnacl performance is significantly lower than a native build.

Last time I check with Chess, SF was around 30% slower on PNACl (that's around -30 Elo), not a big lost.

Why not type the moves or use a uci board to run stockfish natively?

I didn't found an easy way to set up Stockfish in a graphical interface ( Winboard ? ) yet.

@Vinvin20
Copy link
Author

Vinvin20 commented Dec 2, 2016

Stream will be here : https://www.youtube.com/watch?v=wkowG9AmGUs

@ianfab
Copy link
Collaborator

ianfab commented Dec 2, 2016

@Vinvin20 XBoard with the most recent UCI2WB adapter works well for me with Stockfish, so I think it should also work in Winboard. You can even play variants that are not supported by XBoard itself, but by Stockfish (like horde chess).

@Vinvin20
Copy link
Author

Vinvin20 commented Dec 2, 2016

Is there some doc to explain step-by-step how to interface SF-zh and WB ?
I will be quite busy until the match.

@sf-x
Copy link

sf-x commented Dec 2, 2016

Winboard-AA package from H.G.M. works out of the box for me (after replacing engine with 32-bit one).. See Stalkchess forum.

@Vinvin20
Copy link
Author

Vinvin20 commented Dec 2, 2016

I would want to get a compilation of SF-zh for Windows 64Bits too ...

@sf-x
Copy link

sf-x commented Dec 2, 2016

#131 (comment)

@Vinvin20
Copy link
Author

Vinvin20 commented Dec 2, 2016

Thanks, I got it !

@ddugovic
Copy link
Owner

ddugovic commented Dec 3, 2016

I'll try to figure out how to compile bee468b for Windows from my Linux PC... I've spent hours before with no luck.

As far as I know, latest "stable" release tested at 120s + 1.2s and at ? + ?s is bee468b.

@scchess
Copy link

scchess commented Dec 3, 2016

What's the best GUI for SF crazyhouse?

@ddugovic
Copy link
Owner

ddugovic commented Dec 3, 2016

Good question. The only compliant GUI I'm aware of is http://lichess.org/ as I have not tested whether WinBoard now supports UCI.

I've heard something about CuteChess but I am unfamiliar.

@scchess
Copy link

scchess commented Dec 3, 2016

That's bad because it's not easy to hook up a new variant engine with the web server... Is there anything we can do?

@ddugovic
Copy link
Owner

ddugovic commented Dec 3, 2016

Yes, I'd appreciate help figuring out, "Does Stockfish work with CuteChess?" However, CuteChess' readme states:

The GUI doesn't have a binary release at this time.

so it looks like simply installing CuteChess could be a struggle for non-developers.

Alternatively, maybe it's possible to connect cutechess-cli to my FICS clone at relaychess.org:5000 and use WinBoard-4.5TM.exe GUI to connect to the same server and play the games that way.

@alwey
Copy link

alwey commented Dec 3, 2016

@ddugovic: Which platform (OS) is to be used for cutechess / cutechess-cli?

@scchess
Copy link

scchess commented Dec 3, 2016

@alwey cutechess-cli has binary for all major platforms. But the GUI doesn't work. We have a framework for playing automatic matches, but very hard for a human player to try the engine.

@ianfab
Copy link
Collaborator

ianfab commented Dec 3, 2016

XBoard/Winboard usually works well with Stockfish as explained in this comment. For testing, however, I do not want to rely on a GUI and hence use scripts.

@ianfab
Copy link
Collaborator

ianfab commented Dec 3, 2016

Even though 90% of the discussion in the last weeks is about crazyhouse, it is only one of many supported variants and for convenience I want to have one solution for all with regard to testing patches. That is why have not switched to CuteChess yet, although I have tried it once for crazyhouse and it worked well.

To avoid confusion: This post refers to a deleted comment.

@alwey
Copy link

alwey commented Dec 3, 2016

@ianfab : I am just working on cutechess support for Threecheck, Horde (both ready for PR and review: today or tomorrow), Suicide, Giveaway, Anti (these still have some issues).

Using own scripts is good, cutechess might then be an additional tool.

@ianfab
Copy link
Collaborator

ianfab commented Dec 3, 2016

@alwey Thanks, that is good news.

@alwey
Copy link

alwey commented Dec 3, 2016

@ianfab : You are welcome. I will have to discuss with maintainers of cutechess on one hand and with you and the other developers here some details (regarding compatibility, extensibilty). For example there may be a variant name mismatch threecheck and 3check. Cutechess currently is all too strict by requiring all engines to use exactly the same variant name. This is a known open issue in the list.

@ianfab
Copy link
Collaborator

ianfab commented Dec 3, 2016

There have been some discussions on FENs, move strings, variant names, UCI option names, etc. and there is a consensus with @HGMuller on most parts with respect to compatibility to XBoard/Winboard. Since there are only very few standards for variants, compatibility between different engines and GUIs often is difficult to achieve.

@alwey
Copy link

alwey commented Dec 3, 2016

@ianfab: Exactly: I followed the discussion with @HGMuller. He suggested to use an extended FEN format for three-check with reverse counters after the en passant field (like 3+3). To use backward counters is a clever idea. Stockfish uses this type of FEN now. Lichess uses forward counters appended to the FEN (like +0+0). I used the backwards counters (3+3) but I suggest to put them behind the standard FEN so you have upwards compatibility with standard chess data. You can use standard data, books etc. The cutechess reader can read Lichess format, HGM/Stockfish format and this proposed cutechess work format.

@ddugovic
Copy link
Owner

ddugovic commented Dec 3, 2016

@Vinvin20 I'm glad to see you have it; I'm testing WinBoard right now using instructions from #151 which suggest there is a problem! (Sorry my attention is somewhat divided between 14 open issues.)

@HGMuller
Copy link

HGMuller commented Dec 3, 2016 via email

@Vinvin20
Copy link
Author

Vinvin20 commented Dec 3, 2016

The time control was changed just after first game (lost on time) to 5m+15s.
SF-zh won 6-0 !
Video here : https://www.youtube.com/watch?v=uP3nHtK2ndw

I finally used the PNaCl (Chrome) version. Not enough time to test .exe and WB interface.

@alwey
Copy link

alwey commented Dec 3, 2016

@HGMuller Hello, regarding 3check FEN counter field: You suggested to put the reverse counters after the en passant field. Would it not be better to append the field to the standard FEN (compatibility with older data from standard variant)? I am just implementing three-check for cutechess (actually N-check).

@HGMuller
Copy link

HGMuller commented Dec 3, 2016 via email

@alwey
Copy link

alwey commented Dec 4, 2016

@HGMuller : Thank you very much. This rationale is a good guide. I asked you because the earlier these topics are discussed the easier to avoid incompatible implementations and double work.

@HGMuller
Copy link

HGMuller commented Dec 4, 2016 via email

@alwey
Copy link

alwey commented Dec 4, 2016

@HGMuller Thank you for these observations.

First I have to say that I am not owner of the cutechess project. I am an interested user, a contributor. I have no deep insights into most parts of this project. I am contributing when I feel that a functionality is missing or when I found a problem. Also I am a quite lazy guy, so I use as much of functionality and code and symmetries as possible.

The nature of cutechess is twofold. At the core there is a library. Two applications make use of this library: The CLI (cutechess-cli for engine-engine matches) and the GUI (cutechess supporting engine and human players). The library currently supports UCI engines, xboard v2 and xboard v1 engines.

The two applications work as arbiters and check move legality. They do not trust engines and they do not trust players. When cutechess says a game is lost then it is lost. The GUI has a helper function to indicate legal moves for a position.

The GUI also can import games and positions and maintain a game database. So cutechess has to deal with external data and older data from various sources.

If cutechess imports a game, e,g, a lichess.org RacingKings game it extracts the variant name and knows which rules to apply. RacingKings - no checks allowed, setup without pawns, finishing on eighth rank wins with one chance for black to get level. I have not implemented more than this.

Sometimes code reuse does not work immediately, e.g. Horde Chess has no white king. This is illegal in chess. White is dead. I can not enter a Horde starting position, Cutechess will check for the king fields - game over. So I have to generalise the king assertion in the base classes (WesternBoard in this case) and prepare an override to give Horde Chess a chance. Override the assertion with WhiteKingsCount == 0; BlackKingsCount == 1. But wait - there is Dunsany's Chess. The main difference is the starting position. And there is the original Horde Chess with a black horde, so generalise again: WhiteKingsCount + BlackKingsCount == 1.

Every time such a generalisation enters the base classes this is an example for the general chess you mentioned. But: cutechess is an arbiter, and it will check rules not in general but in detail for a variant.
So some changes will enter the base classes but some specialities will then enter the variant classes.
Which part enters the base and which enters the variant is a case of negotiation with the project owners, @ilaripih and @artoj. The check counting rule (3check) would be an example for this.

So cutechess difficulties /wrt to some of your suggestions arise from some of its base functionalities:

  • arbiter function (rules check and track game status)
  • human interaction
  • import, export, storage and presentation of game data

Best regards - an Xboard fan.

@ilaripih
Copy link

ilaripih commented Dec 4, 2016

Thanks @alwey that's actually a very accurate description of the CuteChess internals. The main point is that CuteChess must fully understand the variant, and variant support must not be dependent on engines; otherwise these variants would be second-class citizens in several ways.

But these are implementation details that don't need to be standardized. PGN/FEN is another matter, and @HGMuller makes a good point about the ply counter and move number that I hadn't thought of - they should be the last two tokens in any FEN because they can be omitted. In CuteChess this probably means that the base class FEN methods may need some refactoring to make it easy to support new variants via subclassing, but it shouldn't be that difficult.

And the requirement that all engines must use the same variant names is an easily solved problem.

@HGMuller
Copy link

HGMuller commented Dec 4, 2016 via email

@ilaripih
Copy link

ilaripih commented Dec 5, 2016

Isn't that an exaggeration? If a variant only differs from orthodox
Chess by having a different initial position, either a loaded PGN game
would provide the position through its FEN tag, or the engine could
provide it through the setup command. Everything else would work exactly
as in a variant the engine did know. So what would be 'second class'?

I don't think it's an exaggeration because adding new variants to CuteChess is usually quite easy and doesn't require a lot of code. To add a new variant that only differs from FIDE chess by name and starting FEN, this is enough:

class LIB_EXPORT FooBarBoard : public StandardBoard
{
	public:
		// Inherited from StandardBoard
		virtual Board* copy() const { return new FooBarBoard(*this); }
		virtual QString variant() const { return "foobar"; }
		virtual QString defaultFenString() const { return "foobar_fen"; }
};

Also, the "no King = finished game" rule seems a rather
counter-productive deviation from FIDE rules. According to the latter
you can only lose (except for externel reasons such as clocks or
misbehavior) when you are in check, and if you have no King you
obviously cannot be in check. No variant I know has the "no King = lost"
rule. Several have no King, however. (And in some variants capturing a
King = win.) WinBoard unintentionally applied the "no King = lost" rule
too, but I changed that to make it more generally useful.

There's really no "no king = finished game" rule in CuteChess. It's rather "white and/or black doesn't have exactly one king in the given FEN => reject FEN as invalid" rule.

I am not against this, but OTOH I did not think it is my place to
enforce XBoard (CECP) standards on UCI.

I actually meant that CuteChess can easily adapt to the situation where a variant has several aliases. It's not a nice thing, especially if new aliases keep popping up for the same variant, but at this point it may be the only realistic option.

@HGMuller
Copy link

HGMuller commented Dec 5, 2016 via email

@ilaripih
Copy link

ilaripih commented Dec 5, 2016

But the proposed "easy solution" requirescoding and recompilation, which
the bulk of the users would consider totally incomprehensible. A far
easier solution would be to configure the GUI to start games from a
position supplied by the user, e.g. in a .fen file. Most users would be
able to do that, and I suppose that is possible in CuteChess too.
Nevertheless people experienceit as pretty cumbersome to have to create
a file with positions.

I actually thought about it at some point that it would be cool if new variants could be added via a simple config file. But I found it a lot easier for myself to just write the code. I realize that random users can't then easily add new variants but that's okay.

OK, thanks for clearing that up; I was just going by what the other guy
had explained. Nevertheless, isn't this a /needless/ degradation of the
use fulness? You might want to flag such a condition as a likely error,
but it seems a warning would be sufficient.

Well, after the n-check variant changes by @alwey this check can be reimplemented (even removed completely) by variants that have kingless positions.

@alwey
Copy link

alwey commented Dec 5, 2016

Yes, actually I implemented Antichess (Losing Chess), Giveaway and Suicide Chess before Horde. Here the kings can be captured and removed. Pawns can promote to King. Some issues remain. Possibly @ilaripih already found the solution while fixing a Horde problem ...

@HGMuller
Copy link

HGMuller commented Dec 5, 2016 via email

@ddugovic
Copy link
Owner

ddugovic commented Dec 5, 2016

@HGMuller Sorry, my relay variant support is incomplete and I have now disabled it. (I don't recall how I enabled it while fixing issues but enabling it was unintentional.)

@ddugovic
Copy link
Owner

ddugovic commented Dec 5, 2016

@HGMuller FYI when I connect to my VICS at relaychess.org:5000, the following examine commands (fixed in my VICS server instance) display a normal board:

ex b seirawan
ex b seirawan 0
ex b wild seirawan

@HGMuller
Copy link

HGMuller commented Dec 5, 2016 via email

@niklasf
Copy link
Collaborator

niklasf commented Dec 13, 2016

Closing this since the match is over and a new one is coming up: #171. If I missed anything it's probably best to open new tickets for any remaining issue at this point.

ianfab pushed a commit to ianfab/Stockfish that referenced this issue May 8, 2022
Fixed a bug that an assertion fails in the trainer.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants