Port to using CMake. #3012
Port to using CMake. #3012
Conversation
|
How well will this work with all the existing chess GUIs today? That is one of the beauties of Stockfish today, it is cross platform and easily accessible by many and different GUIs today. The approach has worked well this far. I also do not believe we should necessarily kowtow to Apple and iOS - they are the ones who set their own rules , not us. Apple has made enough money off the backs of others. Edit/. I'm also assuming you will be making your GUI free to all - and if so , you should state that. That may help your cause. |
|
Making Stockfish as a library: sure, you could consider this as a home project on your personal git branch. Be careful to read the GNU licence carefully if you want to distribute your cross-platform GUI, however, as it may very quickly require that you distribute your GUI source code and the modified Stockfish source code along with the binaries. About cmake: probably not an option, because each build tool we add needs extra maintenance, and we prefer to keep SF toolchain as simple as possible. |
|
@MichaelB7 Stockfish works well when a platform supports calling external processes. But on iOS or even Android (where it's possible but still a pain) or any other embedded platform, it's just not practical. For my case specifically, I was working on GUI in Qt. Since I'm using the open-source version, GPL was already a given. But my example was just a use case. There are plenty of other use cases where something like this would be useful: Stockfish stockfish;
while(true) {
std::string move = convertGuiActionToString();
stockfish.process(move);
if (stockfish.isCheck()) { ... }
}That's just pseudo-code by the way. @snicolet If the official Stockfish developers aren't interested in Stockfish as a library, I will remove that code from CMake. I don't have enough time to maintain my own branch. This is just about contributing a CMake port to use as a replacement for GNU Make. I think using CMake would actually be far simpler than using Make. It builds natively on Windows (removing the need for MinGW unless desired). Try comparing Some other advantages of CMake:
If the Stockfish developers are not interested in using CMake as a replacement for GNU Make, then it's probably best to just close this. |
It checks if the compiler supports it, but not if the hardware does, so the binary compiled with default settings crashed with |
|
Fair enough. I already have avx2 and avx512 disabled by default (enabled by using |
|
|
This now behaves similarly to `make x86-64-modern`. Also, port Appveyor.
|
@syzygy1 do you have cmake installed ( I fixed the issue with the compiler flags. By default, it behaves similarly to I also changed Appveyor. That was surprisingly easy, since it already used CMake. |
|
@Symbitic I don't, which is a bit my point ;-) |
No longer need to run CMake with 2 different commands. Just use `-DENABLE_PROFILING=generate`, and CMake handles the rest. Also add script to download NNUE.
|
@syzygy1 Makefile works... when run under very specific circumstances (Unix-like platform with compilers, tools, and libraries installed). CMake fits a lot more use case scenarios. It's also just a bit more natural if you're not a Unix-geeks like us ;-) As for the PGO build: I just fixed that. All it takes now is just running cmake with I also rewrote the script for downloading the NNUE file in pure CMake, so it doesn't require Wget or cURL (or even bash). Profiling should work without problems even on Windows as soon as I add the MSVC compiler flags. |
|
"Not a Unix-geeks like us"....... |
|
For what it’s worth, I strongly prefer CMake to traditional Makefiles. |
|
BTW, I'm open to look into this, the current Makefile and build process and CI is getting stretched to its limits, cmake is a pretty common tool these days, and might be able to solve some to the issues. I can even sympathize with the idea of a (static) library being part of the build results, which could facilitate integrating stockfish in other (GPLed, obviously) tools. It is somewhat orthogonal to this patch, but this patch facilitates this, of course. I would approach this PR in a conservative way, and I'd like to see it pass CI before considering (so needs rebasing, probably after I merge the Android ndk patch to the Makefile). I would also need some fishtest worker changes, obviously. So neither accept nor reject right now. |
|
"About cmake: probably not an option, because each build tool we add needs extra maintenance, and we prefer to keep SF toolchain as simple as possible." Hard to believe this when Stockfish already uses CMake for Appveyor builds. Since CMake supports more platforms than makefile this doesn't seem to make sense. |
Ironically my experience is that
This can very well be done in make too. But ARCH is a feature - it allows crosscompilation |
|
I'll close this as this is now outdated quite a bit. In case the PR is updated we can reopen the discussion |
|
I never got a clear answer on if people were really interested enough in this. I can update it easy enough since there are no changes to the code itself, but keeping on top of the frequent updates to Stockfish requires some effort. If the devs are willing and ready to accept a switch to CMake, I will update this. If not, I'll leave this fork up in case someone wants to use it as a reference someday. |
|
Pulled your branch and built on Windows 11 w/ cmake 3.28.1 with no alterations to your code. Used Microsoft Visual Studio 17 2022 without any errors. Very cool. Here you can see me running the baby's first After successfully building your branch with CMake on Windows 11, I've become a staunch advocate for adopting CMake as the primary build system for Stockfish. The seamless integration with Microsoft Visual Studio 17 2022 and the absence of errors during the build process speak volumes about CMake's robustness and efficiency. Make on Windows is just not as accessible as some here believe it to be. CMake's cross-platform support is unparalleled, making it an ideal choice for a globally collaborated project like Stockfish. Unlike GNU Make, which often presents hurdles on Windows systems, CMake facilitates a more inclusive and frictionless development environment. This inclusion is especially critical for a project with diverse contributions like Stockfish. Moreover, the flexibility CMake offers is impressive. It accommodates various build systems like Make, MSVC, and Ninja, providing developers the freedom to choose their preferred tools without compromising the build process's integrity. This adaptability is crucial for a project as dynamic and complex as Stockfish. In light of these advantages, I strongly believe that migrating Stockfish to CMake and ending maintenance of the makefile would be a strategic move, enhancing the project's accessibility and scalability while streamlining the development workflow. I know this is woefully out of date, but are there any others here that would be interested in this development in a new PR? |
|
@Greendogo I'd be more than happy to restart this, especially since porting GitHub workflows will be a lot easier than Travis, but I never got an answer about whether the devs are willing to stop using Make and use CMake from now on. @syzygy1 seemed resistant to the idea. |
|
@syzygy1 hasn't really been active in Conan for 3 years. I don't know if they're still around or not. |
One suggestion I'll make is that you should definitely remove the library separation stuff. It's a great idea, but it's more important to have success with cmake first and not to make this important migration polluted with separate concerns. That library-ization task can be tackled in a separate PR after cmake is accepted. |
|
I'd prefer to get an acknowledgement that the devs are interested before I start working on this again. |
|
@Symbitic To give you some kind of answer, vondele was always relatively open to this, and I guess his main concern was if everything can be ported over to CMake easily, which would be good to know before. Regarding #4626 there also have been some concerncs raised as to support a newer build tool like CMake. Regarding the library support, we actually have been thinking about going in that direction, but a lot needs to be done in that direction. Stuff like the net download can probably be more easier implemented in a script which is then just called? So my question is: Is porting everything from the Makefile to CMake feasible, without having to rely on some hacks to make things work? |
|
I don't want to start a long discussion about all possible build tools but Meson might be an alternative, and it looks a bit more readable than cmake and looks similar to make. Lc0 also uses it. |
|
@Disservin It is absolutely feasible. I already have the net download implemented in pure CMake. Doesn't even need curl or wget. So to answer your question: yes, everything can be ported from Makefile to CMake. |
|
@Symbitic Ok, while I cant give you a certain answer if we will switch to CMake, I think having a working up to date version of this would be interesting. |
|
Now I understand if you dont want to invest time into this if the answer to the adoption is uncertain.. |
|
What would make you certain about switching to CMake? Also, should we open this as a new Issue for discussion instead of doing this discussion in the old closed PR? |
An initial working version of porting the build system to CMake.
CMake includes many advantages over hand-written Makefiles, including:
Instructions on everything are included in the updated Readme.
Now, there are some things to be aware of:
make netcommand. I might need some help with that.-DENABLE_PROFILE=generate(which will automatically run the finished binary when done), then running again with-DENABLE_PROFILE=use.One final thing to note: It works by building a libstockfish static library then linking that to the main stockfish binary. There are no changes to the code; the static library just compiles all the files except main.cpp then the binary target compiles just that file.
The reason I did this is because I'm hoping Stockfish can be made into a library (aka can have strings passed directly instead of using getline). I'm working on a cross-platform Chess GUI, and one of the most frustrating things was the fact that no major chess engine includes a library - they just expect frontends to run them as an external process and then forward them messages. That doesn't work well when dealing with Windows and iOS especially.
That's my take. If the official Stockfish developers are uninterested in that, I can remove that part of the build script very easily.