Not compiled with Eigen backend. #305
Comments
|
You haven't installed Eigen correctly and/or you haven't configured CMake correctly. For windows, Eigen's documentation on how to install it for use with cmake is incorrect and somewhat misleading: https://eigen.tuxfamily.org/dox/GettingStarted.html claims that you don't have to install Eigen but actually the required cmake files are not there if you download it. Even though Eigen is header-only, the cmake files still need to be built, but then the next page relating to cmake https://eigen.tuxfamily.org/dox/TopicCMakeGuide.html skips right to assuming you've already figured out how to build and install it, rather than just unzipping the source. The actual answer is provided in this thread: For linux, just installing the appropriate thing through your package manager should work fine (such as libeigen3-dev). |
|
Because this isn't documented well by Eigen and seems unnecessarily complicated for a header-only library that in theory shouldn't require a build and installation step, probably we should figure out something in KataGo's cmake that instructs users to just work around the mess and point cmake directly at the header files (so that you can simply unzip Eigen and have it work, without having to "build" it). |
|
Thanks! I have downloaded (https://software.opensuse.org/search?utf8=%E2%9C%93&baseproject=openSUSE%3AFactory&q=eigen3) eigen3-devel (noarch.rpm) and shall try it (eigen3 itself has only src.rpm). |
|
Also again, I encourage you to use the LZ chat rather than opening issues here for things like this. :) Because again, things like this are usually just missing packages that you haven't installed or set up on your particular system, rather than something that necessarily needs to be fixed in KataGo, and more people than just me might be able to help out. Although, for Windows I am trying to update KataGo's cmake setup to present and an easier workaround for users, so there does need to be some improvement in KataGo's docs regarding this. |
|
I don't know, how usable the chat will be for me, if I don't using Go engines on this PC with Internet... |
|
Translated: You still don't have Eigen installed correctly. If you're currently running OpenSuse, you can install it by simply using If you are continuing to have problems after these steps, please come to the chat: http://discord.gg/5vacH5F |
|
How You can see (https://software.opensuse.org/search?utf8=%E2%9C%93&baseproject=ALL&q=eigen3) there is only eigen3...src.rpm (i.e. source code). And there is non official libeigen3-devel (+ eigen3-devel, that I had installed). Thanks in advance. |
|
As was expected, there is "nothing to do" with ...src.rpm:
Нечего выполнять. So, because I didn't get answer here, how Katago developer expects his product must be installed on end users systems (except 1 particular OS - relic of the past), I had tried next steps (without of eigen3-devel packet uninstalling).
So, there isn't any differences with installed or not eigen3 packet: it's Katago developer mistake, when the product doesn't work on end user OS (may be except on 1 particular)... |
|
Install the project... |
|
For comparison the file list of standardly installed packet (from Yast2 Program Manager): eigen3-devel - C++ Template Library for Linear Algebra /usr/include/eigen3 |
|
It sounds like you have a sounds like a cmake/eigen3 issue - cmake is probably failing to find the right path for your eigen3 installation, or eigen3 is misconfigured somehow. The root problem is: Your program manager claims that That means cmake, when instructed to require Eigen, must be failing to find and add So here are some routes that you might try. More then one of them might work, so pick your favorite, and try it, and if it fails, try another one:
The last solution is of course hacky and not very generalizable. But once you've gotten the issue solved, in the event that you want to help make things better for other users in your situation, you could consider helping debug the situation, looking at your unique build environment and why things went wrong. Due to the very poor documentation of Eigen3+cmake on Windows, I added a bunch of specific MSVC-specific error guidance and workarounds to try to help the common case a user might get stuck. Once you've got it working, if you think there is error guidance that KataGo could add to its cmakelists would help in your case (Linux, rather than Windows) that would also not be misleading and trigger when the error was something else instead, I could try adding that. |
|
" As I understand you, your cmake is no longer complaining about failing to find eigen ever since you installed eigen3-devel. But then instead make fails to find it. Correct?" I'm unsure, that "Found Eigen3 at " (i.e. with void ${EIGEN3_INCLUDE_DIRS}) is that, what it must be, and this is correct... Then I had edited CMakeLists.txt: and had got: Comparing of Eigen3 and ZLIB gives (for my non developer opinion) the next result: " At any point, you may run make like make VERBOSE=1 for it to output every command in full as it runs it, to see exactly what arguments your compiler is getting. In particular, you will be able to tell here if it's specifying the extra system include directory for Eigen incorrectly. So... what system path is cmake adding that is not /usr/include/eigen3? " Your program manager is telling you that the following has been installed: Legacy variables, do not use. May be removed in the future.set (EIGEN3_FOUND 1) and had got: Of course, it's better instead of "hardcoded" path to get (and set) it somehow from "find_package"... Then I had built KataGo: Is it OK now? (Benchmark had run correct, but I have one remark on it: it will be more usable, if number of benchmarked threads will increase in succession from minimum until rate decrease (maximum 2 consecutive times), because threads overnumbering (12 in my case) leads to memory thrashing, when with optimal (5) and next numbers of threads the memory usage is normal (<|~ a half of full 16GB RAM with middle weights)...) |
|
Thanks, I'm glad you got it running. Do you happen to have time to help investigate what went wrong with your cmake? It would be great if there is something I can adjust in the official CMakeLists.txt so as to fix this problem in a general way, for anyone else with a system like yours. In particular, you discovered that it was failing to provide a value for EIGEN3_INCLUDE_DIRS. However, you also observed that /usr/share/eigen3/cmake/Eigen3Config.cmake was present in your system, and had the command I'm a little surprised that you notice heavy memory usage with 12 threads. I thought I had fixed that as of this release - Eigen Memory Bugfixes. Do you happen to be attempting to compile with a version prior to that? |
|
How You had written: ".../usr/share/eigen3/cmake/Eigen3Config.cmake... this file was hardcoded to look for a path that in a very specific relative location, and for some usages, this path would point to the wrong place...". I had downloaded zip archive of master branch, when I had written about it in previous issue: #292 (comment) |
|
Yes you are on an old version then. Please use the most recent version to get better memory usage. I still don't understand why your cmake configuration led to such problems, and since it's your machine, not mine, I don't have any way to run the appropriate tests to figure out more. So I don't feel like I understand well enough to report the issue, but if you think you understand it well enough and you are sure it's an Eigen/CMake bug, then yes theoretically the best way would be to raise the issue with Eigen or Cmake developers (whichever is appropriate) and have them fix it. |
|
As long as you have any familiarity with git or other version control, then in general, it is not recommended to download the zip archive and attempt to compile that way. Instead it's better to just 'git clone' the repo directly, so that it's easy to 'git pull' or 'git fetch' and stay up to date with bugfixes - the memory use problems you just reported have already been fixed for a few weeks. :) |
|
How I understood, You had the same problem with Your cmake configuration: ".../usr/share/eigen3/cmake/Eigen3Config.cmake... this file was hardcoded to look for a path that in a very specific relative location, and for some usages, this path would point to the wrong place...", didn't You? How I had written several times, I'm using Go on PC without any net connection. So, I didn't understand Your recommendation about "git clone"... So, is there any reason to download now a zip archive, to unpack it on PC without Internet, to change in /usr/share/eigen3/cmake/Eigen3Config.cmake the value of PACKAGE_PREFIX_DIR to "/usr", to try to compile new version without CMakeLists.txt modification and then to write here the results? |
|
Ah sorry, I do keep forgetting - it's not common for me to talk to someone who has no internet on a machine. I'm not convinced the problem I ran into was the same one you ran into. For example I only ran into that problem on Windows with a custom unzipping of Eigen3, but when I "installed" it via MSVC build, there was no issue and the path was correct. So the problem was merely that Eigen3 didn't document how it should be installed on windows. Similarly, when I installed Eigen3 via package manager on my Linux, there was no problem. So I'm definitely not sure why your install had issues with setting the path correctly. |
|
OK. I'll download and report here about compiling and about benchmark... |
|
Editing of Eigen3Config.cmake file doesn't work. "find_package" function finds package directory by file "signature_of_eigen3_matrix_library", but .cmake files are at another location... and had got: Benchmark had run correct. There wasn't dependency of memory usage from numSearchThreads and was some not a very big - from weights size. There was logical recommendation of 4 numSearchThreads using for PC with 4 core CPU, but I didn't understand, why the 12 numSearchThreads usage was rated worse, then 3 with the next results: |
|
I had downloaded binary release versions. So, because eigen binary release works without any eigen package installed, I'm closing the issue. |
|
I'm glad you got it working finally. By the way, to answer an earlier question from you that I missed - the reason why it shows 6,8,12 threads as worse even though they have a similar visits/s, is because having more threads decreases the quality of the search. If when you increase the number of threads the speed stays the same, or even increases slightly but not enough to compensate, then the overall strength will go down. |
|
And the "EloDiff" number is attempting to estimate what the change to Elo might be. It's not a perfect estimate, but should give an indication of what tradeoff point you want between more threads (to improve performance) and fewer threads (to limit the harm to the search quality). |
|
Thanks. |
.../KataGo-master/cpp> cmake . -DUSE_BACKEND=EIGEN
-- The C compiler identification is GNU 8.1.1
-- The CXX compiler identification is GNU 8.1.1
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
Building 'katago' executable for GTP engine and other tools
-DUSE_BACKEND=EIGEN, using Eigen CPU backend
Including Git revision in the compiled executable, specify -DNO_GIT_REVISION=1 to disable
-- Found Git: /usr/bin/git (found version "2.17.1")
CMake Error at CMakeLists.txt:233 (find_package):
By not providing "FindEigen3.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "Eigen3", but
CMake did not find one.
Could not find a package configuration file provided by "Eigen3" with any
of the following names:
Add the installation prefix of "Eigen3" to CMAKE_PREFIX_PATH or set
"Eigen3_DIR" to a directory containing one of the above files. If "Eigen3"
provides a separate development package or SDK, be sure it has been
installed.
-- Configuring incomplete, errors occurred!
The text was updated successfully, but these errors were encountered: