-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
issue with large page commit (second part of the issue). #2681
Comments
yep will be fixed shortly. The compiler doesn't support c++11 completely, but we can work around. |
gcc < 5 doesn't fully support the c++11 `std::is_trivially_copyable<Entry>::value` Remove it, as it is not essential. fixes official-stockfish#2681 No functional change.
stockfish engine developer from 14/5/2020 not working on chesspartner ( uci) |
@samer707 can you clarify your comment. Are you using latest master, what error do you see, and which compiler do you use? |
ok i have (software) on my pc called chesspartner version 6.0.4.0 and when i want to import engine stockfish developer in 14/5/2020 as uci selection , my software program freeze and close when i select stockfish developer engine in 14/5/2020 , but in 13/5/2020 it works fine |
are you referring to the abrok builds ? There are multiple builds on the 13th and 14th can you be precise which one is the last to work, which one is the fist to fail? |
beginning from this one , not working : Add support for Windows large pages for users that set the needed privilige "Lock Pages in Memory" This expert setting might improve speed, 5% - 30%, depending closes #2656 fixes #2619 No functional change and all before is working good |
this one not working |
from this one not work Add support for Windows large pages for users that set the needed privilige "Lock Pages in Memory" This expert setting might improve speed, 5% - 30%, depending closes #2656 fixes #2619 No functional change |
@vondele Seems I have an issue on win7.. When I start a first game against sf with a big hash (>50% of ram) via Cutechess GUI - that's ok.. Then if abort a game and start new one quickly - I have time out from gui or a long freeze.. In last case I have a message about Large Pages (can't allocated) |
not failing in windows 8.1 , 10 before last developer version |
@samer707 does it fail outside of the gui? can you try in a command line window? |
why when Sami Kiminki make his developer stockfish this error seen ? |
thank you at all for your responding : |
i do not know how to correct that by my self |
ok stockfish_20051319_x64_modern.exe worked fine for chesspartner v 6.0.4.0 |
@d3vv This looks like Windows is still dealing with hash memory deallocation of the previous SF process, while a new SF process is already being launched? To confirm, could you check whether swap is being used? You should see this from Task Manager. If swapping is the case, I'm not sure what can be done here other than falling back to malloc() for small pages if the previous versions worked for you. However, for some reason, malloc() provides slow memory for some Opteron boxes, so the fix is not so clear.
My guess is that the root cause is about printing the info string
before Here are some ideas how to fix this in order of my preference:
But still, this root cause is just an educated guess at the moment. I don't have ChessPartner to confirm. |
i am not a programmer , i want development build stockfish version that opened and work in chesspartner v6.0.4.0 , because chesspartner is easy to import and play chess engines than other , and before (20051320_x64_modern.exe) version all development builds opened on chesspartner , and all versions from and after that version (20051320_x64_modern.exe) is make chesspartner freeze and close |
please you are programmers and you can make new development build stockfish that fix this case |
@samer707 For the time being, I suggest using an older dev version. The fix ideas list was meant for other developers. We need to discuss our options before rushing into coding. |
@samer707 we can try to fix if we can understand the problem. Until we have enough info or can reproduce ourselves, there is not much we can do. So, can you run 20051320_x64_modern.exe in a different gui or on the command line (type bench if it started), just to see if it starts on your system? If so we, can related it to the particular gui used. @skiminki I agree the sending the info string early might be an issue. |
ok thank you , i am waiting |
can i send pictures that explain the case ? |
I did mention that after reload(s) there might be no more LP available. |
thank you and we hope to see fixes in next upcoming development builds |
@skiminki Could it possible to give me sf binary for modern(popcnt) win version for test? And please be advised that |
@skiminki Thank You in Advance. Output Here:
|
Seems that we're in luck! Two different error codes here: Failed to allocate large pages: err=1450 = ERROR_NO_SYSTEM_RESOURCES Failed to allocate large pages: err=1455 = ERROR_COMMITMENT_LIMIT I'll double check the error codes on win10 later. But we may be able to use err=1450 to detect a transient error condition and have SF try again. |
Do not send the following info string on the first call to aligned_ttmem_alloc() on Windows: info string Hash table allocation: Windows large pages [not] used. The first call occurs before the 'uci' command has been received. This confuses some GUIs, which expect the first engine-sent command to be 'id' as the response to the 'uci' command. (see #2681) closes #2689 No functional change.
@skiminki As I know more or less quality implementation for Win LP persist into postgresql.. I think here: https://github.com/postgres/postgres/blob/master/src/backend/port/win32_shmem.c |
thank you very much for your concern , latest 19/5/2020 build is working , thank you , you are sincere in your job |
and all staff is good , because they cared for fixing problems |
I suggest that we output the error code (if there is one) at the end of the info line, as such:
This would help debug problems with large pages in the future, while remaining compliant with the UCI protocol. |
@d3vv From the PostgreSQL code:
Coincidentally, this was just what I was about to try: If we encounter error 1450, we keep trying for 5..10 secs. |
Updated #2687 with a test patch for @d3vv 's issue. Locally compiled unoptimized binary: stockfish-2681-pending-free-v1.zip @d3vv Could you give this a try? Unfortunately, I can't quite reproduce this behavior on my Windows 10, even when compatibility modes are used. Also, it turns out that Windows may return error 1450 (ERROR_NO_SYSTEM_RESOURCES) even if sizes larger than the physical memory size are requested, so this workaround may actually be a regression for some people who have enabled large pages on Windows 10 but who expect the allocation to fall back to small pages... Not sure how important corner case this is, though. Finally, there's no trivial way to detect the Windows version, it seems, so we can't simply enable this workaround for pre-Win10 boxes only. |
Is supporting Large Pages really worth all the trouble??? Edit and PS: This whole Hash and the related UCI and Threading stuff has become way too convoluted and complicated recently, imho. |
|
Ok, thanks. I take it that if you wait for a while after the previous SF process has terminated, then SF will again be able to use the large pages? For the record, we're doing the same with allocation what PostgreSQL is doing with the large page file mappings. There are all sorts of fallbacks that we could try such as allocating the hash table in smaller chunks in case the full alloc fails, and then we'd probably be able to allocate the hash table at least mostly with the large pages. But I'm not quite sure whether it's worth the effort, since that kind of stuff tends to become a bit complicated and possibly fragile. I'd also expect almost zero chance that if I write that 100-300 lines worth of semi-complex code to workaround this Win 7 problem that the code can actually get merged... At least the large pages seem to work better with Windows 10. I'm not sure how much comfort this is for the Win 7 users. One more note. Windows 7 extended support has officially ended in Jan 2020. Windows 8.1 extended support ends in Jan 2023 (mainstream ended in Jan 2018). Would it make sense to consider an upgrade to Windows 10? Anyways, I'm open for suggestions wrt what we should do with d3vv's issue. |
There will be issues under Windows when users do not have adequate memory for heavy usage of hash - hash that exceeds 50% of RAM. You do not need to use massive hash to get a 10% speed up. 2048 Mb hash size works wonderfully for most people with limited RAM. I consider anything under 128 Gb limited when using Large Pages. Ymmv of course. Edit : Obviously the best case is for Windows 10 users and it may not be suitable for outdated OS or for those with a limited amount of RAM. |
My 2cents: the code looks OK as it stands now. If the OS says it can't provided the large pages, we shouldn't negotiate with it, that's it, and we have a functional fallback. After all, it is marked as 'expert use'... experts will quickly figure out what the sweat spot for the (optional) feature is. |
I'm kinda agreeing with that. The only thing I'm not completely sure about is whether there should be a UCI/commandline/build-time option to disable/enable LP. To me, it sounds entirely feasible to consider that someone would want to use SF with both settings without going to the group policy editor to toggle the feature on/off. For example: Doing the regular analysis: LP ON. Running SF under cutechess: LP OFF. |
I still prefer to have no additional option, at least for now. It is easy to add an option, it is very difficult to remove. There will always be use-cases for more options, but the majority is covered with the current setup. |
Works for me |
I think it’s not necessary to support Win7 if everything is so complicated. In fact I have used Win7 for legacy system tests only. |
OK, thanks. I'll close the issue, and the testing PR. |
with your permission ,i want to know , how run the stockfish bench in window |
you need to open a windows command line window and in that window execute |
Thank you |
you need to configure your GUI so that the network file is found, i.e. set EvalFile to the proper value including the full path. https://github.com/vondele/Stockfish#evalfile |
See 86ee4eb#commitcomment-39204441
The text was updated successfully, but these errors were encountered: