Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrong number of CPU cores detected in Windows Server multi Processor Group environment #771

Closed
twisniewski78 opened this issue Feb 16, 2016 · 34 comments

Comments

@twisniewski78
Copy link

commented Feb 16, 2016

When running psutil.cpu_count(logical=False) I am getting 60 instead of 120 cores. Similarly when running psutil.cpu_percent(interval=1, percpu=True) I am getting utilization just for half of the CPU cores.
My system has two Processor Groups (each with 60 cores) and it seams that psutil is not processor group aware. On large systems processor resources are split into groups and it requires using special API to get correct numbers.

@giampaolo

This comment has been minimized.

Copy link
Owner

commented Sep 22, 2016

Sorry for such a big delay in responding. Can you provide more info about what kind of processors are these? Otherwise I really have no idea what kind of alternative API we could use.

@twisniewski78

This comment has been minimized.

Copy link
Author

commented Sep 22, 2016

I have eight Xeons E7-2890 v2 with 15 cores each. My first guess would be NUMA architecture and Processor Group awareness rather then type of CPU. I was having the same issues on different system with 80 cores and different CPU model.

@giampaolo

This comment has been minimized.

Copy link
Owner

commented Nov 25, 2016

Looking back at this: what does len(psutil.cpu_times(percpu=True)) return? 60 or 120?

@giampaolo

This comment has been minimized.

Copy link
Owner

commented Dec 1, 2016

The problem may be here:

if (ptr->Relationship == RelationProcessorCore)

As per:
https://msdn.microsoft.com/en-us/library/windows/desktop/ms686694(v=vs.85).aspx
...if (ptr->Relationship == RelationNumaNode) we may have to increment the counter by 1

Still, that would not match len(psutil.cpu_percent(interval=1, percpu=True)) which according to your report would keep returning half the cores.

cpu_times(percpu=True) internally relies on dwNumberOfProcessors:

for (i = 0; i < si.dwNumberOfProcessors; i++) {

...which should be "correct", so I'm really not sure what to think.

@twisniewski78

This comment has been minimized.

Copy link
Author

commented Dec 2, 2016

@twisniewski78

This comment has been minimized.

Copy link
Author

commented Dec 2, 2016

@giampaolo

This comment has been minimized.

Copy link
Owner

commented Dec 2, 2016

Indeed MS doc states:

dwNumberOfProcessors
The number of logical processors in the current group. 
Note  For information about the physical processors shared by logical processors, 
call GetLogicalProcessorInformationEx with the RelationshipType parameter set to 
RelationProcessorPackage (3).

It looks like GetLogicalProcessorInformationEx should give us what we want:

When this function is called with a relationship type of 
RelationProcessorCore, it returns a PROCESSOR_RELATIONSHIP 
structure for every active processor core in every processor group 
in the system. 
@giampaolo

This comment has been minimized.

Copy link
Owner

commented Dec 2, 2016

Main problem is that I don't have the right hardware to test this against.

@giampaolo

This comment has been minimized.

Copy link
Owner

commented Dec 2, 2016

Note: os.cpu_count() on Python 3 relies on GetSystemInfo / dwNumberOfProcessors so it has the same problem.

@twisniewski78

This comment has been minimized.

Copy link
Author

commented Dec 2, 2016

@FJplant

This comment has been minimized.

Copy link

commented Dec 3, 2016

@giampaolo

This comment has been minimized.

Copy link
Owner

commented Dec 3, 2016

It is (still investigating and not fixed).

@FJplant

This comment has been minimized.

Copy link

commented Dec 3, 2016

@waterkirin

This comment has been minimized.

Copy link

commented Jan 17, 2017

I may got the same issue. I have dual E5 2686 v3 with 18 cores each(36 cores and 72 logical processors ). When I ran psutil.cpu_count(logical = False) , I got 18. When I ran psutil.cpu_count(logical = True), i got 32. This is confusing.

@giampaolo

This comment has been minimized.

Copy link
Owner

commented Jan 17, 2017

Can you confirm os.cpu_count() has the same issue?

@giampaolo

This comment has been minimized.

Copy link
Owner

commented Jan 19, 2017

Here it is suggested that this may be due to 32 bit processes. Have you tried using Python 64 bit?

@FJplant

This comment has been minimized.

Copy link

commented Jan 19, 2017

Not completed... ^^

@giampaolo

This comment has been minimized.

@robbuckley

This comment has been minimized.

Copy link

commented Apr 28, 2017

hi, i have the same issue. I am running 64 bit python 3.6.0 (anaconda) on 2 similar platforms
platform1 - 2x Xeon E5-2698v4. 20 cores/CPU = total 80 logical cpus with hyperthreading
platform2 - 2x Xeon E5-2697v3. 14 cores/CPU = total 56 logical cpus with hyperthreading

os.cpu_count(), multiprocessing.cpu_count() and psutil.cpu_count() all report the same thing: 40 cores on platform1 and 56 on platform2.

sysinternals coreinfo shows that platform1 has 2 processor groups and platform2 has 1 group.

i have a simple python program using a threadpool to launch multiple subprocess of an external tool which is entirely CPU bound. On platform 2 i can get it to 100% CPU for sustained periods. on Platform1 it tops out at ~50%

i would really like to be able to get this using 100% of platform1.

Im happy to run any test to help with this issue

@giampaolo

This comment has been minimized.

Copy link
Owner

commented Apr 28, 2017

@robbuckley can you confirm os.cpu_count() has the same issue? That has been introduced in Python 3.4. If you're on 2.7 you can use multiprocessing.cpu_count().

@robbuckley

This comment has been minimized.

Copy link

commented Apr 28, 2017

@giampaolo yes I confirm. I didn't realise they are the same. Im new to multiprocessing in python :-)

@giampaolo

This comment has been minimized.

Copy link
Owner

commented Apr 28, 2017

Could you please try Python 3.7 + os.cpu_count(). I suppose we should file a ticket on the Python bug tracker. Hopefully we can get some insight from there. The main problem here is that I don't have the right hardware to test this against.

@robbuckley

This comment has been minimized.

Copy link

commented May 3, 2017

How do I install python 3.7? according to https://www.python.org/dev/peps/pep-0537/ there doesnt seem to be an 3.7.0 alpha 1 yet, and i couldnt find anything in the conda channels including conda-forge.

I'm happy to help but dont have a lot of time to spend on it. Surely reporting a bug against the latest released version (3.6.1) should be OK?

@giampaolo

This comment has been minimized.

Copy link
Owner

commented May 3, 2017

Yeah, 3.6 is also OK.

@robbuckley

This comment has been minimized.

Copy link

commented May 4, 2017

@giampaolo is there anything more you need from me now?

@giampaolo

This comment has been minimized.

Copy link
Owner

commented May 5, 2017

Is os.cpu_count() on python 3.6 broken in the same manner?

@robbuckley

This comment has been minimized.

Copy link

commented May 16, 2017

yes, i confirm. See attached python session with os,cpu_count() reporting 40 cores, and output of microsoft coreinfo tool showing 80 logical processors in 2 processor groups

ci.txt
osc.txt

@giampaolo

This comment has been minimized.

Copy link
Owner

commented May 16, 2017

OK, so can you please file an issue on http://bugs.python.org/?

@robbuckley

This comment has been minimized.

Copy link

commented Jun 6, 2017

@giampaolo

This comment has been minimized.

Copy link
Owner

commented Mar 28, 2018

@giampaolo

This comment has been minimized.

Copy link
Owner

commented Mar 28, 2018

I created a psutil branch using GetLogicalProcessorInformation (not GetLogicalProcessorInformationEx) to determine both logical and physical CPUs:
#1257
@twisniewski78 / @robbuckley : can you try it?
If you don't have Visual Studio I can send you a .whl file for Python 2.7 32 bit.

@twisniewski78

This comment has been minimized.

Copy link
Author

commented Mar 28, 2018

Sadly I have no longer access to this environment or anything like it.

@giampaolo giampaolo reopened this Mar 28, 2018

@giampaolo

This comment has been minimized.

Copy link
Owner

commented Apr 13, 2018

OK, this should finally be fixed as of #1257.

@giampaolo

This comment has been minimized.

Copy link
Owner

commented Oct 20, 2018

Closing out as fixed.

@giampaolo giampaolo closed this Oct 20, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.