CPU usage calculated incorrectly for servers

[Sandtato]

Background (please complete the following information):

• Panel or Daemon: Panel (may be daemon, not sure. Can update if needed)
• Version of Panel/Daemon: 0.7.10 (Daemon: 0.6.7)
• Server's OS: Ubuntu 18.04 x64 (Behind Hyper-V hypervisor, also tested with VMWare)
• Your Computer's OS & Browser: Windows 10, Microsoft Edge (also tested with Chrome and Firefox)

Describe the bug
CPU usage percentage indicator appears to be abnormally high when starting a Spigot server (also includes Paper), occasionally reading as high as 6000%+ CPU usage. This issue is even more apparent when starting up the server, although it happens even when players are online.

To Reproduce
Steps to reproduce the behavior:

1. Go to any server that is running Spigot (if none exist, create one)
2. Start the server (wait for it to install if newly created)
3. Watch the CPU usage counter.

Expected behavior

100% usage typically equals 1 core, so theoretically the maximum % my CPU usage should be is 800% (8 threads total). My Hyper-V instance however is only 3 threads, so the maximum theoretical usage should be 300%. The test instance I ran on my VMWare Hypervisor had 4 threads, so the maximum theoretical usage should be 400%. I am aware that recent versions of Paper have things such as ASync World Generation, but to see CPU usage over 300% seems completely abnormal to me. Attached is a screenshot of the bug in action while the server was starting.

[DaneEveritt]

You can go over those percentages, its just poor performance when that is happening.

You'll need to provide metrics from Docker while this is happening, just showing a percentage in the panel does not help narrow down if this is occurring due to bad math on my end, or if it is actually correct.

[Sandtato]

@DaneEveritt
Just figured I'd update you. I happened to test on both a Ubuntu 16.04 and 18.04 Vultr instance (KVM), which showed a lot lower on the panel's graph than my VMWare 18.04 instance. It indicated ~350% usage at its highest when starting up. However, I checked docker stats and that showed lower usage. Below is a screenshot of the docker data:

Ubuntu 16.04:

I also pulled the same data from my Ubuntu 18.04 instance (VMWare in this case), which is showing the ~6000%+ cpu usage. Here are the docker stats for that instance:

Ubuntu 18.04:

This VMWare instance was running off of 4 cores of my Ryzen 7 1700, while the Hyper-V instance which is also affected runs off 3 cores of a Intel Xeon E3-1271v3 processor. This leads me to believe that it is less likely a performance issue, and more of a issue somewhere in the code. If you do have anymore questions feel free to ask me.

[DaneEveritt]

Dumb question, but could you do a side by side of the panel usage display and the docker stats? Theoretically they should be lined up almost exactly (a gif would be perfect if possible).

Could you do this on the server view itself, and not the server list (since that updates much slower). That should help me figure out whats going funky. Thanks again.

[Sandtato]

@DaneEveritt Here's the gif you requested. This was taken off of my VMWare instance.

[DaneEveritt]

Do you have a CPU limit set?

[Sandtato]

I tested with both no CPU limit and the limit set at 200% (to allow for Paper's Async World Generation). Both still resulted in the abnormal CPU Usage percentage on the graphic, but server startup performance was negatively impacted with the CPU limit in place. If you would like I can grab a Docker shot with the CPU limit in place (will take me a little bit though).

[DaneEveritt]

Nah thats fine, I just can't reproduce it at all.

[Sandtato]

Just updating to mention that this issue doesn't seem to occur on latest panel + daemon version on my KVM test instance. Still happens on Hyper-V instance but going to write it up as a issue somewhere in that VM. Going to close issue unless further information is requested.

[steak-sauce]

Updating as well. This is still happening on the latest panel and daemon.

Background (please complete the following information):

• Panel or Daemon: Panel
• Version of Panel/Daemon: Panel 0.7.14, Daemon 0.6.12
• Server's OS: Ubuntu 18.04 LTS
• Your Computer's OS & Browser: Windows 10, Chrome

Describe the bug
Panel shows CPU usage as 1000%+. Moving the decimal over one makes it 100%, which is still wrong compared to docker stats, which shows ~7.5% for this container. Host shows VM utilization as 10-15%.
Panel: PHP 7.2.19-1+ubuntu18.04.1+deb.sury.org+1
Kernal: 4.15.0-55-generic #60-Ubuntu SMP Tue Jul 2 18:22:20 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Docker: 19.03.0
Host: KVM

To Reproduce
Steps to reproduce the behavior:

1. Go to 'My Servers' on CLIENT panel (not admin)
2. See error

Expected behavior
Show docker stats CPU utilization.

This is a test environment and I'm happy to provide credentials to Panel/Daemon/VMs.

[sich97]

I just wanted to add that this also happens to me.
Panel version: 0.7.14
Daemon version: 0.6.12
My computers OS & Browser: Windows 10 and Chrome

Using Hyper-V, pterodactyl is running in a 5-core Ubuntu 18.04.2 VM.
The CPU Limit is set to 0%

On the game server I have that uses CPU Limit, the graph rarely shows. Maybe because the CPU Limit limits the graphs visible area, while the line is way above this visible area. Just a thought.

Here are some pictures.
Idle:

Startup:

Back to idle:

Player logon:

[sich97]

I found something that might be of help.
There's a difference in how the CPU usage is calculated when using the Android app.

I'll just post the pictures:
Dashboard overview:

Server overview:

Back to dashboard:

As you can see, the dashboard view is at least a little bit more realistic, although I would expect the CPU usage to be more than that.

Additionally, I checked docker stats to see if it corresponded. It did not.
Docker stats showed 15% cpu usage while the dashboard overview in the app showed 4% cpu usage.

[paTaNiNho]

Are there any changes about this issue? I have the same result...

On panel:

On docker stats:

[DaneEveritt]

This is fixed in the development builds.