Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decrease idle cpu/power usage #351

Closed
Mike1445 opened this issue Mar 19, 2017 · 11 comments
Closed

Decrease idle cpu/power usage #351

Mike1445 opened this issue Mar 19, 2017 · 11 comments

Comments

@Mike1445
Copy link

@Mike1445 Mike1445 commented Mar 19, 2017

When idling nzbget seems to still use quite a bit of cpu. Looking at powertop it is by far the most consuming process on my machine.

powertop:
Usage Events/s Category Description
1.8 ms/s 212.3 Process /usr/bin/nzbget -c /etc/nzbget/nzbget.conf -D

top shows around 0.3% cpu usage:

top:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
5451 nzbget 20 0 823788 13184 4880 S 0.3 0.3 0:10.04 nzbget

strace shows only sleep nanosleep system calls every 100 milliseconds:

strace:
restart_syscall(<... resuming interrupted nanosleep ...>) = 0
nanosleep({0, 100000000}, NULL) = 0
nanosleep({0, 100000000}, NULL) = 0
nanosleep({0, 100000000}, NULL) = 0

Maybe increasing the hardcoded timeout from 100 m.s. to 1 second will help?

		// sleep longer in StandBy
		int sleepInterval = downloadStarted ? 0 : standBy ? **100** : 5;
		usleep(sleepInterval * 1000);
@hugbug
Copy link
Member

@hugbug hugbug commented Mar 19, 2017

Longer wait intervals may reduce responsiveness.
Nonetheless, why don't you try different values and tell us how it performs?

@RollingStar
Copy link

@RollingStar RollingStar commented Apr 29, 2017

Here's the file if anyone wants to investigate.

// sleep longer in StandBy

@hazcod
Copy link

@hazcod hazcod commented Aug 9, 2018

So we would only reduce responsiveness (so start of a download) whenever nothing is downloading?
Does this include the web thread as wel?

In any case, upping this to 500ms shouldn't be too hard of an impact.
Can you try that @Mike1445 ?

@vdboor
Copy link

@vdboor vdboor commented Aug 16, 2018

What does nzbget need to monitor when nothing is downloading? Web requests can block on the socket, queues might wait for fnotify. Hopefully nothing needs to be polled all the time.

@fedux
Copy link
Contributor

@fedux fedux commented Nov 3, 2018

I found the same problem today (high CPU usage) when trying to optimize my power usage. I agree with @vdboor and most loops could be changed to event based instead of polling. Is there any interest in this to happen? I'd try to do this myself in my spare time and submit the patches if so.

@hugbug
Copy link
Member

@hugbug hugbug commented Nov 3, 2018

I don't think the 0.3% is high. If there is a simple way to improve this I'm for it but if the change requires a considerable rework of the program I don't think it's worth it.

Generally when doing performance optimisations it's important to identify bottlenecks first. Actually it was just a wild guess that the CPU usage is caused by loops, in particular by the loop mentioned in the first post. If you have high CPU usage you can increase wait intervals in the loops, compile the program with different wait intervals and do the measurements.

And the most important: any change must preserve compatibility with currently supported operation systems (Linux, Windows, Mac, BSD, Android) and may not add dependencies (other libraries).

@fedux
Copy link
Contributor

@fedux fedux commented Nov 3, 2018

Well, it's not really about performance optimization but about power usage optimization when idle. And for me it's the same are the original reporter said, nzbget is the process using the most CPU when idle.

A quick debug shows that even when idle there are 7 threads waking up very frequently and doing nothing. Maybe a first approach could be to suspends those threads when idle?

@vdboor
Copy link

@vdboor vdboor commented Nov 15, 2018

@hugbug the problem isn't the % of CPU usage. It's the frequency of wake-ups that no longer allow the CPU to throttle down to low-power mode. Every few seconds, nzbget reminds the CPU that it shouldn't sleep, thus adding to more power consumption (and battery usage) of an otherwise idle machine.

@hugbug
Copy link
Member

@hugbug hugbug commented Jan 3, 2019

Thanks to @fedux for reworking thread/mutex primitives and for CPU usage improvements.

I've measured CPU usage in idle mode on macOS.

Parameter before #593 after #593
Article cache On
CPU usage (article cache on) 0.4 0.4
Idle wakeups per second 222 200
Article cache Off
CPU usage 0.1 0.1
Idle wakeups per second 52 30

As you can see the main issue here is the article cache manager which wakes up every 5 ms. The second thing is the queue coordinator which wakes up every 100 ms.

@fedux: I have an idea for a quick win here: in cache manager count the number of non productive wake ups in a row and increase sleep time after several seconds: from default 5 ms to 100 ms and even to 1000 ms after a few minutes. And a similar approach in queue coordinator. Would you like to try this (or other approaches) or should I?

@fedux
Copy link
Contributor

@fedux fedux commented Jan 3, 2019

I have other commits as work-in-progress here https://github.com/fedux/nzbget/commits/idle and my test instance has ~10 wake-ups per second, but it still needs some work.

Still, if you want to try this other idea it would be nice to test a different approach.

hugbug added a commit that referenced this issue Jan 11, 2019
to improve CPU usage when idle
hugbug added a commit that referenced this issue Jan 19, 2019
hugbug added a commit that referenced this issue Jan 19, 2019
UrlCoordinator is now completely paused (waits without wake ups) when
there are no work for it.
hugbug added a commit that referenced this issue Jan 21, 2019
Now sleeping much longer, up to next scheduled work, instead of often
wake ups to check if the work needs to be performed. This improves CPU
usage in idle.
hugbug added a commit that referenced this issue Jan 21, 2019
to avoid race conditions and lock ups
hugbug added a commit that referenced this issue Jan 27, 2019
In idle sleeps for 0.5 sec. Wake up immediately when a new item is
added to queue. Waking up from paused queue can take longer (up to 0.5
sec).
hugbug added a commit that referenced this issue Jan 27, 2019
from Unit “Options.cpp”. The latter now contains only program options
(which cannot be changed without reload).
hugbug added a commit that referenced this issue Jan 27, 2019
to replace direct calls to “usleep”, with parameter in milliseconds
instead of microseconds.
hugbug added a commit that referenced this issue Jan 27, 2019
hugbug added a commit that referenced this issue Jan 27, 2019
If there are no active downloads the disk service can now sleep for 10
seconds instead of 1.
hugbug added a commit that referenced this issue Jan 27, 2019
hugbug added a commit that referenced this issue Jan 27, 2019
Up to 2 second when queue is empty or downloads are paused.
hugbug added a commit that referenced this issue Jan 27, 2019
from history.
@hugbug
Copy link
Member

@hugbug hugbug commented Feb 2, 2019

With further changes the number of wake ups in idle has been reduced to as low as 1-2 per second or even less. MacOS and Linux report 0.0% CPU usage now.

All numbers refer to nzbget running in daemon mode. CPU usage is higher if nzbget is running with console output (started with -s switch instead of -D).

hugbug added a commit that referenced this issue Feb 2, 2019
This reduces CPU usage, especially in idle.
hugbug added a commit that referenced this issue Feb 9, 2019
(only in Windows app)
@hugbug hugbug closed this Feb 9, 2019
hugbug added a commit that referenced this issue Feb 13, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants