New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
High CPU usage when using --reload #338
Comments
You should only use |
Yes I use it only in dev, I just wanted to know if it was normal to hear my fans ramp up. |
We could bump down the frequency at which we check the file timestamps? Reloader implementation: https://github.com/encode/uvicorn/blob/master/uvicorn/supervisors/statreload.py |
* Changed reload check time from 100ms to 300ms Every 100ms seemed too fast and unnecessarily adding CPU load when debugging is turned on. If 1000ms is too much, maybe 500ms? This is in response to #338 * Changed to 300ms check time * Black compatible * Now black compatible
Should we make reload check time configurable? |
No to configuring reload check time - that’s too many dials. Tho there’s more efficient file watching systems (eg gunciorn has an optional dependency, and only falls back to stat checks if needed) we should probably do the same. It could also be reasonable for us to allow controls into which files are being watched (eg perhaps it’s having to scan a very large, unchanging, virtualenv, as well as the actual source code?) |
This make sense. I think that we can choose which folder to watch will be great. |
I found that there are |
@tomchristie How do you feel about using watchdog (https://github.com/gorakhargosh/watchdog) for file watching? It would add a dev dependency to the project, but it looks like it handles file watching very efficiently for most OS's with a fallback to the collect all files and then stat them every X milliseconds. I'm happy to attempt a PR with it. |
Yes, but probably (?) as an optional dependency, rather than mandatory. |
This should not be closed, it's very much still an issue, and it's a bug that uvicorn takes up 50% of CPU just for polling. |
Have you tried the new watchGod reloader?
What os are you on?
Mac os coupled with docker seems a common problem.
Without more information it's impossible to help you.
Le jeu. 7 mai 2020 à 6:45 PM, Stavros Korokithakis <notifications@github.com>
a écrit :
… This should not be closed, it's very much an issue and a bug that uvicorn
takes up 50% of CPU just for polling.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#338 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAINSPUQWG4NVOKR7VBWJ5LRQLQTZANCNFSM4HCAKVHQ>
.
|
I have not tried watchdog, is it available? My info is:
|
on 0.11.5 it should be imported if watchdog is found so either you tell us if this helps, what is the number of files you're watching ? Last time I tried on @rafalp misago project which watches quite a large number of files, I was at a nice 4% cpu all along in both statsreloader and watchgod mode, and on the same @rafalp was at 20% on a macos, so it's rather hard to tell what's going on, I'm on debian |
https://www.uvicorn.org/settings/
On my mac, I find this keeps the docker containers running with very low CPU when inactive - 1-3% using |
I'm watching around 20-30 files and it was taking up 50% CPU, I'll try watchdog and reply, thank you. |
Are you sure you are watching only 20-30 files? Are you using reload-dir to specify the directory and not adding in the virtualenv directory? That's an enormous idle load for FastAPI/uvicorn for just watching 20-30 files unless you are on a Raspberry Pi Linux server. |
I can check, but the virtualenv is in a different directory completely. That's why I was so aghast at the cpu usage. |
Yep:
uvicorn is taking up 45% of one core just watching those 43 files. With I have half a mind to write an abstraction library over inotify/kqueue/etc, as this is a very common thing that should exist. Could this issue be reopened until |
I made 2 tests for 5min on a small project ie 93 files watched, the other from @rafalp 889 files watched for 10 minutes. both with watchgod reloader, on my laptop, which is my slowest machine and not a beast (overall rank is 1018 according to https://www.cpubenchmark.net/cpu.php?cpu=Intel+Core+i5-6300U+%40+2.40GHz&id=2609).
for misago, aka the big project: for the small project: so it's fair to say that on 800+ files watched it's sweating, but on a rather small one we're at 1-2% I dont know what to add, would gladly help but as you can see I really can't reproduce. Maybe others will have ideas as to things to look at, I have to admit I'm at a loss right now. |
then use reload_dirs:
now fan is quiet and my MBP isn't too hot to put on my lap anymore ... wow! |
Here's another observation for Misago in Docker: When I disable Here's CPU without reloader: Here's with reloader: This CPU usage makes sense for statreload because its the one that keeps poking filesystem a lot. Watchgod is supposed to listeen for filesystem events and react to those instead. So I've checked what Watchgod is actually doing, and then realized it doesn't use filesystem events for changes detection. Instead it uses same approach that So we actually lie to people in the docs when we say that watchgod is faster because it uses filesystem events for change detection. I think we should implement something like Watchman to actually have performant in-dev code reload strategy. I know Django does this. There's also an issue of frequency at which filesystem is checked for changes. Django checks filesystem every second. Uvicorn every 250ms. Filesystem ops are relatively cheap in Docker, as long as they are infrequent. Here we are doing IO operation potentially thousands times a second. For comparison, here's my CPU graph after I've edited Uvicorn to pool every second: CPU usage is still noticeable, but it's not enough to get my laptop's fans to spin. So we still have performance issue. Its not for small projects or people running on watcher natively, but it's still a deal-breaker for people using dev setups based on docker. |
I had this issue with my project. With only By adding Maybe a decent solution is to have uvicorn ignore common virtualenv directory names, like |
I've learned I have to be very specific in what folders get mounted via docker, and what folders are watched to not end up using things like |
Let's close this. If what we've got in is not enough for somebody, we'll open new issue to track work for event-based reloader. |
ATTENTION: You need to: If the server starts up, you will then see:
|
This only affects the dev case and fixes it according to encode/uvicorn#338 (comment)
This only affects the dev case and fixes it according to encode/uvicorn#338 (comment)
This only affects the dev case and fixes it according to encode/uvicorn#338 (comment)
installing uvicorn[standard] fixed this problem for me: |
Note if your venv is in project root along with your .py , this will also majorly slow things down. Passing the additional setting to ignore scanning the venv reduced uvicorn usage to ~1% down from 100% on rpi4.
|
|
I am using https://gitlab.com/Taywee/asyncinotify/ for a |
Using the example code
and running
uvicorn main:app
, CPU usage is unnoticeable (less than 1% of 1 core).But when running with
uvicorn main:app --reload
, CPU usage jumps to 54% of 1 core.Environment:
python -V
: Python 3.7.2uvicorn
: 0.6.1uname -a
:Linux architect 5.0.4-arch1-1-ARCH #1 SMP PREEMPT Sat Mar 23 21:00:33 UTC 2019 x86_64 GNU/Linux
The text was updated successfully, but these errors were encountered: