Use asyncio.Event to stop stream in heartbeat route #7932
Conversation
🪼 branch checks and previews
Install Gradio from this PR pip install https://gradio-builds.s3.amazonaws.com/4751e5e682d2cd34cbde0f35dcb8e1032831a3a2/gradio-4.25.0-py3-none-any.whlInstall Gradio Python Client from this PR pip install "gradio-client @ git+https://github.com/gradio-app/gradio@4751e5e682d2cd34cbde0f35dcb8e1032831a3a2#subdirectory=client/python" |
🦄 change detectedThis Pull Request includes changes to the following packages.
With the following changelog entry.
Maintainers or the PR author can modify the PR title to modify this entry.
|
freddyaboulton
requested review from
abidlabs,
aliabid94,
aliabd,
dawoodkhan82,
pngwn and
hannahblair
|
Can you describe what this PR is doing conceptually @freddyaboulton? Not sure I understand it |
gradio/routes.py
Outdated
| # For some reason tests are failing when using events only on CI | ||
| # so we use a different approach for CI but | ||
| # locally and in deployment we should use the event-based approach |
There was a problem hiding this comment.
Slightly concerned about this. What's true on the CI runners may be true in some of users' deployments
There was a problem hiding this comment.
Agreed. Do you know why it doesn;t work on CI? Ideally we use the event approach everywhere
There was a problem hiding this comment.
Looking into it. It may be a 3.8 issue.
|
We need to stop the heartbeat messages when the server is shutting down because starlette waits for all connections to close before exiting. The old way we were doing that was by polling the is_running prop but that was taking up too much of the event loop. Some gradio spaces like gradio/calculator were taking a really long time to load (30 seconds). So we're switching to this event-based approach that works better than polling. |
|
Is the PR Deploy Space building for you @freddyaboulton? I'm seeing errors even after factory restarting |
|
Nice PR, the event approach is cleaner but I don't think this should be merged until we know why it doesn't work on E2E tests. |
|
@abidlabs The deployed space is up! |
freddyaboulton
force-pushed
the
use-event-to-stop
branch
from
ea3bfb6
to
ebf3db1
Compare
gradio/routes.py
Outdated
| # For some reason tests are failing when using events only on CI | ||
| # so we use a different approach for CI but | ||
| # locally and in deployment we should use the event-based approach | ||
| if os.getenv("GRADIO_IS_E2E_TEST") and sys.version_info < (3, 9): |
There was a problem hiding this comment.
Not sure what's happening with CI but it's failing when using the "else" branch here. This makes it so that locally we use the "else" branch and on CI we use the other branch.
Tested both locally and they work.
gradio/routes.py
Outdated
| # For some reason tests are failing when using events only on CI | ||
| # so we use a different approach for CI but | ||
| # locally and in deployment we should use the event-based approach |
There was a problem hiding this comment.
Looking into it. It may be a 3.8 issue.
|
CI is finally green @abidlabs @aliabid94 ! It was a python 3.8 issue |
|
Thanks for the reviews everyone!! |
Description
This may fix some of the slowdowns we've seen in spaces loading recently.
🎯 PRs Should Target Issues
Before your create a PR, please check to see if there is an existing issue for this change. If not, please create an issue before you create this PR, unless the fix is very small.
Not adhering to this guideline will result in the PR being closed.
Tests
PRs will only be merged if tests pass on CI. To run the tests locally, please set up your Gradio environment locally and run the tests:
bash scripts/run_all_tests.shYou may need to run the linters:
bash scripts/format_backend.shandbash scripts/format_frontend.sh