Skip to content

fix(serve): ensure a single event loop for python 3.11#1528

Merged
rapids-bot[bot] merged 1 commit intoNVIDIA:developfrom
willkill07:wkk-ensure-nat-serve-works-with-py311
Jan 30, 2026
Merged

fix(serve): ensure a single event loop for python 3.11#1528
rapids-bot[bot] merged 1 commit intoNVIDIA:developfrom
willkill07:wkk-ensure-nat-serve-works-with-py311

Conversation

@willkill07
Copy link
Member

@willkill07 willkill07 commented Jan 30, 2026

Description

This PR ensures that a single event loop is in use at all times when issuing nat serve

  • catch KeyboardInterrupt cleanly to reduce error propagation
  • manually start the server rather than call uvicorn.run()

Closes

By Submitting this PR I confirm:

  • I am familiar with the Contributing Guidelines.
  • We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
    • Any contribution which contains commits that are not Signed-Off will not be accepted.
  • When the PR is ready for review, new or existing tests cover these changes.
  • When the PR is ready for review, the documentation is up to date with these changes.

Summary by CodeRabbit

  • Bug Fixes
    • Improved graceful shutdown handling across startup paths to properly catch interrupts, log the interruption, and ensure clean termination of the application and web front-end.

✏️ Tip: You can customize this high-level summary in your review settings.

@willkill07 willkill07 self-assigned this Jan 30, 2026
@willkill07 willkill07 requested a review from a team as a code owner January 30, 2026 19:24
@willkill07 willkill07 added bug Something isn't working non-breaking Non-breaking change labels Jan 30, 2026
@coderabbitai
Copy link

coderabbitai bot commented Jan 30, 2026

Walkthrough

Adds KeyboardInterrupt handling to the NAT CLI start command and refactors the FastAPI front-end startup from blocking uvicorn.run() to creating a uvicorn.Config and uvicorn.Server and awaiting server.serve(), with explicit interrupt handling and graceful shutdown logging.

Changes

Cohort / File(s) Summary
CLI interrupt handling
src/nat/cli/commands/start.py
Added KeyboardInterrupt handling in run_plugin: logs the interruption and returns None.
FastAPI server interrupt handling
src/nat/front_ends/fastapi/fastapi_front_end_plugin.py
Replaced uvicorn.run(...) with uvicorn.Config + uvicorn.Server and await server.serve(). Added try/except KeyboardInterrupt around serve() to log and shut down gracefully; preserved existing log formatting and Gunicorn branch.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: fixing event loop handling for Python 3.11 in the serve command, which aligns with the KeyboardInterrupt handling and uvicorn server refactoring in the changeset.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@willkill07 willkill07 force-pushed the wkk-ensure-nat-serve-works-with-py311 branch from 1779e43 to 588fe18 Compare January 30, 2026 19:27
@copy-pr-bot
Copy link

copy-pr-bot bot commented Jan 30, 2026

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@willkill07
Copy link
Member Author

/ok to test 588fe18

Copy link
Contributor

@yczhang-nv yczhang-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the fix - well-scoped and easy to understand

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@src/nat/front_ends/fastapi/fastapi_front_end_plugin.py`:
- Around line 202-218: The code currently constructs uvicorn.Config and calls
server.serve() which ignores multiprocess/reload; add a pre-check in the method
that builds the uvicorn.Config (the code block that reads self.front_end_config
and creates uvicorn.Config / server) to validate that if
self.front_end_config.workers > 1 or self.front_end_config.reload is True and
use_gunicorn (or the flag that controls using Gunicorn) is False, you reject the
combination: raise a clear ValueError (or log an error and exit) explaining that
workers>1 and reload=True require the Gunicorn/supervisor path and instruct the
user to enable use_gunicorn or set workers=1 and reload=False; reference
self.front_end_config.workers, self.front_end_config.reload, and the
uvicorn.Server(config).serve() call so the guard runs before creating
uvicorn.Server.

Signed-off-by: Will Killian <wkillian@nvidia.com>
@willkill07 willkill07 force-pushed the wkk-ensure-nat-serve-works-with-py311 branch from 588fe18 to 7661b64 Compare January 30, 2026 19:47
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@src/nat/front_ends/fastapi/fastapi_front_end_plugin.py`:
- Around line 214-217: In the except KeyboardInterrupt block that wraps await
server.serve(), replace the logger.info call with logger.exception (or
logger.error with exc_info=True) so the KeyboardInterrupt stack trace is
recorded; locate the try/except around server.serve() in
fastapi_front_end_plugin and change the logging call in the except
KeyboardInterrupt: handler to logger.exception("Received interrupt, shutting
down FastAPI server.") to preserve exception details.

@willkill07
Copy link
Member Author

/merge

@rapids-bot rapids-bot bot merged commit 2d1230b into NVIDIA:develop Jan 30, 2026
16 of 17 checks passed
willkill07 added a commit to willkill07/NeMo-Agent-Toolkit that referenced this pull request Feb 4, 2026
This PR ensures that a single event loop is in use at all times when issuing `nat serve`

- catch KeyboardInterrupt cleanly to reduce error propagation
- manually start the server rather than call `uvicorn.run()`

Closes

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/NeMo-Agent-Toolkit/blob/develop/docs/source/resources/contributing/index.md).
- We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
  - Any contribution which contains commits that are not Signed-Off will not be accepted.
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.



## Summary by CodeRabbit

* **Bug Fixes**
  * Improved graceful shutdown handling across startup paths to properly catch interrupts, log the interruption, and ensure clean termination of the application and web front-end.

<sub>✏️ Tip: You can customize this high-level summary in your review settings.</sub>

Authors:
  - Will Killian (https://github.com/willkill07)

Approvers:
  - Yuchen Zhang (https://github.com/yczhang-nv)

URL: NVIDIA#1528
bledden added a commit to bledden/nemo-agent-toolkit that referenced this pull request Feb 4, 2026
Backport the fix from PR NVIDIA#1528 (commit 2d1230b) to the release/1.4 branch.

Python 3.11 introduced stricter event loop requirements. When running
'nat serve', users encounter this error:

  Error: Runner.run() cannot be called from a running event loop

This happens because uvicorn.run() tries to create a new event loop
when one already exists from asyncio.run().

The fix changes how we start the uvicorn server:
- Use uvicorn.Config + uvicorn.Server instead of uvicorn.run()
- This lets us await server.serve() instead of blocking
- Add KeyboardInterrupt handling for clean shutdown

This maintains the same behavior while working correctly with Python
3.11's event loop restrictions.

Tested on develop branch in PR NVIDIA#1528. This is a clean backport with
no modifications to the original fix.

Fixes NVIDIA#1554

Signed-off-by: Brandon Ledden <bledden@users.noreply.github.com>
@willkill07 willkill07 deleted the wkk-ensure-nat-serve-works-with-py311 branch February 25, 2026 12:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants