Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ml): more graceful shutdown #5689

Merged
merged 1 commit into from
Dec 14, 2023
Merged

fix(ml): more graceful shutdown #5689

merged 1 commit into from
Dec 14, 2023

Conversation

mertalev
Copy link
Contributor

Description

This PR addresses some of the issues with the ML service's automatic restart.

  1. The service now performs more checks before beginning a shutdown.
    • In particular, it now checks that there are no active requests and that no model is being loaded/downloaded (i.e. the model loading lock is not acquired).
  2. The service does more cleanup.
    • It now shuts down the thread pool and closes log handlers.
    • It lets Uvicorn perform its own cleanup by sending a shutdown signal instead of canceling async tasks and stopping the loop.
  3. Because of the use of a shutdown signal, it presents an INFO log instead of an ERROR log when restarting.
  4. It sets the LD_BIND_NOW flag, which results in more deterministic RAM usage.

How Has This Been Tested?

All tasks work as normal and the server is able to restart after idling. RAM usage as inspected from htop is normal as well, including after restart.

Copy link

cloudflare-pages bot commented Dec 14, 2023

Deploying with  Cloudflare Pages  Cloudflare Pages

Latest commit: c44915b
Status: ✅  Deploy successful!
Preview URL: https://01c68da6.immich.pages.dev
Branch Preview URL: https://fix-ml-improve-shutdown.immich.pages.dev

View logs

Copy link
Contributor

@jrasm91 jrasm91 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@alextran1502 alextran1502 merged commit d729c86 into main Dec 14, 2023
19 checks passed
@alextran1502 alextran1502 deleted the fix/ml-improve-shutdown branch December 14, 2023 19:51
jonhnet pushed a commit to jonhnet/immich that referenced this pull request Dec 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants