Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add healthiness check to avoid hanging during model initialization #221

Merged
merged 3 commits into from Mar 30, 2023
Merged

Add healthiness check to avoid hanging during model initialization #221

merged 3 commits into from Mar 30, 2023

Conversation

krishung5
Copy link
Contributor

@krishung5 krishung5 commented Mar 22, 2023 •
edited

This PR adds healthiness check to fix the hanging issue when the stub process gets killed during function initialize.
Added test case: triton-inference-server/server#5554
Fixes: triton-inference-server/server#5381

@krishung5 krishung5 force-pushed the krish-pyt-oom branch 2 times, most recently from 2ee98a7 to 9be9481 Compare March 27, 2023 07:38
@krishung5 krishung5 changed the title Add stub healthiness check to avoid hanging Add healthiness check to avoid hanging during model initialization Mar 27, 2023
@krishung5 krishung5 marked this pull request as ready for review March 27, 2023 08:19
@krishung5 krishung5 mentioned this pull request Mar 27, 2023
Merged
Tabrizian
Tabrizian reviewed Mar 27, 2023
View reviewed changes
src/stub_launcher.cc Outdated Show resolved Hide resolved
Tabrizian
Tabrizian reviewed Mar 27, 2023
View reviewed changes
Copy link
Member

@Tabrizian Tabrizian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor comment, otherwise looks good!

src/stub_launcher.h Outdated Show resolved Hide resolved
Tabrizian
Tabrizian previously approved these changes Mar 28, 2023
View reviewed changes
@krishung5
Copy link
Contributor Author

@Tabrizian Rebased to include the fix for torch.hub.load compatibility issues for CI. Please re-approve, thanks.

@krishung5 krishung5 merged commit 1b6fa77 into main Mar 30, 2023
2 checks passed
@krishung5 krishung5 deleted the krish-pyt-oom branch March 30, 2023 23:54
tanmayv25 pushed a commit that referenced this pull request Apr 8, 2023
@krishung5 @tanmayv25
Add healthiness check to avoid hanging during model initialization (#221
36498fd 
)

* Add healthiness check to avoid hanging during model initialization

* Address comment

* Fix the type of the argument
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Tabrizian Tabrizian Tabrizian approved these changes

Assignees
No one assigned
Labels
None yet
Milestone
No milestone
2 participants
@krishung5 @Tabrizian