Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add NVIDIA MPS documentation to doc index #2205

Merged
merged 2 commits into from
Mar 29, 2023
Merged

Add NVIDIA MPS documentation to doc index #2205

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
616de44
Add NVIDIA MPS documentation to doc index
mreso Mar 28, 2023
be58cd5
Add mps doc to content + change title
mreso Mar 28, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/contents.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
model_zoo
request_envelopes
server
mps
snapshot
sphinx/requirements
torchserve_on_win_native
Expand Down
1 change: 1 addition & 0 deletions docs/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,3 +49,4 @@ TorchServe is a performant, flexible and easy to use tool for serving PyTorch ea
* [TorchServe on Kubernetes](https://github.com/pytorch/serve/blob/master/kubernetes/README.md#torchserve-on-kubernetes) - Demonstrates a Torchserve deployment in Kubernetes using Helm Chart supported in both Azure Kubernetes Service and Google Kubernetes service
* [mlflow-torchserve](https://github.com/mlflow/mlflow-torchserve) - Deploy mlflow pipeline models into TorchServe
* [Kubeflow pipelines](https://github.com/kubeflow/pipelines/tree/master/samples/contrib/pytorch-samples) - Kubeflow pipelines and Google Vertex AI Managed pipelines
* [NVIDIA MPS](mps.md) - Use NVIDIA MPS to optimize multi-worker deployment on a single GPU
2 changes: 1 addition & 1 deletion docs/mps.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Enabling NVIDIA MPS in TorchServe
# Running TorchServe with NVIDIA MPS
In order to deploy ML models, TorchServe spins up each worker in a separate processes, thus isolating each worker from the others.
Each process creates its own CUDA context to execute its kernels and access the allocated memory.

Expand Down