Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for MPS in the basehandler #3022

Open
agunapal opened this issue Mar 13, 2024 · 1 comment
Open

Add support for MPS in the basehandler #3022

agunapal opened this issue Mar 13, 2024 · 1 comment
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@agunapal
Copy link
Collaborator

馃殌 The feature

Based on a config in model-config.yaml, ex: mps:enable, Basehandler should move the model to mps https://pytorch.org/docs/stable/notes/mps.html

Motivation, pitch

To make use of the GPU on apple silicon mac

Alternatives

CPU support for apple silicon already exists

Additional context

No response

@agunapal agunapal added this to the v0.10.1 milestone Mar 13, 2024
@lxning lxning added the enhancement New feature or request label Mar 26, 2024
@lxning lxning added this to In progress in v0.11.0 lifecycle Mar 26, 2024
@lxning lxning moved this from In progress to Backlog in v0.11.0 lifecycle Mar 26, 2024
@lxning
Copy link
Collaborator

lxning commented Mar 27, 2024

The ticket requires:

  1. a model can be deployed to either CPU or GPU on Mac based on model-config.yaml
  2. a mac can have 1+ GPU. a model can be deployed to a specified GPU device id on Mac.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Development

No branches or pull requests

3 participants