Allow users to force a target_device #15
Conversation
Xaenalt
requested review from
taneem-ibrahim,
cfchase,
Jooho,
lucferbux and
lugi0
device in OVMS This is a workaround to the adapter that allows a user to set OVMS_FORCE_TARGET_DEVICE in the builtInAdapter.env section. This is then written into the config for each model that the model server manages Signed-off-by: Sean Pryor <spryor@redhat.com>
Xaenalt
force-pushed
the
ovms-force-device-opendatahub
branch
from
f52aade
to
453f769
Compare
|
@Xaenalt I have couple of questions.
|
|
@Jooho Needs to go in 1.27 release by code freeze tomorrow. This is temporary until the AUTO plugin supports Nvidia, but may be desired longer term to enable more of the functions of the target_device field. It may get restructured later on if that's the case |
Xaenalt
changed the title
Xaenalt
changed the title
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: Jooho, Xaenalt The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/cherry-pick release-v0.11.0-alpha |
|
@Xaenalt: only opendatahub-io org members may request cherry picks. You can still do the cherry-pick manually. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
/cherry-pick release-v0.11.0-alpha |
|
@Jooho: only opendatahub-io org members may request cherry picks. You can still do the cherry-pick manually. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
This is a workaround to the adapter that allows a user to set OVMS_FORCE_TARGET_DEVICE in the builtInAdapter.env section. This is then written into the config for each model that the model server manages
This is needed right now because when on a node with an Nvidia GPU, OVMS does not always ensure that the model goes onto the right device.
The changes needed to the runtime are:
Note, this actually accepts any value from https://github.com/openvinotoolkit/model_server/blob/main/docs/accelerators.md so it enables a lot of the interesting behavior OVMS is capable of
Same changes as red-hat-data-services#3