GPU serving setting

[cfchase]

Allow ServingRuntimes to specify allowed number of GPUs

Closes: #940

Description

I've added an annotation to the ServingRuntime specification in the servingruntimes-configmap.yaml

data:
  override-config: |
    apiVersion: serving.kserve.io/v1alpha1
    kind: ServingRuntime
    metadata:
      annotations:
        opendatahub.io/gpu-setting: '1' #same as notebook gpuSetting

This is meant as a stopgap feature until full support for GPUs is implemented and tested, but hopefully follows the same development flow for servingruntimes-configs without the annotation.

How Has This Been Tested?

Change redhat-ods-applications servingruntimes-config to add an annotation to the override servingruntime

opendatahub.io/gpu-setting: '1'
opendatahub.io/gpu-setting: '5'
opendatahub.io/gpu-setting: '0'
opendatahub.io/gpu-setting: 'hidden'
no annotation

Verify the gpu selection shows as expected and test .

Configure a model server is spawned with correct limits/requests/affinities

Verify any pods spawned for the model server contain the correct limits/requests/affinities

Tested on OCP 4.11 with GPU operator installed using live build quay.io/cfchase/rhods-operator-live-catalog:1.22.0-w7

• The commits are squashed in a cohesive manner and have meaningful messages.
• Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
• The developer has manually tested the changes and verified that the changes work

[andrewballantyne]

Looks pretty good on face-value -- I'll need to test it tomorrow; maybe @lucferbux can test this in his morning.

I think due to the amount of conflicts we will have -- you should probably merge first and I can go back through and try to fix all my PRs. Otherwise we'll be sharing in the conflicts between my various PRs.

[andrewballantyne]

@cfchase I see that this is a WIP -- hopefully it's reasonably done otherwise we're going to have a race of conflicts tomorrow.

[cfchase]

@andrewballantyne I should have removed the WIP last night after I finished testing. Removed it

[cfchase]

@andrewballantyne nevermind.... WIP back on while I rebase off of yours.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andrewballantyne

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [andrewballantyne]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment