Ability to change parameters of of the default Deployment created from ServingRuntime and InferenceService

[averyngo34]

/kind feature

Describe the solution you'd like
I'd like to have the ability to modify some of the Deployment.spec for the predictor deployment created from ServingRuntime and InferenceService. These parameters include strategy for adjusting rolling update strategy, template.spec.securityContext for adjusting the security context to name a few. The ideal solution would be to override any Deployment.spec inside the ServingRuntime and InferenceService.

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]
Currently, the deployment created by ServingRuntime and InferenceService use the default Kubernetes deployment values.

Links to the design documents:
[Optional, start with the short-form RFC template to outline your ideas and get early feedback.]
[Required, use the longer-form design doc template to specify and discuss your design in more detail]

[terrytangyuan]

/assign

[yuzisun]

I think we need to think about how we can unify these fields with serverless mode, I am not sure if we want to simply add the deployment.spec.

• in serverless mode, progressDeadlineSeconds is specified as a knative annotation and terminationGracePeriodSeconds is set according to the timeout field.
• enableServiceLinks is a legacy field which I think should not be exposed.
• deploymentStrategy is probably only applicable to raw deployment mode.
• securityContext should be already defined on the podSpec of inferenceservice

[terrytangyuan]

For deployment spec, we'd like to customize the following:

• spec.progressDeadlineSeconds: in serverless mode, this can be configured through KNative annotation serving.knative.dev/progress-deadline, I am proposing that we add a new field in ComponentExtensionSpec.ProgressDeadlineSeconds that can be used for both modes and document the precedence over the KNative annotation under serverless mode.
• spec.deploymentStrategy: expose this through a new field ComponentExtensionSpec.RawDeploymentStrategy.

For deployment template spec, we'd like to customize the following:

• spec.template.spec.terminationGracePeriodSeconds: can be set via the timeout field.
• spec.template.spec.enableServiceLinks: @averyngo34 this is a legacy field. Is it really necessary to customize it?
• spec.template.spec.securityContext: can already be customized in podSpec of inferenceservice

In short, we will only introduce two new fields:

1. ComponentExtensionSpec.ProgressDeadlineSeconds
2. ComponentExtensionSpec.RawDeploymentStrategy

WDYT? Anything I missed here? @averyngo34 @yuzisun

[averyngo34]

I agree with the approach you suggested. Thank you very much

[terrytangyuan]

Update: ComponentExtensionSpec.ProgressDeadlineSeconds is no longer needed by @averyngo34. Some known issues for reference kubernetes/kubernetes#106697, knative/serving#14835, knative/serving#14835 (comment)

[terrytangyuan]

We discussed in the community meeting today. I'll update my PR to introduce a new field ComponentExtensionSpec.DeploymentStrategy.