Fix validation for custom storageUri

[greenmoon55]

What this PR does / why we need it:
We introduced storage container CRD in #3060 which allows users to use any custom storage uri and customize the container spec.
For more information, please take a look at the design doc and the default StorageContainer CR.

However, we reject storage uris which are supported in a storage container CR because we have another validation in the validatingwebhook, which rejects uris not supported by the default storage initializer image.

This PR fixes the validation to allow custom uris that are supported by a storage container CR, and moves the validation logic from validatingwebhook to the controller.

Alternatives considered

1. Move the validation to storage initailizer injector (pod mutator) [WIP] Move storage uri check from validator to pod mutator #3131. It was abandoned because the error only appears in the logs which is confusing to users.
2. Fetch storage container CRs in validating webhook. This seems promising but there are some issues.
   • Creating an isvc might take longer (not sure)
   • It could be a bigger change because we need to pass the kubebuilder client to many places (search Validate()) including tests.
   • It is a good idea to validate the isvc based on another CR (storage container) in the validating webhook?

Breaking change

Some invalid Azure uris will pass the validation if they have the default StorageContainer CR installed because the last line allows all http(s) uris.

Examples:

https://blob.core.windows.net/triton/simple_string/
https://foo.blob.core.windows.net/"

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #3121

Type of changes
Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Feature/Issue validation/testing:

Please describe the tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.

Test 1 Custom storage uri + custom StorageContainer CR

apiVersion: "serving.kserve.io/v1alpha1"
kind: ClusterStorageContainer
metadata:
  name: abc
spec:
  container:
    name: storage-initializer
    image: kserve/storage-initializer:latest
    resources:
      requests:
        memory: 100Mi
        cpu: 100m
      limits:
        memory: 1Gi
        cpu: "1"
  supportedUriFormats:
    - prefix: abc://

aiVersion: "serving.kserve.io/v1beta1"
kind: InferenceService
metadata: 
    name: sklearn-iris
spec:
  predictor:
    model:
      modelFormat:
        name: sklearn
      storageUri: "abc://kfserving-examples/models/sklearn/1.0/model"

Inference service created and no error in the controller

Test 2 Custom storage uri + default StorageContainer CR + custom StorageContainer CR

Same result as above.

Test 3 Custom storage uri + no StorageContainer CR

➜  ~ k get isvc
NAME           URL   READY   PREV   LATEST   PREVROLLEDOUTREVISION   LATESTREADYREVISION   AGE
sklearn-iris                                                                               25s
➜  ~ k describe isvc sklearn-iris
Name:         sklearn-iris
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  serving.kserve.io/v1beta1
Kind:         InferenceService
Metadata:
  Creation Timestamp:  2023-09-22T00:49:23Z
  Finalizers:
    inferenceservice.finalizers
  Generation:        1
  Resource Version:  167831
  UID:               fc9100e6-4816-45d1-a41a-9bf9da26bbf6
Spec:
  Predictor:
    Model:
      Model Format:
        Name:  sklearn
      Name:    
      Resources:
      Storage Uri:  abc://kfserving-examples/models/sklearn/1.0/model
Events:
  Type     Reason         Age                 From                Message
  ----     ------         ----                ----                -------
  Warning  InternalError  10s (x13 over 31s)  v1beta1Controllers  StorageURI not supported: storageUri, must be one of: [gs://, s3://, pvc://, file://, https://, http://, hdfs://, webhdfs://] or match https://{}.blob.core.windows.net/{}/{} or be an absolute or relative local path. StorageUri [abc://kfserving-examples/models/sklearn/1.0/model] is not supported.

Test 4 Supported uri + no StorageContainer CR

apiVersion: "serving.kserve.io/v1beta1"
kind: InferenceService
metadata: 
    name: sklearn-iris
spec:
  predictor:
    model:
      modelFormat:
        name: sklearn
      storageUri: "gs://kfserving-examples/models/sklearn/1.0/model"

➜  ~ k get isvc
NAME           URL                                       READY   PREV   LATEST   PREVROLLEDOUTREVISION   LATESTREADYREVISION            AGE
sklearn-iris   http://sklearn-iris.default.example.com   True           100                              sklearn-iris-predictor-00001   59s

Test 5 Supported uri + default StorageContainer CR

Same result as above

• Logs

Special notes for your reviewer:

1. Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

Checklist:

• Have you added unit/e2e tests that prove your fix is effective or that this feature works?
• Has code been commented, particularly in hard-to-understand areas?
• Have you made corresponding changes to the documentation?

Release note:

Fix validation for custom storageUri. Now users to use any custom storageUri defined in storage container CRs.

[greenmoon55]

This uri is actually supported in the new CR because "https" is a valid prefix. Just want to note that this is a behavior change.

[greenmoon55]

ditto

[yuzisun]

what happens if the storage container CR does not exist?

[greenmoon55]

Added comments. If no CR exists, it will be exactly the same as the current behavior.

[yuzisun]

Thanks @greenmoon55 !

/lgtm
/approve

[kserve-oss-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: greenmoon55, yuzisun

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [yuzisun]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment