Is it possible to make top level KFServing virtual service optional?

[nak3]

/kind feature
/kind question

Describe the solution you'd like

• As KFServing users, we would like to use KFServing without Istio. For example, KNative supports other Ingress like Contour, Kourier, Ambassador, Gloo, etc but KFServing always requires Istio.
• The top level KFServing's virtual service needs Istio but the virtualservice could be optional? We cannot access the URL in isvc, but it still can work with the URL in ksvc. The URL in isvc is necessary?

Anything else you would like to add:

• Especially this Ingress reconciler codes https://github.com/kubeflow/kfserving/blob/ec9e8ca569ea173de3717a41130e37cfb756ac8d/pkg/controller/v1beta1/inferenceservice/controller.go#L138-L147 should be optional. For example, if ingressGateway and ingressService are not configured, the controller could skip the reconciler and then KFServing controller does not touch VirtualService and Gateway at all.

[yuzisun]

@nak3 we are planning to make istio optional, the top level virtual service currently is used to route across multiple InferenceService components under a single domain for predictor, transformer and explainer which are created as separate knative services.

[nak3]

@yuzisun That's awesome. And that is really clear answer why the top level VirtualService is necessary 😄 Thank you so much.

[Ark-kun]

we are planning to make istio optional

Is there an issue to track?

[PatrickGhosn]

I would like to know that as well. For internal cluster only traffic, the local gateway is more than enough.

[grounded042]

Any update on this? The docs say

Istio ingress is recommended, but you can choose to install with other Ingress controllers.

but that is not the case as noted here.

I do not want to run the mess that is Istio in order to run kserve.

[swapkh91]

@yuzisun is istio optional now?
GKE has also deprecated Istio, but its working for some time as per this
The link says it wont support istio on existing clusters as well after September 2022