InferenceModel status conditions update

[ahg-g]

InferenceModel defines a Accepted condition type with three possible reasons:

• Pending, which is the default when the object is created
• ModelNameInUse, which is set if the ModelName is used by another InferenceModel
• Accepted, which is set when the model conforms to the state of the InferencePool it references.

The question here is if we want the epp to populate this condition, or some other component like the gateway controller.

As of right now, there is no reason for the gateway controller to be aware of the InferenceModel API, it only cares about the InferencePool to establish the connection between the proxy and the epp.

One problem with having the epp update the condition is that it may run in HA active-active mode, which complicates synchronizing status updates. It also adds the requirement on other epp implementations to do so.

This issue is to open and track the discussion on this topic, and bring awareness that the conditions are currently not being updated by any component.

[ahg-g]

One interesting direction here is to have a third component (inference-gateway-controller), that does status updates for all inferencePool/Model objects in the cluster, including other potential status tracking like reporting the number of ready endpoints on the inferencePool. There are however status updates on the InferencePool that should be owned by the gateway controller, and so this is also not a clear path since ideally the status of an object should be owned by a single controller.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale