[Enhancement] Auto-deploy ML Model when predict

[Zhangxunmt]

Currently the ML models are manually "deployed" or "loaded" into the memory which requires customers to manually invoke a "deploy" API before using any ML models. Also after usage, ml-common requires a manual "undeploy" or "unload" from end users. This is adding more overhead to the system and end users to use ml-common in the workflow.

We should build a auto-deploy mechanism to get rid of these "deploy" and "undeploy" operations in the workflow. Instead, we should auto deploy the model when customers use a model in the first time and setup a TTL to auto-undeploy from the system. In this way, the deploy and un-deploy APIs can be removed from the workflow and user experience are much simplified.

[hijakk]

+1, automating management of availability of specific models would simplify operations significantly

[owaiskazi19]

@Zhangxunmt thanks for the proposal. This looks like a much asked feature. Couple of questions around the same:

1. How does the API experience will look like for the auto deploy model? Would we have a param when registering a model something like _register?auto_deploy=true?

2. Will we still support deploy API for users to deploy the model manually or will we deprecate the API?
   As part of automating the setup of ml-commons we support DeployModelStep and UndeployModelSetp in flow framework. We might need to deprecate them here as well.

[Zhangxunmt]

@owaiskazi19 , the BWC is still valid. Nothing is changed from your side. You can still setup deploy and undeploy in the flow frameworks. The API experience remain the same too. The model registration of "_register?auto_deploy=true" is still valid.

This change only handles the case when a cluster scale up and down, restart, or node replacement, etc. We need to auto-deploy the models in the "Prediction" stage so customers don't need to keep manually deploying again and again after each event.

[jngz-es]

#2206