-
Notifications
You must be signed in to change notification settings - Fork 125
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Enhancement] Auto-deploy ML Model when predict #1148
Comments
+1, automating management of availability of specific models would simplify operations significantly |
@Zhangxunmt thanks for the proposal. This looks like a much asked feature. Couple of questions around the same:
|
@owaiskazi19 , the BWC is still valid. Nothing is changed from your side. You can still setup deploy and undeploy in the flow frameworks. The API experience remain the same too. The model registration of "_register?auto_deploy=true" is still valid. This change only handles the case when a cluster scale up and down, restart, or node replacement, etc. We need to auto-deploy the models in the "Prediction" stage so customers don't need to keep manually deploying again and again after each event. |
Currently the ML models are manually "deployed" or "loaded" into the memory which requires customers to manually invoke a "deploy" API before using any ML models. Also after usage, ml-common requires a manual "undeploy" or "unload" from end users. This is adding more overhead to the system and end users to use ml-common in the workflow.
We should build a auto-deploy mechanism to get rid of these "deploy" and "undeploy" operations in the workflow. Instead, we should auto deploy the model when customers use a model in the first time and setup a TTL to auto-undeploy from the system. In this way, the deploy and un-deploy APIs can be removed from the workflow and user experience are much simplified.
The text was updated successfully, but these errors were encountered: