New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AWS: allow users to specify that their node ASG should use spot instances #21200
Comments
In addition to the spot based ASGs we've been using in our cloudformation scripts, we've also begun experimenting with spot fleets for multiple instances types and more granular control over instance termination on cluster spin down as our long running jobs complete. |
+1 for spot fleet, that would sweeet |
Does this need any new code to detect in a timely fashion that a node is getting evicted? Amazon recommends checking metadata for the termination notice every five seconds, but the AWS cloudprovider's isAlive() already considers a machine in "shutting down" state as not present. That could use better log messages, but is it already enough? I haven't tried it, but I assume that the spot instance would also be in "shutting down" state during those two minutes. And the nodecontroller runs every five seconds.. right? |
I believe an ASG using spot instances will work with today's code. Maybe we'll make some poor scheduling decisions if we don't listen to notifications, but it should look like our instances are just a little less reliable. |
I think we should probably leave this undocumented for now, until we have a better way to launch multiple sets of nodes, but it's great for cost savings while testing! Fix kubernetes#21200
Once we've built the master, we can build kubeconfig. By doing so, if we time out waiting for the nodes, the system is still configured correctly. In particular, spot instances can be slow to launch. Related to issue kubernetes#21200
Particularly now we have KUBE_USE_EXISTING_MASTER, it would make sense to allow users to add an ASG which uses spot instances.
Probably yet another env var.
The text was updated successfully, but these errors were encountered: