Asg offhours fixes

[gleework]

• EC2 instances have eligibility requirements before they can be stopped/started.
• When ASG suspends/resumes, it should only stop/start eligible instances.

[kapilt]

Thanks for the pull request! Much appreciated, If you don't mind could you sign the CLA at the bottom of the README? Direct link here: https://docs.google.com/forms/d/e/1FAIpQLSfwtl1s6KmpLhCY6CjiY8nFZshDwf_wrmNYx1ahpsNFXXmHKw/viewform

[gleework]

Glad to be able to contribute back in some small way! I think that I'm supposed to be covered by a Corporate CLA submitted by Gil Yehuda on 1/2/19. It should have covered me and some teammates from Oath (now Verizon Media). Would the Individual CLA be equivalent for me?

[kapilt]

we typically ask folks to do both, on the corp (if their doing as part of a corporate activity) and on the individual as teams/groups change membership. if you cover on the individual we're good to go. thanks!

[kapilt]

is this something that would be better handled with the use of a launch-filter? ie are the asgs here homongenous or are they being patched with spot and ephemeral instances at runtime ? i'm concerned about the additional api calls here and the implicit filtering, better would be an explicit filter on launch-config or adding in a new instance filter. but thats on the basis that the asgs are mostly homogenous. The only common use tool i'm aware of that does a lot of live patching of instances into asgs is something like https://github.com/AutoSpotting/AutoSpotting.. in which case we should probably just avoid the asg, buts thats easily done via adding a filter for the auto spot tag.

[tjstansell]

This is biting us now ... what code changes are required to get this merged? FWIW, when stopping instances and ASGs with spot, we get the following error:

botocore.exceptions.ClientError: An error occurred (UnsupportedOperation) when calling the StopInstances operation: You can't stop the Spot Instance 'i-08a8540xxxxxxxxxx' because it is associated with a one-time Spot Instance request. You can only stop Spot Instances associated with persistent Spot Instance requests.

So, ideally we'd be able to detect the difference between one-time spot instances and persistent ones and still stop persistent ones.

[glc-gplassard]

Hello, we are also facing the issue described above. Can we help to get this merged ?