New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bug 2029993: Prevent Machine from being considered provisioned until it exists in AWS #431
Bug 2029993: Prevent Machine from being considered provisioned until it exists in AWS #431
Conversation
Co-authored-by: Joel Speed <joel.speed@hotmail.co.uk>
@elmiko: This pull request references Bugzilla bug 2029993, which is invalid:
Comment In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
/label backport-risk-assessed |
missed a reference, fixing it up now |
Co-authored-by: Joel Speed <joel.speed@hotmail.co.uk>
/bugzilla refresh |
@elmiko: This pull request references Bugzilla bug 2029993, which is invalid:
Comment In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
…y issues Co-authored-by: Joel Speed <joel.speed@hotmail.co.uk>
Co-authored-by: Joel Speed <joel.speed@hotmail.co.uk>
/cherry-pick release-4.8 |
@elmiko: once the present PR merges, I will cherry-pick it on top of release-4.8 in a new PR and assign it to you. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
/approve |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: JoelSpeed The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
/retest |
@cblecker: This pull request references Bugzilla bug 2029993, which is valid. The bug has been moved to the POST state. 6 validation(s) were run on this bug
Requesting review from QA contact: In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
/lgtm |
/label cherry-pick-approved |
@elmiko: All pull requests linked via external trackers have merged: Bugzilla bug 2029993 has been moved to the MODIFIED state. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
@elmiko: #431 failed to apply on top of branch "release-4.8":
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
i'll manually pick to 4.8 on monday |
We are seeing instances where AWS is not returning the instance we just created within a second or two of the initial creation time, returning a 400 error saying that the instance does not exist.
This makes the machine go into failed right now, and, in conjunction with an MHC, can lead to leaked machines as the instances are deleted before AWS acknowledges the existence of the Machine.
This PR is an attempt to improve this experience.
It does:
The only flaw in this that I can currently think of is that if an instance is terminated before Exists ever succeeds, then we will requeue forever as Exists currently just silently ignores these kinds of errors. To improve this, Exists could return a "Instance was terminated" error which we can unwrap and handle specifically, though this would need to be handled in the core Machine controller, so needs further work.
this is a cherry-pick of openshift/machine-api-provider-aws#11