New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bug 1504927 - if apbs fail, mark them as failed. #534
Conversation
TEST failed provision marked as failed
|
TEST image pull
|
|
Basically what's happening is that we were never looking at the failed text from the exec AND we ignored the error altogether. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, nicely done. However, I want to test this but I can't with catasb. If you could provide the catasb ref to use or rebase your changes, that would be great.
| if err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| if bindOutput == nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎉 good call.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tricky...
| @@ -61,12 +62,19 @@ func monitorOutput(namespace string, podname string, log *logging.Logger) ([]byt | |||
| credsNotAvailable := errors.New("exit status 2") | |||
|
|
|||
| output, err := runtime.RunCommand("kubectl", "exec", podname, gatherCredentialsCMD, "--namespace="+namespace) | |||
|
|
|||
| // cannot exec container, pod is done | |||
| podFailed := strings.Contains(string(output), "current phase is Failed") | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
We were ignoring an error when an apb failed during provisioning and marking the apb as completed. This resulted in failed apbs looking as though they provisioned fine.
| podCompleted := strings.Contains(string(output), "current phase is Succeeded") || | ||
| strings.Contains(string(output), "cannot exec into a container in a completed pod") | ||
|
|
||
| if err == nil { | ||
| log.Notice("[%s] Bind credentials found", podname) | ||
| return output, nil | ||
| } else if podFailed { | ||
| // pod has completed but is in failed state | ||
| log.Notice("[%s] APB failed", podname) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this be logged as an error?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of log.Notice
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TESTED ![]()
* Bug 1504927 - if apbs fail, mark them as failed. We were ignoring an error when an apb failed during provisioning and marking the apb as completed. This resulted in failed apbs looking as though they provisioned fine.


We were ignoring an error when an apb failed during provisioning and
marking the apb as completed. This resulted in failed apbs looking as
though they provisioned fine.
Which issue this PR fixes (This will close that issue when PR gets merged)
Fixes #506 - apb provisioning sets ServiceInstance as successful when apb fails
Fixes Bug 1504927 - ASB Failed provision marked successful even on pod error
Fixes Bug 1470851 - Image Pull Error is being marked as Success