-
Notifications
You must be signed in to change notification settings - Fork 38.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add some more logging so that we can debug PD delete errors. #10001
Conversation
@ixdy too. |
GCE e2e build/test passed for commit 0bf936d. |
@@ -214,7 +214,12 @@ func deletePD(pdName string) error { | |||
zone := testContext.CloudConfig.Zone | |||
|
|||
// TODO: make this hit the compute API directly. | |||
return exec.Command("gcloud", "compute", "--project="+testContext.CloudConfig.ProjectID, "disks", "delete", "--zone="+zone, pdName).Run() | |||
cmd := exec.Command("gcloud", "compute", "--project="+testContext.CloudConfig.ProjectID, "disks", "delete", "--zone="+zone, pdName) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any interest in stdout vs stderr?
If this is flaky, doing some parsing and retrying on transient errors might be a good thing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
combined output gets both. I want the data first, then we can see what to do with it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm pretty sure that the error is that the PD is still in use (i.e. attached). In that case, retrying the deletion probably won't help (although retrying the detachment probably will). As per @brendandburns, let's see what data this this unearths before deciding. I'm fairly certain that what we'll want here in the end will be explicit confirmation that the detachment has completed (not just that the detach API call has succeeded), and also retries on the PD deletion.
As an aside, I'm not 100% convinced that this test should actually be made to fail if only the PD deletion fails. That seems like a GCE rather than a K8S issue. But I could be convinced either way.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah, +1, I think this test should probably not fail if GCE delete via gcloud fails, esp. if the disk is no longer attached.
Sorry, I just saw this now. LGTM. |
Add some more logging so that we can debug PD delete errors.
Yes. I believe that the disk cannot be deleted because it is still attached to Q On Thu, Jun 18, 2015 at 9:45 PM, Satnam Singh notifications@github.com
|
@quinton-hoole