Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix azure disk mounter issue #52260

Merged
merged 1 commit into from
Sep 15, 2017
Merged

fix azure disk mounter issue #52260

merged 1 commit into from
Sep 15, 2017

Conversation

andyzhangx
Copy link
Member

@andyzhangx andyzhangx commented Sep 11, 2017 •
edited by wojtek-t
Loading

What this PR does / why we need it:
fix azure disk mounter issue, it's a P1 bug, it exists in 1.7, 1.8 release, should cherry pick to 1.7, 1.8

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
fixes #52261

consider following issue:

  1. A pod mounting an azure disk in a k8s agent
  2. The kubelet is restarted in that k8s agent
  3. The pod could not start up, it always reports error as following:

4d 1m 3065 kubelet, 14777acs9000 Warning FailedMount MountVolume.SetUp failed for volume "pvc-7a0cdeb9-92c7-11e7-b86b-000d3a36d70c" : azureDisk - No
t a mounting point for disk andykubewin175-dynamic-pvc-7a0cdeb9-92c7-11e7-b86b-000d3a36d70c on \var\lib\kubelet\pods\d146c023-92c7-11e7-b86b-000d3a36d70c\volumes\kubernetes.io~azure-disk\pvc-7a0cdeb9-92c7-11
e7-b86b-000d3a36d70c
4d 1m 3157 kubelet, 14777acs9000 Warning FailedMount Error syncing pod

Special notes for your reviewer:
If you take a look at following implementation of vsphere or gce, it will return nil instead of error:
https://github.com/kubernetes/kubernetes/blob/master/pkg/volume/vsphere_volume/vsphere_volume.go#L217-L220
https://github.com/kubernetes/kubernetes/blob/master/pkg/volume/gce_pd/gce_pd.go#L273-L275

The logic of return info parsing here, it's wrong to return error
https://github.com/kubernetes/kubernetes/blob/master/pkg/volume/util/operationexecutor/operation_generator.go#L469-L475

Release note:

Fix issue of mounting azure disks when kubelet restarts in the meantime.

@k8s-ci-robot k8s-ci-robot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 11, 2017
@k8s-ci-robot
Copy link
Contributor

Hi @andyzhangx. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@andyzhangx
Copy link
Member Author

@rootfs @jdumars

@k8s-github-robot k8s-github-robot added the do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. label Sep 11, 2017
@andyzhangx
Copy link
Member Author

/sig azure

@andyzhangx
Copy link
Member Author

@feiskyer for your interest

@feiskyer
Copy link
Member

/ok-to-test

@k8s-ci-robot k8s-ci-robot removed the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Sep 11, 2017
rootfs
rootfs reviewed Sep 11, 2017
View reviewed changes
pkg/volume/azure_dd/azure_mounter.go Outdated
@@ -79,7 +79,8 @@ func (m *azureDiskMounter) SetUpAt(dir string, fsGroup *int64) error {
return err
}
if !mountPoint {
return fmt.Errorf("azureDisk - Not a mounting point for disk %s on %s", diskName, dir)
glog.V(2).Infof("azureDisk - already mounted to target %s", dir)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't log or V(4)

@andyzhangx
fix azure disk mounter issue
2a9cb38 
change loginfo level from 2 to 4
@andyzhangx
Copy link
Member Author

/test pull-kubernetes-federation-e2e-gce

@jdumars jdumars added release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Sep 14, 2017
@jdumars jdumars added this to the v1.8 milestone Sep 14, 2017
@brendandburns
Copy link
Contributor

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 14, 2017
@k8s-github-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andyzhangx, brendandburns

Associated issue: 52261

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these OWNERS Files:

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

@k8s-github-robot k8s-github-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 14, 2017
@k8s-github-robot
Copy link

Automatic merge from submit-queue (batch tested with PRs 52452, 52115, 52260, 52290)

@k8s-github-robot k8s-github-robot merged commit b7953a7 into kubernetes:master Sep 15, 2017
@andyzhangx
Copy link
Member Author

@jdumars This PR should be cherry picked to v1.7, v.18, do you know who will do this? Thanks.

@feiskyer
Copy link
Member

@andyzhangx You could file cherry-pick PRs yourself, or I can help if you'd like.

@feiskyer
Copy link
Member

v1.8 cherry pick is not required because it is still in code freeze period.

@jdumars
Copy link
Member

jdumars commented Sep 15, 2017

@andyzhangx feiskyer is correct - this will be in 1.8, and you use the cherry pick tool to do the other two. Contact @wojtek-t for 1.7.x and @enisoc for 1.6.x

@andyzhangx andyzhangx mentioned this pull request Sep 18, 2017
Merged
andyzhangx added a commit to andyzhangx/kubernetes that referenced this pull request Sep 19, 2017
@andyzhangx
merge kubernetes#52260: fix azure disk mounter issue
9271316
@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-none Denotes a PR that doesn't merit a release note. labels Oct 10, 2017
k8s-github-robot pushed a commit that referenced this pull request Oct 10, 2017
Merge pull request #52627 from andyzhangx/automated-cherry-pick-of-#5…
25252be 
…2260-upstream-release-1.7

Automatic merge from submit-queue.

Automated cherry pick of #52260

Cherry pick of #52260  on release-1.7.
#52260 : fix azure disk mounter issue
@jdumars @rootfs for awareness 

**Release note**:

```
Fix issue of mounting azure disks when kubelet restarts in the meantime.
```
@jpbetz
Copy link
Contributor

jpbetz commented Oct 25, 2017

Removing 1.8 milestone as this was committed to 1.8 branch before 1.8.0 release.

@jpbetz jpbetz removed this from the v1.8 milestone Oct 25, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rootfs rootfs rootfs left review comments

Assignees

@pmorie pmorie

@thockin thockin

@brendandburns brendandburns

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

azureDisk - Not a mounting point for disk
10 participants
@andyzhangx @k8s-ci-robot @feiskyer @brendandburns @k8s-github-robot @jdumars @jpbetz @rootfs @pmorie @thockin