Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix parse devicePath issue on Azure Disk #74499

Merged
merged 1 commit into from Feb 26, 2019

Conversation

@andyzhangx
Copy link
Member

andyzhangx commented Feb 25, 2019

What type of PR is this?
/kind bug

What this PR does / why we need it:
fix parse devicePath issue on Azure Disk

There is some corner case that after attaching an Azure disk, disk device path is not displayed immediately under /dev/disk/azure/scsi1/lun?, instead it returns like /dev/sd?, which will make disk mount failure(like below failure), this PR also handles such corner case, if failed, it logs the error and then use azure API(diskController.GetDiskLun) to get disk LUN of that disk.

As I could observe, in VMSS, disk detection after disk attachment is slow, sometimes user would get below error, workaround is delete the pod and let k8s remount.

 Warning  FailedMount  1m (x6 over 13m)    kubelet, k8s-clcpyqmetcas-38966124-vmss000000  Unable to mount volumes for pod 'solr-0_default(6bbc79aa-2ab7-11e9-88e7-000d3ad25c8a)': timeout expired waiting for volumes to attach or mount for pod 'default'/'solr-0'. list of unmounted volumes=[solrhome]. list of unattached volumes=[solrhome]
 Warning  FailedMount  51s (x15 over 15m)  kubelet, k8s-clcpyqmetcas-38966124-vmss000000  MountVolume.WaitForAttach failed for volume 'pvc-04e61231-18c4-11e9-88e7-000d3ad25c8a' : cannot parse deviceInfo: /dev/sdf
Several unsuccessful restarts of k8s pod were attempted.  Then the k8s pod was removed from instance of VMSS (vmss000000) - so it was started on the new instance VMSS on which it started to work without problems.

Which issue(s) this PR fixes:

Fixes #74421

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

fix parse devicePath issue on Azure Disk

/kind bug
/assign @feiskyer
/priority important-soon
/sig azure

cc @khenidak

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

k8s-ci-robot commented Feb 25, 2019

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andyzhangx

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@andyzhangx

This comment has been minimized.

Copy link
Member Author

andyzhangx commented Feb 26, 2019

@feiskyer PTAL

@feiskyer
Copy link
Member

feiskyer left a comment

LGTM. Could you add unit tests for this?

@feiskyer
Copy link
Member

feiskyer left a comment

/LGTM

@k8s-ci-robot k8s-ci-robot added the lgtm label Feb 26, 2019

@k8s-ci-robot k8s-ci-robot merged commit b3ca826 into kubernetes:master Feb 26, 2019

16 checks passed

cla/linuxfoundation andyzhangx authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-cross Skipped.
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-100-performance Job succeeded.
Details
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-godeps Skipped.
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce-big Job succeeded.
Details
pull-kubernetes-local-e2e Skipped.
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details
pull-publishing-bot-validate Skipped.
tide In merge pool.
Details

k8s-ci-robot added a commit that referenced this pull request Feb 27, 2019

Merge pull request #74575 from andyzhangx/automated-cherry-pick-of-#7…
…4499-upstream-release-1.13

Automated cherry pick of #74499: fix parse devicePath issue on Azure Disk

k8s-ci-robot added a commit that referenced this pull request Mar 5, 2019

Merge pull request #74578 from andyzhangx/automated-cherry-pick-of-#7…
…4499-upstream-release-1.11

Automated cherry pick of #74499: fix parse devicePath issue on Azure Disk

@feiskyer feiskyer added this to Done in Cloud Provider Azure Mar 5, 2019

k8s-ci-robot added a commit that referenced this pull request Mar 7, 2019

Merge pull request #74577 from andyzhangx/automated-cherry-pick-of-#7…
…4499-upstream-release-1.12

Automated cherry pick of #74499: fix parse devicePath issue on Azure Disk
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.