Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix parse devicePath issue on Azure Disk #74499

Merged

Conversation

andyzhangx
Copy link
Member

What type of PR is this?
/kind bug

What this PR does / why we need it:
fix parse devicePath issue on Azure Disk

There is some corner case that after attaching an Azure disk, disk device path is not displayed immediately under /dev/disk/azure/scsi1/lun?, instead it returns like /dev/sd?, which will make disk mount failure(like below failure), this PR also handles such corner case, if failed, it logs the error and then use azure API(diskController.GetDiskLun) to get disk LUN of that disk.

As I could observe, in VMSS, disk detection after disk attachment is slow, sometimes user would get below error, workaround is delete the pod and let k8s remount.

 Warning  FailedMount  1m (x6 over 13m)    kubelet, k8s-clcpyqmetcas-38966124-vmss000000  Unable to mount volumes for pod 'solr-0_default(6bbc79aa-2ab7-11e9-88e7-000d3ad25c8a)': timeout expired waiting for volumes to attach or mount for pod 'default'/'solr-0'. list of unmounted volumes=[solrhome]. list of unattached volumes=[solrhome]
 Warning  FailedMount  51s (x15 over 15m)  kubelet, k8s-clcpyqmetcas-38966124-vmss000000  MountVolume.WaitForAttach failed for volume 'pvc-04e61231-18c4-11e9-88e7-000d3ad25c8a' : cannot parse deviceInfo: /dev/sdf
Several unsuccessful restarts of k8s pod were attempted.  Then the k8s pod was removed from instance of VMSS (vmss000000) - so it was started on the new instance VMSS on which it started to work without problems.

Which issue(s) this PR fixes:

Fixes #74421

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

fix parse devicePath issue on Azure Disk

/kind bug
/assign @feiskyer
/priority important-soon
/sig azure

cc @khenidak

@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. kind/bug Categorizes issue or PR as related to a bug. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. sig/azure release-note Denotes a PR that will be considered when it comes time to generate release notes. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Feb 25, 2019
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andyzhangx

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. sig/storage Categorizes an issue or PR as relevant to SIG Storage. labels Feb 25, 2019
@andyzhangx
Copy link
Member Author

@feiskyer PTAL

Copy link
Member

@feiskyer feiskyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Could you add unit tests for this?

Copy link
Member

@feiskyer feiskyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/LGTM

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 26, 2019
@k8s-ci-robot k8s-ci-robot merged commit b3ca826 into kubernetes:master Feb 26, 2019
k8s-ci-robot added a commit that referenced this pull request Feb 27, 2019
…4499-upstream-release-1.13

Automated cherry pick of #74499: fix parse devicePath issue on Azure Disk
k8s-ci-robot added a commit that referenced this pull request Mar 5, 2019
…4499-upstream-release-1.11

Automated cherry pick of #74499: fix parse devicePath issue on Azure Disk
@feiskyer feiskyer added this to Done in Provider Azure Mar 5, 2019
k8s-ci-robot added a commit that referenced this pull request Mar 7, 2019
…4499-upstream-release-1.12

Automated cherry pick of #74499: fix parse devicePath issue on Azure Disk
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/storage Categorizes an issue or PR as relevant to SIG Storage. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
Development

Successfully merging this pull request may close these issues.

cannot parse deviceInfo on azure disk mount
3 participants