Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only create 1MiB-aligned disk images #6754

Merged
merged 2 commits into from
Feb 17, 2022
Merged

Conversation

jean-edouard
Copy link
Contributor

@jean-edouard jean-edouard commented Nov 8, 2021

What this PR does / why we need it:
Recent libvirt/qemu now refuse to start if any disk is not 1MiB-aligned.
This PR ensures disk image creation/resize will always produce 1MiB-aligned images by rounding the size down to the nearest 1MiB boundary.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #6712

Special notes for your reviewer:
2G (2000000000) happens to be 1k-aligned (!), so I used 1G in the test to ensure mis-alignment. (1k used to be fine, and might still be in some cases)

Release note:

New and resized disks are now always 1MiB-aligned

@kubevirt-bot kubevirt-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. size/S labels Nov 8, 2021
@jean-edouard
Copy link
Contributor Author

/cc @rmohr

@rmohr
Copy link
Member

rmohr commented Nov 11, 2021

I think we also have this place:

func createSparseRaw(fullPath string, size int64) (err error) {

It is used by host-disks and PVCs (PVCs are mapped to hostdisks in virt-launcher). I think that this is also the main place which causes the issue. Can that be?

@jean-edouard
Copy link
Contributor Author

I think we also have this place:

func createSparseRaw(fullPath string, size int64) (err error) {

It is used by host-disks and PVCs (PVCs are mapped to hostdisks in virt-launcher). I think that this is also the main place which causes the issue. Can that be?

Oh, good catch, will add

@maya-r
Copy link
Contributor

maya-r commented Nov 17, 2021

/lgtm

@kubevirt-bot kubevirt-bot added lgtm Indicates that a PR is ready to be merged. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Nov 17, 2021
@kubevirt-bot kubevirt-bot removed lgtm Indicates that a PR is ready to be merged. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Nov 17, 2021
@maya-r
Copy link
Contributor

maya-r commented Nov 30, 2021

The CI failures aren't all flakes - can you run make generate?

@jean-edouard
Copy link
Contributor Author

The CI failures aren't all flakes - can you run make generate?

Thank you, sorry I missed that comment

@rmohr
Copy link
Member

rmohr commented Dec 17, 2021

/approve

@maya-r leaving the lgtm to you once you are satisfied.

@kubevirt-bot kubevirt-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 17, 2021
@jean-edouard
Copy link
Contributor Author

/retest

@kubevirt-bot kubevirt-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Dec 31, 2021
@kubevirt-bot kubevirt-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 4, 2022
@jean-edouard
Copy link
Contributor Author

/retest

@jean-edouard
Copy link
Contributor Author

/retest
@maya-r any chance you could have a second look and maybe re-LGTM? Thank you!

@jean-edouard jean-edouard changed the title Only create 4k-aligned disk images Only create 4MiB-aligned disk images Feb 15, 2022
@jean-edouard jean-edouard changed the title Only create 4MiB-aligned disk images Only create 1MiB-aligned disk images Feb 15, 2022
@jean-edouard
Copy link
Contributor Author

Added a commit to align on 1MiB instead of 4KiB

@awels
Copy link
Member

awels commented Feb 15, 2022

/lgtm
/cherrypick release-v0.49

@kubevirt-bot
Copy link
Contributor

@awels: once the present PR merges, I will cherry-pick it on top of release-v0.49 in a new PR and assign it to you.

In response to this:

/lgtm
/cherrypick release-v0.49

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@kubevirt-bot kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label Feb 15, 2022
@kubevirt-bot kubevirt-bot removed the lgtm Indicates that a PR is ready to be merged. label Feb 15, 2022
Signed-off-by: Jed Lejosne <jed@redhat.com>
@jean-edouard
Copy link
Contributor Author

/retest

size = util2.AlignImageSizeTo1MiB(size, log.Log.With("image", imagePath))
if size == 0 {
return fmt.Errorf("%s must be at least 1MiB", imagePath)
}
cmd := exec.Command("/usr/bin/qemu-img", "resize", preallocateFlag, imagePath, strconv.FormatInt(size, 10))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to add a check here that does the following:

  1. Get the current virtual size of the disk.img
  2. Make sure that aligned size >= current virtual size
  3. Do the actual resize.
    I think right now in some edge cases it is possible that when we align to 1Mi we will try to shrink an existing disk.img and that will fail.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is already taken care of by shouldExpandOffline(), which calls possibleGuestSize(), which accounts for alignment.
Am I missing something?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No you are right, I was seeing ghosts.

tests/storage/storage.go Outdated Show resolved Hide resolved
@awels
Copy link
Member

awels commented Feb 16, 2022

/lgtm

@kubevirt-bot kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label Feb 16, 2022
@aglitke
Copy link
Member

aglitke commented Feb 16, 2022

/hold cancel

@kubevirt-bot kubevirt-bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 16, 2022
@mhenriks
Copy link
Member

/approve

@kubevirt-bot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mhenriks, rmohr

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@mhenriks
Copy link
Member

/retest

1 similar comment
@awels
Copy link
Member

awels commented Feb 16, 2022

/retest

@kubevirt-commenter-bot
Copy link

/retest
This bot automatically retries jobs that failed/flaked on approved PRs.
Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

@kubevirt-bot kubevirt-bot merged commit 3d8e2c1 into kubevirt:main Feb 17, 2022
@kubevirt-bot
Copy link
Contributor

@awels: cannot checkout release-v0.49: error checking out release-v0.49: exit status 1. output: error: pathspec 'release-v0.49' did not match any file(s) known to git

In response to this:

/lgtm
/cherrypick release-v0.49

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@aglitke
Copy link
Member

aglitke commented Feb 17, 2022

/cherrypick release-0.49

@kubevirt-bot
Copy link
Contributor

@aglitke: new pull request created: #7247

In response to this:

/cherrypick release-0.49

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Virtualmachineinstance cannot start because of virthandler server error
10 participants