Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make size optional when cloning using the Storage API #2222

Merged
merged 14 commits into from May 31, 2022

Conversation

alromeros
Copy link
Collaborator

@alromeros alromeros commented Mar 31, 2022

What this PR does / why we need it:

When cloning a Data Volume, the size of the target can be potentially obtainable via the source PVC. This discards the need to explicitly specify it.

Considering that, this PR aims to modify the cloning process in order to allow omitting the resources.request.storage field when cloning a PVC, which will be obtained from the source instead.

Special notes for your reviewer:

Release note:

Make size optional when cloning using Storage API

@kubevirt-bot kubevirt-bot added release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. dco-signoff: no Indicates the PR's author has not DCO signed all their commits. labels Mar 31, 2022
@kubevirt-bot
Copy link
Contributor

Hi @alromeros. Thanks for your PR.

I'm waiting for a kubevirt member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@kubevirt-bot kubevirt-bot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/M labels Mar 31, 2022
@awels
Copy link
Member

awels commented Mar 31, 2022

When you create a commit make sure to add -s to your command like this git commit -s this will automatically sign the commit so the DCO bot won't complain. To do this with an existing commit do git commit --amend -s to amend the commit message with the signing

@alromeros alromeros force-pushed the make-size-optional-when-cloning branch from 3f2ffdc to 0f25b40 Compare April 1, 2022 08:55
@kubevirt-bot kubevirt-bot added dco-signoff: yes Indicates the PR's author has DCO signed all their commits. and removed dco-signoff: no Indicates the PR's author has not DCO signed all their commits. labels Apr 1, 2022
@akalenyu
Copy link
Collaborator

akalenyu commented Apr 3, 2022

/ok-to-test

@kubevirt-bot kubevirt-bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Apr 3, 2022
pkg/controller/util.go Outdated Show resolved Hide resolved
// When cloning a PVC, if the target's size is not specified,
// said value can be attainable from the source PVC
if selectedCloneStrategy != NoClone {
targetRequest, hasTargetRequest := pvcSpec.Resources.Requests[corev1.ResourceStorage]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if I understand correctly at this point hasTargetRequest can't be false right? maybe return an error if somehow it is?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, It shouldn't be, but I wanted to make sure. I'll later add proper error handling, thanks!

Copy link
Contributor

@ShellyKa13 ShellyKa13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

over all looks good! had some questions. great job!

@kubevirt-bot kubevirt-bot added size/L and removed size/M labels Apr 6, 2022
pkg/controller/util.go Outdated Show resolved Hide resolved
@brybacki
Copy link
Contributor

brybacki commented Apr 7, 2022

Very good direction.
👍

@alromeros alromeros force-pushed the make-size-optional-when-cloning branch 2 times, most recently from f2c09fb to 02c6215 Compare April 7, 2022 17:17
@kubevirt-bot kubevirt-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 7, 2022
@alromeros alromeros force-pushed the make-size-optional-when-cloning branch from 1f8cd07 to d556888 Compare May 26, 2022 07:36
Signed-off-by: Alvaro Romero <alromero@redhat.com>
…etection pod

This commit introduces additional handling in case of error after and during the size-detection pod is created.

It also updates several related unit tests.

Signed-off-by: Alvaro Romero <alromero@redhat.com>
@alromeros alromeros force-pushed the make-size-optional-when-cloning branch from d556888 to 19c4ba4 Compare May 26, 2022 10:25
@alromeros
Copy link
Collaborator Author

/retest

@kubevirt-bot kubevirt-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-none Denotes a PR that doesn't merit a release note. labels May 26, 2022
@alromeros
Copy link
Collaborator Author

/test pull-containerized-data-importer-e2e-hpp-latest

@brybacki
Copy link
Contributor

good, I cannot find the new test but I think it is ready to go
/lgtm

But it is a big PR so it would be good to have another reviewer do the last check.
@awels WDYT?

@kubevirt-bot kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label May 27, 2022
Copy link
Contributor

@brybacki brybacki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that in detectCloneSize, the branch when source is block needs additional logic

		targetSize, err = inflateSizeWithOverhead(r.client, imgSize, pvcSpec)

@kubevirt-bot kubevirt-bot removed the lgtm Indicates that a PR is ready to be merged. label May 27, 2022
@@ -916,6 +975,40 @@ func cloneStrategyToCloneType(selectedCloneStrategy cloneStrategy) string {
return ""
}

// reconcileSizelessClone handles the reconciling process if the size-detection mechanism is not able to obtain the img size
func (r *DatavolumeReconciler) reconcileSizelessClone(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When seeing this function this was screaming, use me in all 3 variations.
The csi clone, the snapshot clone, and the no target size clone. I would call it something like isSourceReadyToClone that returns a boolean and error. The error is to catch the normal errors like now. When calling this function I would do it something like this for the csi clone and smart clone options.

if readyToClone, err := r.isSourceReadyToClone(dv, cloneStrategy); err != nil {
  return reconcile.Result{}, err
} else if !readyToClone {
  return reconcile.Result{Requeue: true},
    r.updateCloneStatusPhase(cdiv1.CloneScheduled, datavolume, nil, selectedCloneStrategy)
}

For the size less clone option something like this

else if !done {
  if readyToClone, err := r.isSourceReadyToClone(dv, cloneStrategy); err != nil {
    return reconcile.Result{}, err
  } else if !readyToClone {
    return reconcile.Result{Requeue: true},
      r.updateCloneStatusPhase(cdiv1.CloneScheduled, datavolume, nil, cloneStrategy)
  }
  return reconcile.Result{}, nil
}

Expect(sourceMD5).To(Equal(targetMD5))
},
Entry("Block to block (empty storage size)", v1.PersistentVolumeBlock, v1.PersistentVolumeBlock),
Entry("Block to filesystem (empty storage size)", v1.PersistentVolumeBlock, v1.PersistentVolumeFilesystem),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if you are skipping due to that bug, I would remove the manual skip code above and make this a PEntry (which will skip the entry).

…y storage size

* Modified the size-detection mechanism so we account for fsOverhead when cloning to filesystem volume mode in all cases
* Clean up the code for reconciling when cloning a PVC that is not ready
* Minor fix in functional test so it works when cloning from block to filesystem volume mode

Signed-off-by: Alvaro Romero <alromero@redhat.com>
@alromeros alromeros requested review from brybacki and awels May 27, 2022 16:04
@alromeros
Copy link
Collaborator Author

/test pull-containerized-data-importer-e2e-ceph

@kubevirt-bot kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label May 27, 2022
@brybacki
Copy link
Contributor

/lgtm

Copy link
Member

@awels awels left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@kubevirt-bot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: awels

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubevirt-bot kubevirt-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 31, 2022
@kubevirt-bot kubevirt-bot merged commit 7eebbfa into kubevirt:main May 31, 2022
@alromeros alromeros deleted the make-size-optional-when-cloning branch May 31, 2022 16:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. lgtm Indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants