Allow specifying download format in vmexport #10400

alromeros · 2023-09-07T15:47:59Z

What this PR does / why we need it:

This PR aims to add a new --raw flag in virtctl vmexport, so users can request downloading the raw image instead of the compressed one. It also adds a --decompress flag to download and decompress gzipped files.
Usage example:

  # Download and extract a volume from an already existing VirtualMachineExport
  {{ProgramName}} vmexport download vm1-export --volume=volume1 --output=disk.img --format=raw

Special notes for your reviewer:

Planning to add a flag to decompress the image if needed too.

Release note:

Add new vmexport flags to download raw images, either directly (--raw) or by decompressing (--decompress) them

kubevirt-bot · 2023-09-07T15:48:01Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

mhenriks · 2023-09-07T17:08:59Z

pkg/virtctl/vmexport/vmexport.go

@@ -261,6 +264,7 @@ func NewVirtualMachineExportCommand(clientConfig clientcmd.ClientConfig) *cobra.
 	cmd.Flags().StringVar(&localPort, "local-port", "0", "Defines the specific port to be used in port-forward.")
 	cmd.Flags().BoolVar(&includeSecret, "include-secret", false, "When used with manifest and set to true include a secret that contains proper headers for CDI to import using the manifest")
 	cmd.Flags().BoolVar(&exportManifest, "manifest", false, "Instead of downloading a volume, retrieve the VM manifest")
+	cmd.Flags().BoolVar(&rawImg, "raw", false, "Used to download the raw image. By default, we always attempt to download the compressed one")


I think it would be better to always download the compressed file but have virtctl unzip while downloading/writing to the target

Just added a --decompress flag so we allow both downloading raw and decompressing the gzipped file. wdyt?

This commit adds a new "--raw" flag in virtctl vmexport, so users can request downloading the raw image instead of the compressed one. Signed-off-by: Alvaro Romero <alromero@redhat.com>

This commit adds a new virtctl vmexport flag so gzipped images can be downloaded and decompressed in the same step. Signed-off-by: Alvaro Romero <alromero@redhat.com>

alromeros · 2023-09-12T17:10:36Z

/test pull-kubevirt-e2e-k8s-1.27-ipv6-sig-network
/test pull-kubevirt-e2e-k8s-1.27-sig-compute

mhenriks · 2023-09-12T20:15:13Z

pkg/virtctl/vmexport/vmexport.go

@@ -70,6 +71,8 @@ const (
 	SNAPSHOT_FLAG       = "--snapshot"
 	INSECURE_FLAG       = "--insecure"
 	KEEP_FLAG           = "--keep-vme"
+	RAW_FLAG            = "--raw"
+	DECOMPRESS_FLAG     = "--decompress"


I'm a little confused here. Why two options? Don't they ultimately bean the same thing?

What do you think of this, forget about these to flags for a sec. What if we:

always fetch the gzip image

ad "-o" and/or "--output" arg that accepts "gzip" or "raw" (gzip is default to be consistent with current invocation)

if raw is specified, unzip the image while downloading

So I initially wanted to allow both downloading the raw file (--raw) and downloading and decompressing the gzipped one (--decompress). I know they produce the same output but thought it was nice to allow both ways (some users might care about downloading bigger files, others won't). I also prefer boolean flags since they are more immediate and less confusing. But if you prefer to have a single flag to specify the output I can change it, just wanted to share the rationale behind this design.

I think this is a case of too many knobs leading to user confusion.

Okay so I added a --format flag to specify either raw or gzip. --output is already a flag so couldn't use that one.

dominikholler · 2023-09-13T06:36:53Z

pkg/virtctl/vmexport/vmexport.go

 	bar.Start()

+	if decompress {
+		// Create a new gzip reader
+		gzipReader, err := gzip.NewReader(barRd)


Optional nice to have: Is this code able to create a sparse file?

I guess we can decompress the file and then rewrite it to be sparse but that sounds out of scope for the PR imo. gzip can't directly decompress it to be sparse, maybe there's some other package that does it. I can take a better look.

Signed-off-by: Alvaro Romero <alromero@redhat.com>

alromeros · 2023-09-18T13:57:39Z

/retest-required

alromeros · 2023-09-18T16:21:30Z

/test pull-kubevirt-e2e-k8s-1.26-sig-storage

mhenriks · 2023-09-19T14:03:12Z

@alromeros is this still WIP?

alromeros · 2023-09-19T14:07:54Z

@alromeros is this still WIP?

oops, forgot to change the PR name.

alromeros · 2023-09-20T14:54:17Z

/test pull-kubevirt-e2e-k8s-1.26-sig-storage

mhenriks · 2023-09-20T20:47:34Z

/lgtm
/approve

kubevirt-bot · 2023-09-20T20:47:42Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mhenriks

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [mhenriks]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

kubevirt-bot · 2023-09-21T00:56:38Z

@alromeros: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-kubevirt-e2e-k8s-1.28-sig-storage	`7dfa5ba`	link	unknown	`/test pull-kubevirt-e2e-k8s-1.28-sig-storage`

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

kubevirt-commenter-bot · 2023-09-21T03:36:25Z

/retest-required
This bot automatically retries required jobs that failed/flaked on approved PRs.
Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

kubevirt-bot requested review from maiqueb and vasiliy-ul September 7, 2023 15:48

kubevirt-bot added the size/M label Sep 7, 2023

alromeros force-pushed the specify-vmexport-format branch from 33a9770 to 2913913 Compare September 7, 2023 16:49

alromeros marked this pull request as ready for review September 7, 2023 16:51

kubevirt-bot requested review from acardace and iholder101 September 7, 2023 16:51

mhenriks reviewed Sep 7, 2023

View reviewed changes

alromeros added 2 commits September 12, 2023 10:33

Allow specifying download format in vmexport

bd0d3bf

This commit adds a new "--raw" flag in virtctl vmexport, so users can request downloading the raw image instead of the compressed one. Signed-off-by: Alvaro Romero <alromero@redhat.com>

Add flag to decompress gzip downloads

6037b06

This commit adds a new virtctl vmexport flag so gzipped images can be downloaded and decompressed in the same step. Signed-off-by: Alvaro Romero <alromero@redhat.com>

alromeros force-pushed the specify-vmexport-format branch from 2913913 to 6037b06 Compare September 12, 2023 12:58

kubevirt-bot added size/L and removed size/M labels Sep 12, 2023

mhenriks reviewed Sep 12, 2023

View reviewed changes

dominikholler reviewed Sep 13, 2023

View reviewed changes

Merge format-related flags in vmexport into a single "--format" flag

7dfa5ba

Signed-off-by: Alvaro Romero <alromero@redhat.com>

alromeros mentioned this pull request Sep 19, 2023

[WIP] Investigate failing test in k8s-1.26-sig-storage lane #10449

Closed

alromeros changed the title ~~[WIP] Allow specifying download format in vmexport~~ Allow specifying download format in vmexport Sep 19, 2023

kubevirt-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 19, 2023

kubevirt-bot assigned mhenriks Sep 20, 2023

kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label Sep 20, 2023

kubevirt-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 20, 2023

kubevirt-bot merged commit eb743d9 into kubevirt:main Sep 21, 2023
35 of 36 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow specifying download format in vmexport #10400

Allow specifying download format in vmexport #10400

alromeros commented Sep 7, 2023 •

edited

kubevirt-bot commented Sep 7, 2023

mhenriks Sep 7, 2023

alromeros Sep 12, 2023

alromeros commented Sep 12, 2023

mhenriks Sep 12, 2023

alromeros Sep 13, 2023

mhenriks Sep 13, 2023

alromeros Sep 14, 2023

dominikholler Sep 13, 2023

alromeros Sep 14, 2023

alromeros commented Sep 18, 2023

alromeros commented Sep 18, 2023

mhenriks commented Sep 19, 2023

alromeros commented Sep 19, 2023

alromeros commented Sep 20, 2023

mhenriks commented Sep 20, 2023

kubevirt-bot commented Sep 20, 2023

kubevirt-bot commented Sep 21, 2023

kubevirt-commenter-bot commented Sep 21, 2023

Allow specifying download format in vmexport #10400

Allow specifying download format in vmexport #10400

Conversation

alromeros commented Sep 7, 2023 • edited

kubevirt-bot commented Sep 7, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alromeros commented Sep 12, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alromeros commented Sep 18, 2023

alromeros commented Sep 18, 2023

mhenriks commented Sep 19, 2023

alromeros commented Sep 19, 2023

alromeros commented Sep 20, 2023

mhenriks commented Sep 20, 2023

kubevirt-bot commented Sep 20, 2023

kubevirt-bot commented Sep 21, 2023

kubevirt-commenter-bot commented Sep 21, 2023

alromeros commented Sep 7, 2023 •

edited