Skip to content

feat(capture download): Add ability to download capture files based on capture name #1564

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 48 commits into from
Jun 7, 2025

Conversation

kamilprz
Copy link
Contributor

@kamilprz kamilprz commented Apr 28, 2025

Description

This PR introduces the ability to download captures via the Retina CLI.

It extends the retina capture download cmd to be able to download captures based on the capture name / blobURL.

This enables the manual download of captures which occur after this PR.

  • This is because in order to make retrospective downloads possible, the download pods are now annotated with some metadata. Captures prior to this PR won't have this metadata, and it won't be possible to download them.

Change list:

  • Update retina capture list to be more readable.
  • Change BlobURL download to be a flag --blobUrl rather than an ENV variable.
    • This makes it more consistent with the rest of the CLI.
  • Add annotations to capture pods to store metadata about the capture.
  • Download capture file from host node by passing name of capture which created the file.
    • By default, creates a folder in the current directory based on the name of the capture, which will contain the tarball.
    • Can set the output directory. (-o)
      • Example: retina capture download --name download-me -o /tmp/retina/
  • Add an E2E test for retina capture create

Related Issue

N/A

Checklist

  • I have read the contributing documentation.
  • I signed and signed-off the commits (git commit -S -s ...). See this documentation on signing commits.
  • I have correctly attributed the author(s) of the code.
  • I have tested the changes locally.
  • I have followed the project's style guidelines.
  • I have updated the documentation, if necessary.
  • I have added tests, if applicable.

Screenshots

Examples and flags documentation

{50A67E84-9D0C-49ED-997B-1C594C2C646E}

k retina capture list

After
{447597F0-3512-4F12-96C2-09DB0D66A210}

Before
{83B47868-B31C-4A7D-8FDF-2F3BEFDAB1F9}

Downloading a capture based on the capture name

{3CC49A7E-9A7D-45C4-B7E0-C9DCB092B1C0}

{C3B0F8E1-5037-4EDC-9E07-A6530A621B05}

E2E Retina Capture Create

{90CB2A2D-409D-4F8F-A29D-1461862299EE}

@kamilprz kamilprz requested a review from a team as a code owner April 28, 2025 11:21
Copy link
Contributor

@alexcastilio alexcastilio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I miss some tests to make sure everything is working as expected

Copilot

This comment was marked as outdated.

kamilprz and others added 13 commits June 3, 2025 00:10
Signed-off-by: Kamil <kamil.prz@gmail.com>
Signed-off-by: Kamil <kamil.prz@gmail.com>
Signed-off-by: Kamil <kamil.prz@gmail.com>
Signed-off-by: Kamil <kamil.prz@gmail.com>
Signed-off-by: Kamil <kamil.prz@gmail.com>
Signed-off-by: Alex Castilio dos Santos <alexsantos@microsoft.com>
…eJobs; add retrier to verifyJobs

Signed-off-by: Alex Castilio dos Santos <alexsantos@microsoft.com>
Signed-off-by: Alex Castilio dos Santos <alexsantos@microsoft.com>
Signed-off-by: Alex Castilio dos Santos <alexsantos@microsoft.com>
Signed-off-by: Alex Castilio dos Santos <alexsantos@microsoft.com>
Signed-off-by: Alex Castilio dos Santos <alexsantos@microsoft.com>
Signed-off-by: Alex Castilio dos Santos <alexsantos@microsoft.com>
Signed-off-by: Alex Castilio dos Santos <alexsantos@microsoft.com>
@vakalapa vakalapa requested a review from Copilot June 5, 2025 15:14
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for downloading captures by name or blob URL, enhances capture metadata via pod annotations, and refactors timestamp handling to use metav1.Time.

  • Implement kubectl retina capture download command for cluster and blob-based downloads
  • Annotate capture pods with filename, timestamp, and host path
  • Refactor timestamp APIs and update providers, tests, and CRD translation

Reviewed Changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
test/e2e/scenarios/capture/scenarios.go Add E2E scenario for capture create
test/e2e/scenarios/capture/install-retina-plugin.go Install kubectl-retina plugin for E2E tests
test/e2e/retina_e2e_test.go Invoke capture validation in the end-to-end test suite
test/e2e/jobs/jobs.go Register ValidateCapture job
pkg/capture/utils/annotations.go Add helper to extract pod annotations from Capture CR
pkg/capture/provider/network_capture_win.go Update StartTimestamp to use metav1.Time
pkg/capture/provider/network_capture_unix.go Update StartTimestamp to use metav1.Time
pkg/capture/provider/network_capture_test.go Adjust tests to new timestamp type
pkg/capture/file/timestamp.go Replace custom Timestamp with metav1.Time helpers
pkg/capture/file/capture_filename.go Use TimeToString for timestamp formatting
pkg/capture/crd_to_job_test.go Update test to call StringToTime
pkg/capture/crd_to_job.go Add pod annotations and wiring for capture filenames
pkg/capture/constants/job_env.go Add new environment keys for namespace, pod, container
pkg/capture/constants/annotations.go Define annotation keys for capture metadata
pkg/capture/capture_manager_test.go Update test to new timestamp type
pkg/capture/capture_manager.go Refactor captureStartTimestamp to use StringToTime
docs/04-Captures/02-cli.md Document capture download command
cli/cmd/capture/table_util.go Change capture list output to one job per line
cli/cmd/capture/download.go Implement download logic and flags
cli/cmd/capture/create.go Annotate new Capture CRD with StartTime status
Comments suppressed due to low confidence (3)

cli/cmd/capture/download.go:75

  • Add explicit validation to ensure at least one of --name or --blob-url is provided, and return an error or usage message if both are empty.
RunE: func(*cobra.Command, []string) error {

cli/cmd/capture/download.go:384

  • Reading the entire blob into memory can lead to high memory usage for large captures. Consider streaming directly to disk or using an io.Copy to write to a file.
blobData, err := io.ReadAll(readCloser)

pkg/capture/crd_to_job.go:394

  • Remove these debug print statements from production code or guard them behind a verbose/log level flag to avoid cluttering output.
fmt.Println("#########################")

@matmerr
Copy link
Member

matmerr commented Jun 5, 2025

in the unhappy path, if a capture isn't started, or has problem, download fails silently, let's give some feedback

$ ./bin/kubectl-retina capture list
NAMESPACE   CAPTURE NAME     JOB                    COMPLETIONS   AGE
default     retina-capture   retina-capture-8dl8v   0/1   

$ ./bin/kubectl-retina capture download retina-capture

this also seems to be the case just calling

$ ./bin/kubectl-retina capture download 

Signed-off-by: Alex Castilio dos Santos <alexsantos@microsoft.com>
Signed-off-by: Alex Castilio dos Santos <alexsantos@microsoft.com>
Signed-off-by: Alex Castilio dos Santos <alexsantos@microsoft.com>
@alexcastilio
Copy link
Contributor

in the unhappy path, if a capture isn't started, or has problem, download fails silently, let's give some feedback

$ ./bin/kubectl-retina capture list
NAMESPACE   CAPTURE NAME     JOB                    COMPLETIONS   AGE
default     retina-capture   retina-capture-8dl8v   0/1   

$ ./bin/kubectl-retina capture download retina-capture

this also seems to be the case just calling

$ ./bin/kubectl-retina capture download 

Added validation of --name or --blob-url as one of them is required:

$ ./artifacts/kubectl-retina capture download
Error: either --name or --blob-url must be specified
Usage:
  kubectl-retina capture download [flags]

Examples:
  # List Retina capture jobs
  kubectl retina capture list

  # Download the capture file(s) created using the capture name
  kubectl retina capture download --name <capture-name>

  # Download the capture file(s) created using the capture name and define output location
  kubectl retina capture download --name <capture-name> -o <output-location>

@ibezrukavyi ibezrukavyi added this pull request to the merge queue Jun 6, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jun 6, 2025
@nddq nddq added this pull request to the merge queue Jun 7, 2025
Merged via the queue into microsoft:main with commit 8e41f1b Jun 7, 2025
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants