Skip to content

[feat] add mirror dryrun#322

Merged
ldmonster merged 6 commits intomainfrom
feat/add-dryrun
Apr 8, 2026
Merged

[feat] add mirror dryrun#322
ldmonster merged 6 commits intomainfrom
feat/add-dryrun

Conversation

@ldmonster
Copy link
Copy Markdown
Collaborator

@ldmonster ldmonster commented Apr 7, 2026

Summary

Adds a --dry-run flag to d8 mirror pull that resolves versions, extracts image digests, and prints the complete list of images that would be downloaded - without pulling blobs or writing bundle files.

Motivation

  • Operator validation. Verify --source, --license, --deckhouse-tag, --modules and other flags produce the expected image list before committing to a 30-60+ minute download.
  • CI/CD pipelines. Cheap smoke tests that confirm registry access and version resolution without transferring data.
  • E2E tests. Fast verification of flag combinations, module filters, and version resolution.

Usage

d8 mirror pull --dry-run /tmp/bundle \
  --source registry.deckhouse.io/deckhouse/fe \
  --license <token> \
  --deckhouse-tag v1.69.0 \
  --no-modules \
  --no-security-db

Exit code 0 on success, non-zero on errors. Bundle directory remains empty.

How it works

Platform (optimized with layer streaming)

Instead of pulling the full installer image (~554 MB) to a local OCI layout, dry-run streams images_digests.json directly from the remote registry using ExtractFileFromImage - a layer-by-layer extraction that downloads only the layer containing the target file (~0.1 MB). No disk writes, no OCI layout creation.

Dry-run logic lives in a separate file platform_dryrun.go with two methods:

  • pullDeckhousePlatformDryRun - orchestrates version resolution and prints grouped image list
  • extractImageDigestsFromRemote - streams JSON metadata from remote installer image

Other services

Service Behavior in dry-run
Modules Module discovery and version resolution via registry API. Skips blob pull and OCI post-processing.
Security Resolves security DB image refs. Prints and returns before pull.
Installer Resolves installer image refs. Prints and returns before pull.

What is skipped in dry-run

  • All blob downloads (platform, modules, security, installer)
  • OCI layout creation for platform
  • Bundle packing (tar files)
  • GOST digest computation
  • VEX attestation scanning (~319 unnecessary network calls)
  • Release channel pull (versions already resolved via registry API)
  • Standalone installer pull (not needed for image list)

Output

Grouped by category with counts and sorted refs:

[dry-run] Platform images that would be pulled:
  Deckhouse components: 319 images
    registry.deckhouse.io/deckhouse/fe@sha256:01c663f2...
    ...
  Release channels: 6
    registry.deckhouse.io/deckhouse/fe/release-channel:alpha
    ...
  Installer: 1
    registry.deckhouse.io/deckhouse/install:v1.69.0
  Standalone installer: 1
    registry.deckhouse.io/deckhouse/install-standalone:v1.69.0
  Total: 327 platform images
[dry-run] Done. No images were downloaded.

Test execution

# Unit tests (stub registry, ~1s)
go test -v -run "DryRun" ./internal/mirror/...

# Integration test with real registry
D8_TEST_REGISTRY=registry.deckhouse.io/deckhouse/fe \
D8_TEST_LICENSE_TOKEN=<token> \
go test -v -run TestDryRunRealRegistry ./internal/mirror/cmd/pull/

Signed-off-by: Pavel Okhlopkov <pavel.okhlopkov@flant.com>
@ldmonster ldmonster self-assigned this Apr 7, 2026
- Stream images_digests.json directly from remote registry using ExtractFileFromImage (layer-by-layer), avoiding full installer image download to disk
- Skip OCI layout creation, release channel pulls, and standalone installer pulls in dry-run mode - none are needed for image list resolution
- Separate dry-run logic into dedicated path (pullDeckhousePlatformDryRun) keeping the normal pull path free of conditional checks
- Remove implicit VEX scanning from dry-run - (~319 unnecessary network calls)

Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
- Dry-run methods (pullDeckhousePlatformDryRun, extractImageDigestsFromRemote) live in platform_dryrun.go for clear separation from the normal pull path
- Grouped dry-run output with per-category headers and sorted image refs

Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
Glitchy-Sheep
Glitchy-Sheep previously approved these changes Apr 8, 2026
Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
@Glitchy-Sheep Glitchy-Sheep added the enhancement New feature or request label Apr 8, 2026
@ldmonster ldmonster merged commit 931cec1 into main Apr 8, 2026
5 checks passed
@ldmonster ldmonster deleted the feat/add-dryrun branch April 8, 2026 13:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants