Skip to content

flowey: split openvmm-deps downloads for v0.3.0 artifact layout#3483

Merged
benhillis merged 1 commit into
microsoft:mainfrom
benhillis:user/benhill/openvmm-deps-0.3.0-split
May 18, 2026
Merged

flowey: split openvmm-deps downloads for v0.3.0 artifact layout#3483
benhillis merged 1 commit into
microsoft:mainfrom
benhillis:user/benhill/openvmm-deps-0.3.0-split

Conversation

@benhillis
Copy link
Copy Markdown
Member

@benhillis benhillis commented May 13, 2026

Adapts OSS flowey to consume the microsoft/openvmm-deps 0.3.0 release layout.

Companion: microsoft/openvmm-deps#62 (merged, v0.3.0-29 released).

openvmm-deps 0.3.0 artifacts

Each release ships three .tar.gz artifacts per arch:

Artifact Contents
openvmm-deps.<arch>.<ver>.tar.gz sdk + dbgrd + shell + petritools
openvmm-test-initrd.<arch>.<ver>.tar.gz shared test guest-userland initrd
openvmm-test-linux-6.1.<arch>.<ver>.tar.gz 6.1 LTS test kernel: vmlinux/Image, bzImage (x86_64), final config

The Linux kernel + initrd here are a test payload only (Vtl0 Example recipe, Linux-direct VMM tests). Production OpenHCL kernels flow through resolve_openhcl_kernel_package.

Nodes

  • resolve_openvmm_deps — fetches openvmm-deps.<arch>.<ver>.tar.gz and exposes individual files via OpenvmmDepFile (OpenhclCpioDbgrd / OpenhclCpioShell / OpenhclSysroot / PetritoolsErofs).
  • resolve_openvmm_test_initrd — fetches openvmm-test-initrd.<arch>.<ver>.tar.gz and returns the path to the initrd file. Parameterized by arch.
  • resolve_openvmm_test_linux_kernel — fetches openvmm-test-linux-<kver>.<arch>.<ver>.tar.gz and returns the path to a selected file via OpenvmmTestKernelFile: Kernel (the primary vmlinux on x86_64 / Image on aarch64) or BzImage (x86_64 only). Parameterized by (file, arch, kver). Archive download + extraction is deduped on (arch, kver) so requesting both Kernel and BzImage from the same archive only extracts once. The LinuxTestKernelVersion enum currently contains just Linux6_1; DEFAULT_LINUX_TEST_KERNEL_VERSION is exported for callers that don't care which kernel they get. Future kernel lines are added by extending the enum.

All three nodes accept either a version (download from the public GH release with needs_auth: false) or local_paths (use a pre-extracted directory on disk).

Other changes

  • flowey_lib_common::_util::extract::extract_tar_gz_if_new — cached .tar.gz extract helper ({root_dir}/.flowey_info/{filename} tracks file_version, skip re-extract on match), mirroring the existing extract_zip_if_new / extract_tar_bz2_if_new.
  • cfg_versions::OPENVMM_DEPS = "0.3.0-29" drives the version config of all three resolve nodes.
  • init_vmm_tests_env, init_openvmm_magicpath_openvmm_deps, and build_openhcl_igvm_from_recipe request the kernel + initrd from the new nodes; the magicpath and vmm_tests_env consumers also request BzImage on x86_64.
  • Pipeline YAMLs regenerated.

Copilot AI review requested due to automatic review settings May 13, 2026 23:22
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates Flowey infrastructure to consume the upcoming split microsoft/openvmm-deps artifact layout, separating the main dependency archive from per-kernel Linux test payload archives.

Changes:

  • Adds Linux test kernel version selection to download_openvmm_deps.
  • Switches OpenVMM deps extraction/downloads from .tar.bz2 to split .tar.gz artifacts.
  • Regenerates CI workflow YAMLs for the new Flowey graph.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.

Show a summary per file
File Description
flowey/flowey_lib_hvlite/src/init_vmm_tests_env.rs Passes the default Linux test kernel version when requesting VMM test kernel/initrd.
flowey/flowey_lib_hvlite/src/init_openvmm_magicpath_linux_test_kernel.rs Passes the default Linux test kernel version for magic-path setup.
flowey/flowey_lib_hvlite/src/download_openvmm_deps.rs Implements split main/per-kernel artifact download and extraction logic.
flowey/flowey_lib_hvlite/src/build_openhcl_igvm_from_recipe.rs Updates example VTL0 kernel requests to include a kernel version.
flowey/flowey_lib_hvlite/src/_jobs/cfg_versions.rs Bumps OPENVMM_DEPS to the placeholder v0.3.0 artifact version.
flowey/flowey_lib_common/src/_util/extract.rs Adds a reusable .tar.gz extraction helper.
.github/workflows/openvmm-pr.yaml Regenerated PR workflow for the updated Flowey graph.
.github/workflows/openvmm-ci.yaml Regenerated CI workflow for the updated Flowey graph.

Comment thread flowey/flowey_lib_common/src/_util/extract.rs Outdated
Comment thread flowey/flowey_lib_hvlite/src/download_openvmm_deps.rs Outdated
Copilot AI review requested due to automatic review settings May 14, 2026 16:39
Comment thread flowey/flowey_lib_common/src/_util/extract.rs
Comment thread flowey/flowey_lib_hvlite/src/download_openvmm_test_linux_kernel.rs Outdated
@benhillis benhillis force-pushed the user/benhill/openvmm-deps-0.3.0-split branch 2 times, most recently from 4647acf to e15759e Compare May 15, 2026 01:10
Comment thread flowey/flowey_lib_hvlite/src/resolve_openvmm_deps.rs Outdated
benhillis pushed a commit to benhillis/openvmm that referenced this pull request May 15, 2026
Post-split, no `OpenvmmDepFile` variant has an arch-dependent filename
(the only ones that did — kernel and initrd — moved out into their own
nodes). Drop the now-dead `_arch` parameter and update both call sites.

Addresses review feedback on microsoft#3483.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@benhillis benhillis marked this pull request as ready for review May 15, 2026 21:42
@benhillis benhillis requested review from a team as code owners May 15, 2026 21:42
@benhillis benhillis force-pushed the user/benhill/openvmm-deps-0.3.0-split branch 2 times, most recently from bc7d967 to 77bc743 Compare May 15, 2026 22:29
@github-actions
Copy link
Copy Markdown

openvmm-deps 0.3.0 ships three .tar.gz artifacts per arch:
- openvmm-deps.<arch>.<ver>.tar.gz: sdk + dbgrd + shell + petritools
- openvmm-test-initrd.<arch>.<ver>.tar.gz: shared test guest-userland initrd
- openvmm-test-linux-6.1.<arch>.<ver>.tar.gz: 6.1 LTS test kernel image
  (vmlinux/Image), bzImage (x86_64), and final config

Add matching flowey resolve nodes:

- `resolve_openvmm_deps` (slimmed): exposes individual files from the main
  openvmm-deps tarball via `OpenvmmDepFile` (Dbgrd, Shell, Sysroot,
  PetritoolsErofs).
- `resolve_openvmm_test_initrd`: returns the path to the shared
  `initrd` file. Parameterized by arch.
- `resolve_openvmm_test_linux_kernel`: returns paths to files in the
  per-(arch, kver) test kernel archive via `OpenvmmTestKernelFile`
  (`Kernel` for the primary vmlinux/Image image, `BzImage` for the
  x86_64-only bzImage). Parameterized by `(file, arch, kver)`. The
  `LinuxTestKernelVersion` enum currently contains just `Linux6_1`.

All three nodes accept either a `version` (download from the public GH
release with `needs_auth: false`) or `local_paths` (use a pre-extracted
directory on disk). The kernel resolver dedupes archive download +
extraction on `(arch, kver)`, so requesting both `Kernel` and `BzImage`
from the same archive only extracts once.

Add `flowey_lib_common::_util::extract::extract_tar_gz_if_new`, mirroring
the existing `extract_zip_if_new` / `extract_tar_bz2_if_new` helpers.

Wire `init_vmm_tests_env`, `init_openvmm_magicpath_openvmm_deps`, and
`build_openhcl_igvm_from_recipe` to request the kernel + initrd from the
new nodes. Pin `OPENVMM_DEPS = "0.3.0-29"`.

Pipeline YAMLs regenerated.

Companion: microsoft/openvmm-deps#62 (merged as v0.3.0-29).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@benhillis benhillis force-pushed the user/benhill/openvmm-deps-0.3.0-split branch from 77bc743 to ac1b7a0 Compare May 16, 2026 18:40
@github-actions
Copy link
Copy Markdown

@benhillis benhillis merged commit 1c01d82 into microsoft:main May 18, 2026
93 of 97 checks passed
@benhillis benhillis deleted the user/benhill/openvmm-deps-0.3.0-split branch May 18, 2026 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants