Skip to content

MichaelRyom/DSvRepoImport

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 

Repository files navigation

DSvRepoImport

PowerShell module for importing a DSvClient-downloaded VMware repository into vCenter and the VCSA.

The module repackages the on-disk DSv repo (which is structured as an online VMware depot) into the various formats vCenter, vLCM and the VCSA actually accept, and imports them via the appropriate APIs. It is designed for air-gapped environments where the vCenter cannot reach Broadcom's online sources directly.

What it imports

DSv repo item Function Target
all.json Import-VsanHclDatabase vSAN HCL database
results.json Import-VsanReleaseCatalog vSAN release catalog
vvs_data.gz Import-VcgDatabase VCG hardware compatibility database
main/, addon-main/, iovp-main/, vmtools-main/ Import-VlcmOfflineBundle vLCM offline depots (per build)
valm/<version>/ New-VcsaPatchIso Builds an installable VCSA patch ISO

The vLCM importer parses each metadata.zip into per-build subsets and produces self-contained offline bundles that pass vLCM's strict DownloadOfflinePatchTask validation. Three depot shapes are supported:

  • profile-based (ESXi base) — one bundle per ESXi build (e.g. 24022510)
  • addon-based (DEL/HPE/CIS/...) — one bundle per addon JSON
  • flat catalog (IOVP, vmtools) — one bundle per metadata.zip, automatically size-chunked to stay under vCenter's 2 GB offline-depot limit

Requirements

  • Windows PowerShell 5.1
  • VMware PowerCLIVMware.VimAutomation.Core, VMware.VimAutomation.Cis.Core, VMware.VimAutomation.Storage
  • WinSCP PowerShell module — Install-Module WinSCP
  • Posh-SSHInstall-Module Posh-SSH
  • Network access to the vCenter API and SSH access to the VCSA root account

Installation

Place the module folder in any directory listed in $env:PSModulePath:

# Per-user
$dst = "$env:USERPROFILE\Documents\WindowsPowerShell\Modules\DSvRepoImport"
Copy-Item -Recurse -Path .\DSvRepoImport -Destination $dst

# Or system-wide (requires admin)
Copy-Item -Recurse -Path .\DSvRepoImport -Destination "$env:ProgramFiles\WindowsPowerShell\Modules\"

After that:

Import-Module DSvRepoImport

Functions

Import-VsanHclDatabase

Updates the vSAN Hardware Compatibility List from all.json. Wraps PowerCLI's Update-VsanHclDatabase cmdlet. Requires an active Connect-VIServer session.

Import-VsanHclDatabase -FilePath "D:\VMware-repo\all.json"

Import-VsanReleaseCatalog

Uploads the vSAN release catalog (results.json) via the VsanVumSystem vSAN managed object. Requires an active Connect-VIServer session.

Import-VsanReleaseCatalog -FilePath "D:\VMware-repo\results.json"

Import-VcgDatabase

Imports the VCG (VMware Compatibility Guide) database from vvs_data.gz.

The function gunzips the file locally with System.IO.Compression.GZipStream, uploads the JSON to /tmp/vvs_data on the VCSA via SFTP (using WinSCP with the SftpServer = shell /usr/libexec/sftp-server raw setting required by VCSA), then runs hcl_datastore.py update-offline over SSH.

Does NOT need a vCenter API connection — it talks directly to the VCSA OS.

$cred = Get-Credential -UserName root
Import-VcgDatabase -FilePath "D:\VMware-repo\vvs_data.gz" `
                   -VCSAHost "vcsa.domain.local" `
                   -Credential $cred

Import-VlcmOfflineBundle

The main workhorse. Repackages every depot in the DSv repo into vLCM-importable self-contained offline bundles, then uploads and imports each via the /api/esx/settings/depots/offline REST endpoint.

Requires an active Connect-CisServer session and SSH access to the VCSA root account (used for uploading bundles via SFTP and running a temporary HTTP server on the appliance to feed the offline-depot PULL API).

$vcsaCred = Get-Credential -UserName root
Connect-CisServer -Server vcsa.domain.local

# Import every ESXi 8.0.3 build
Import-VlcmOfflineBundle -Path "D:\VMware-repo" `
                         -VCenterServer "vcsa.domain.local" `
                         -Credential $vcsaCred `
                         -VersionFilter '*8.0.3*'

# Import a single specific ESXi build
Import-VlcmOfflineBundle -Path "D:\VMware-repo" `
                         -VCenterServer "vcsa.domain.local" `
                         -Credential $vcsaCred `
                         -VersionFilter '*8.0.3*' `
                         -BuildFilter '25205845'

Parameters

Name Description
Path Path to the DSv VMware-repo root.
VCenterServer vCenter FQDN/IP. Used both for the REST API and for SSH/SFTP into the appliance.
Credential VCSA OS root credentials (separate from the vCenter API user).
VersionFilter Wildcard pattern(s) matched against metadata.zip filenames. Default '*' (all).
BuildFilter Wildcard pattern(s) matched against build numbers within a metadata.zip.
RemotePath Temp dir on VCSA to stage uploaded bundles. Default /tmp/vlcm-import.
HttpPort Port for the temporary Python HTTP server on the VCSA. Default 8888.
LocalTempPath Local temp dir for bundle ZIPs. Default $env:TEMP.
MaxBundleSizeMB Hard size cap per bundle. vCenter's offline-depot download manager uses a signed 32-bit byte counter (~2 GB max), so default is 1800 MB.

How a bundle is constructed

For each metadata.zip in the matched depot folders, the function:

  1. Indexes the metadata.zip in memory: groups bulletins, profiles/addons, vibs/*.xml descriptors, baseimages, configSchemas and vibExports by build.
  2. For each build (or chunk, for flat catalogs that exceed the size limit): a. Generates a per-build metadata.zip with only the entries (vmware.xml bulletin blocks, vibs/*.xml descriptors, bulletin XMLs, profile, addon json, baseimage, configSchemas, vibExports) that the build references. b. Wraps that metadata.zip together with index.xml, vendor-index.xml, and the actual vib20/{component}/*.vib files into a single bundle ZIP at NoCompression (the VIBs are already compressed). c. Uploads the bundle to the VCSA via WinSCP/SFTP. d. Calls the offline-depot REST API with source_type=PULL and location=http://localhost:<HttpPort>/<bundle> — vCenter pulls the bundle from a Python HTTP server we run on the appliance. e. Polls the resulting task with Wait-CisTask until SUCCEEDED/FAILED. f. Removes the local and remote bundle to keep disk usage bounded.

VIBs that have already been uploaded (across builds in the same run) are tracked in a HashSet so the same VIB is only uploaded once.

Get-VlcmOfflineBundle / Remove-VlcmOfflineBundle

List and delete imported offline depots via the REST API. Pipeline-friendly:

# Show all currently imported depots
Get-VlcmOfflineBundle -VCenterServer vcsa.domain.local

# Remove all DSv-imported depots
Get-VlcmOfflineBundle -VCenterServer vcsa.domain.local |
    Where-Object { $_.Description -like 'DSv:*' } |
    Remove-VlcmOfflineBundle

New-VcsaPatchIso

Builds an installable VCSA patch ISO from an unpacked valm/<version> folder. Not wired into any of the import flows because VCSA patching is destructive and should be initiated explicitly. Uses Windows IMAPI2FS COM (built into Windows, no extra tools or admin rights) and stages files via NTFS hardlinks where possible to avoid copying multi-GB RPM payloads.

New-VcsaPatchIso -SourcePath "D:\VMware-repo\valm\8.0.3.00800" `
                 -OutputPath "D:\VMware-VCSA-Patch-8.0.3.00800.iso"

Once built, mount the ISO to the VCSA VM's CD-ROM and from the appliance shell:

software-packages stage --iso
software-packages install --iso

Use this when you have hypervisor/VM access to attach a virtual CD-ROM. If you only have OS access to the appliance, use Install-VcsaPatch instead.

Install-VcsaPatch

Stages or installs a VCSA appliance patch via SSH/SFTP, without needing hypervisor access to attach a virtual CD-ROM. Two mutually exclusive modes — both always build, upload and mount the patch ISO themselves; neither trusts prior session state:

Mode Behavior
-Stage Build ISO → upload → loopback-mount → software-packages stage --iso → cleanup. Validates the patch without installing.
-Install Build ISO → upload → loopback-mount → software-packages stage --isosoftware-packages install --staged → cleanup.

-Install always re-runs the stage call before installing. On already-staged data this is a cheap checksum-verification pass — the appliance's _copyPackageToStageDir short-circuits whenever the destination file exists with a matching SHA-256 — and it's required because the appliance's install-time discovery code reads patch-metadata-scripts.zip from the mounted ISO (that file is never copied into the stage directory by software-packages stage). With our losetup-based workflow the ISO is only mounted during the lifetime of one Install-VcsaPatch call, so stage and install have to run in the same invocation.

Why an ISO instead of an unpacked folder?

The VCSA's software-packages command only supports --iso and --url — there is no --directory option. The appliance Python code (in /usr/lib/applmgmt/update/py/.../update_constants.py) has a hard-coded MOUNTPOINT = "/mnt/iso-contents" that it expects the ISO contents to be mounted at.

This function builds an ISO from valm/<version>/ using New-VcsaPatchIso, SFTP-uploads it to /storage/core/, then loopback-mounts it at the same /mnt/iso-contents path the appliance expects — no virtual CD-ROM device or hypervisor access needed.

$cred = Get-Credential -UserName root

# Build + upload + mount + stage (good for validating a patch without installing)
Install-VcsaPatch -VCSAHost vcsa.domain.local -Credential $cred `
                  -SourcePath D:\VMware-repo\valm\8.0.3.00800 -Stage

# Build + upload + mount + stage + install in one go
Install-VcsaPatch -VCSAHost vcsa.domain.local -Credential $cred `
                  -SourcePath D:\VMware-repo\valm\8.0.3.00800 -Install

# Use a pre-built ISO instead of building it from a folder
Install-VcsaPatch -VCSAHost vcsa.domain.local -Credential $cred `
                  -IsoPath D:\VMware-VCSA-Patch-8.0.3.00800.iso -Install

Not called from Import-DSvRepo.ps1 — VCSA patching is destructive and must be initiated explicitly.

Parameters

Name Description
VCSAHost FQDN/IP of the VCSA appliance.
Credential VCSA OS root credentials.
SourcePath Local valm/<version>/ folder. The function builds the ISO automatically. Mutually exclusive with IsoPath.
IsoPath Local path to a pre-built VCSA patch ISO. Skips the build step.
RemotePath Remote directory for the ISO upload. Default /storage/core (much larger than /tmp on a stock VCSA).
MountPoint Path on the VCSA where the ISO is mounted. Default /mnt/iso-contents (matches update_constants.MOUNTPOINT).
Stage Switch — build + upload + mount + stage + cleanup. Does not install.
Install Switch — build + upload + mount + stage + install + cleanup.
StageTimeoutSeconds SSH timeout for the stage call. Default 1800 (30 minutes).
InstallTimeoutSeconds SSH timeout for the install call. Default 7200 (2 hours).

Caveats

  • software-packages is an appliancesh command, not a bash binary. The function invokes the underlying Python implementation directly via shell python3 /usr/lib/applmgmt/support/scripts/software-packages.py ... rather than going through appliancesh -c, because the appliancesh wrapper hangs Posh-SSH for long-running stage calls. Invoking software-packages from a plain bash shell gives command not found — bash sees no such binary.
  • The SSH session may drop during install. vCenter restarts services as part of the patch and the underlying SSH connection often gets reset. The function catches that, warns about it, and tells you to verify completion via VAMI (https://<VCSAHost>:5480).
  • Disk usage spikes during stage. The uploaded ISO (~8 GB for a modern VCSA 8.0U3 patch) plus software-packages's own staged copy can hit ~16 GB on /storage/core. Cleanup happens at the end of the operation (after install in -Install mode, after stage in -Stage mode).
  • A failed stage leaves the ISO + mount in place. If staging fails, the function tells you the exact cleanup commands so you can investigate without re-uploading. Successful stage (or successful install) triggers automatic cleanup of mount + ISO + mountpoint directory.
  • Run interactively. Don't put this in unattended automation - if anything unexpected happens to vCenter mid-install, you want a human to react.

Notes and constraints

  • All four DSv depot types are supported (main/, addon-main/, iovp-main/, vmtools-main/). The bundles produced for each are individually validated by vLCM's Image Builder and accepted by the legacy DownloadOfflinePatchTask.
  • vLCM offline-depot API has a 2 GB bundle ceiling because the C++ download manager uses a signed 32-bit byte counter. The module enforces this with MaxBundleSizeMB = 1800. Profile/addon builds that don't fit are skipped (their cross-references can't be split safely); flat-catalog builds are bin-packed into chunks.
  • The VCSA's OpenSSH does not allow TCP forwarding (AllowTcpForwarding no), which is why the offline-depot import uses an SFTP upload + a Python HTTP server running on the appliance, instead of an SSH tunnel. The HTTP server only runs for the duration of the import.
  • vLCM internally deduplicates VIBs by content hash, so re-importing the same VIB does not waste disk space on the VCSA. The local dedup in this module only saves upload time and bandwidth.

Companion script

A wrapper script that calls the import functions in order is available in Import-DSvRepo.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors