VCF Installer deployment failed

[saingithub]

Describe the bug

Attempting to deploy VCF 9 on an ESXi host, but encountering an error during the VCF Installer deployment process.

08-08-2025 05:45:52 VCFInstaller[3952]: [INFO] Validating user input for VCF Installer
08-08-2025 05:45:52 VCFInstaller[3952]: [INFO] User input validation successful
08-08-2025 05:45:52 VCFInstaller[3952]: [INFO] Deploying VCF Installer
Opening OVA source: /holodeck-runtime/bin/9.0/VCF-SDDC-Manager-Appliance-9.0.0.0.24703748.ova
The manifest validates
Error: Internal error: Failed to connect to remote host [0.0.7.232] due to [No version for VMODL calls to <<<cs p:000055cffea73fd0, TCP:0.0.7.232:443> >, /sdk>]
Completed with errors
08-08-2025 05:47:52 VCFInstaller[3952]: [ERROR] VCF Installer deployment failed
08-08-2025 05:47:52 VCFInstaller[3952]: [INFO] Failed to deploy OVF template. Retry #1

Reproduction steps

...

Expected behavior

Successfull Deployment

Additional context

No response

[dhruv-tyagi-broadcom]

Are you using a trunk port group for this deployment? Are the required VLANs allowed on the underlying physical network?

[saingithub]

Deployment is being carried out on a standalone host, where the standard switch has no uplink configured.

[dhruv-tyagi-broadcom]

Can you do an nslookup for vcfinstaller-a.site-a.vcf.lab from holorouter and check if it is resolving?

Are you able to ping the target host from holorouter?

Are you able to ping the nested ESX hosts from holorouter?

[saingithub]

Network connectivity has been validated. nslookup for the VCF installer, along with ping tests to the host and nested ESXi, were all successful.

root@holorouter [ ~/.local/share/powershell/Modules/HoloDeck/Modules ]# nslookup vcfinstaller-a.site-a.vcf.lab
Server: 10.1.1.1
Address: 10.1.1.1#53

Name: vcfinstaller-a.site-a.vcf.lab
Address: 10.1.10.250

root@holorouter [ ~/.local/share/powershell/Modules/HoloDeck/Modules ]# ping esx-01a.site-a.vcf.lab
PING esx-01a.site-a.vcf.lab (10.1.1.101) 56(84) bytes of data.
64 bytes from esx-01a.site-a.vcf.lab (10.1.1.101): icmp_seq=1 ttl=64 time=0.210 ms
64 bytes from esx-01a.site-a.vcf.lab (10.1.1.101): icmp_seq=2 ttl=64 time=0.229 ms
64 bytes from esx-01a.site-a.vcf.lab (10.1.1.101): icmp_seq=3 ttl=64 time=0.280 ms
^C
--- esx-01a.site-a.vcf.lab ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2042ms
rtt min/avg/max/mdev = 0.210/0.239/0.280/0.029 ms
root@holorouter [ ~/.local/share/powershell/Modules/HoloDeck/Modules ]#

[dhruv-tyagi-broadcom]

Great, now you can try running the same New-HoloDeckInstance command again while the config is loaded in the powershell session and see if the OVA deployment goes through. If this does not work, we can try running the ovftool command to deploy the OVA manually and see if its an issue with the ovftool

[saingithub]

I have attempted the deployment, but the issue persists

[dhruv-tyagi-broadcom]

Okay. Let's try deploying the OVA manually. Try the following code block assuming your config is loaded and see what output you get:

$site = "a"
$siteCode = "Site-$site"
$target_host = $config.Target
$vcf_installer_config = $config.'holodeck-sddc'.$siteCode.'vcf-installer'
$instanceID = $config.InstanceID 
$ova_path = $vcf_installer_config.ovaPath
$username = $target_host.username
Add-Type -AssemblyName System.Web
$username_encoded = [System.Web.HttpUtility]::UrlEncode($username)
$password = $target_host.password
$password_encoded = [System.Web.HttpUtility]::UrlEncode($password)
$hostname = $target_host.hostname
$datastore = $target_host.datastore


$vcf_installer_hostname = $vcf_installer_config.hostname
if( $config.InstanceID -ne "Default")
{
    $vcf_installer_vmname = $instanceId +"-"+$vcf_installer_config.hostname
}else{
    $vcf_installer_vmname = $vcf_installer_config.hostname
}

$vcf_installer_network = $vcf_installer_config.network1
$vcf_installer_root_password = $vcf_installer_config.rootPassword
$vcf_installer_local_password = $vcf_installer_config.localUserPassword
$vcf_installer_ntp = $vcf_installer_config.ntp
$vcf_installer_ip0 = $vcf_installer_config.ip0
$vcf_installer_netmask0 = $vcf_installer_config.netmask0
$vcf_installer_gateway = $vcf_installer_config.gateway
$vcf_installer_domain = $vcf_installer_config.domain
$vcf_installer_searchpath = $vcf_installer_config.searchpath
$vcf_installer_DNS = $vcf_installer_config.dns

 ovftool `
                    --noSSLVerify `
                    --name=$vcf_installer_vmname `
                    --X:injectOvfEnv `
                    --datastore=$datastore `
                    --diskMode=Thin `
                    --network=$vcf_installer_network `
                    --acceptAllEulas `
                    --prop:ROOT_PASSWORD=$vcf_installer_root_password `
                    --prop:LOCAL_USER_PASSWORD=$vcf_installer_local_password `
                    --prop:vami.hostname=$vcf_installer_hostname `
                    --prop:guestinfo.ntp=$vcf_installer_ntp `
                    --prop:vami.ip0.SDDC-Manager=$vcf_installer_ip0 `
                    --prop:vami.netmask0.SDDC-Manager=$vcf_installer_netmask0 `
                    --prop:vami.gateway.SDDC-Manager=$vcf_installer_gateway `
                    --prop:vami.domain.SDDC-Manager=$vcf_installer_domain `
                    --prop:vami.searchpath.SDDC-Manager=$vcf_installer_searchpath `
                    --prop:vami.DNS.SDDC-Manager=$vcf_installer_DNS `
                    --powerOn `
                    $ova_path `
                    vi://$username_encoded":"$password_encoded"@"$hostname

[saingithub]

Thank you. The OVF has been successfully deployed. Could you please advise on the next steps?

Opening OVA source: /holodeck-runtime/bin/9.0/VCF-SDDC-Manager-Appliance-9.0.0.0.24703748.ova
The manifest validates
Opening VI target: vi://root@abc-dr-03.rcctvm.com:443/
Deploying to VI: vi://root@abc-dr-03.rcctvm.com:443/
Transfer Completed
Powering on VM: hol-vcfinstaller-a
Task Completed
Completed successfully

[dhruv-tyagi-broadcom]

Great. Do you have @ in your target ESX password? or some special character? That could be the reason for failure.

To move forward, run the following in the same powershell session:

Write-Log -Message "Updating VCF Installer Settings"

                    do{
                        $vcf_installer_vm = Get-VM -Name $vcf_installer_vmname
                        Start-Sleep -Seconds 10
                    } until ($vcf_installer_vm.ExtensionData.Guest.ToolsRunningStatus -eq "guestToolsRunning")

                    if(($vcf_installer_config.depotType -eq "offline") -and ($vcf_installer_config.offlineDepotDetails.port -eq "80")){
                        Invoke-VMScript -ScriptType Bash -GuestUser "root" -GuestPassword $vcf_installer_root_password -VM $vcf_installer_vm -ScriptText "sed -i 's/lcm.service.parallel.download.limit=.*/lcm.service.parallel.download.limit=7/g' /opt/vmware/vcf/lcm/lcm-app/conf/application-prod.properties;echo 'lcm.depot.adapter.httpsEnabled=false' >> /opt/vmware/vcf/lcm/lcm-app/conf/application-prod.properties;systemctl restart lcm.service"                    }
                    else{
                        Invoke-VMScript -ScriptType Bash -GuestUser "root" -GuestPassword $vcf_installer_root_password -VM $vcf_installer_vm -ScriptText "sed -i 's/lcm.service.parallel.download.limit=.*/lcm.service.parallel.download.limit=7/g' /opt/vmware/vcf/lcm/lcm-app/conf/application-prod.properties;systemctl restart lcm.service"
                    }
        

                    Write-Log -Message "VCF Installer settings updated successfully"

Then run:

vi $config.state

Find the json object "Deploy-VcfInstaller" and change the "status": "InProgress" to "status": "Success" and save the file. Then run your New-HoloDeckInstance command again and it should skip VCF Installer deployment and start after that step

[saingithub]

With your help, I have successfully progressed past the previous issue ('@' was there in password). I am now facing an error in the bundle installation section. Could you please advise on how to resolve this?

12-08-2025 16:54:54 SddcMgmtDomain[4038]: [INFO] Bundle download status:
12-08-2025 16:54:54 SddcMgmtDomain[4038]: [INFO] Component VCENTER status: INPROGRESS
12-08-2025 16:54:54 SddcMgmtDomain[4038]: [INFO] Component VRSLCM status: SUCCESS
12-08-2025 16:54:54 SddcMgmtDomain[4038]: [INFO] Component NSX_T_MANAGER status: INPROGRESS
12-08-2025 16:54:54 SddcMgmtDomain[4038]: [INFO] Component VCF_OPS_CLOUD_PROXY status: SUCCESS
12-08-2025 16:54:54 SddcMgmtDomain[4038]: [INFO] Component SDDC_MANAGER status: SUCCESS
12-08-2025 16:54:54 SddcMgmtDomain[4038]: [INFO] Component VROPS status: SUCCESS
12-08-2025 16:54:54 SddcMgmtDomain[4038]: [INFO] Bundle download for bundle @{componentType=VRA; version=9.0.0.0.24701403; bundleId=244dcbc0-d592-540c-893a-95eb07877a23; downloadId=4bf32e6f-e4c7-4ca9-b49d-3e275ec992ee; downloadStatus=FAILED; downloadedSize=0; downloadStartTime=1755016051064; downloadEndTime=1755017676123; downloadScheduledTime=1755016041193; isDownloadCancellable=True; message=ConnectionClosedException: Premature end of Content-Length delimited message body (expected: 22731151360; received: 9921651593); isDownloadable=True} failed. Please log in to VCF Installer (https://10.1.10.250) and resolve any issues causing bundle download failure.
Retry bundle download for bundle @{componentType=VRA; version=9.0.0.0.24701403; bundleId=244dcbc0-d592-540c-893a-95eb07877a23; downloadId=4bf32e6f-e4c7-4ca9-b49d-3e275ec992ee; downloadStatus=FAILED; downloadedSize=0; downloadStartTime=1755016051064; downloadEndTime=1755017676123; downloadScheduledTime=1755016041193; isDownloadCancellable=True; message=ConnectionClosedException: Premature end of Content-Length delimited message body (expected: 22731151360; received: 9921651593); isDownloadable=True}? [y/n]: y
12-08-2025 17:52:54 SddcMgmtDomain[4038]: [INFO] Failed to install bundles. Encountered error:

When retrying

12-08-2025 17:53:13 SddcMgmtDomain[4038]: [INFO] Install VCF bundles in VCF Installer
12-08-2025 17:53:13 SddcMgmtDomain[4038]: [INFO] Getting Bundle Details
12-08-2025 17:53:13 SddcMgmtDomain[4038]: [INFO] Received Bundles. Checking if all VCF 9 bundles are available
12-08-2025 17:53:13 SddcMgmtDomain[4038]: [INFO] Received all Bundle Details
12-08-2025 17:53:13 SddcMgmtDomain[4038]: [INFO] Initiating bundle download
12-08-2025 17:53:14 SddcMgmtDomain[4038]: [ERROR] Error initiating download for component:
12-08-2025 17:53:14 SddcMgmtDomain[4038]: [ERROR] Encountered Error:
Exception: Exiting

[dhruv-tyagi-broadcom]

Great. The '@' problem is a known issue where ovftool does not encode the username and password automatically and hence we need to do it before hand. The commands I asked you to run had the fix applied which is why it worked when done manually.

For your new error, looks like the bundle download failed for some reason. Can you login to VCF Installer UI and check if there is any failed download. If yes, just re-trigger the download from the UI manually. Once all bundles downloaded, run:

vi $config.state

on holorouter while the config is loaded in powershell session, look for "Install-VcfInstallerBundles" and change the status from InProgress to Success. Save the file and run the New-HoloDeckInstance command again and it should continue.

[dhruv-tyagi-broadcom]

Can you run $Error and see what the issue is? When you find the exact line, run $Error[i] | Select * where 'i' is the error line number to get more details and share