Merge pull request #1274 from aravindhp/WINC-830-packer-refactor

WINC-830: Packer refactor

docs/vsphere-golden-image.md

@@ -102,18 +102,19 @@ on TCP port `10250` by running the following PowerShell command:
     New-NetFirewallRule -DisplayName "ContainerLogsPort" -LocalPort 10250 -Enabled True -Direction Inbound -Protocol TCP -Action Allow -EdgeTraversalPolicy Allow
 ```
 
-## 5. Install OS-level container networking patch KB5012637 
+## 5. Install Windows OS updates
 
-Download the patch files from the [Microsoft Update Catalog](https://www.catalog.update.microsoft.com/Search.aspx?q=KB5012637).
-Then, install the patch. Windows Command Prompt example:
-* `wusa.exe C:\PATH-TO-UPDATE\windows10.0-kb5012637-x64.msu /quiet /norestart`
+Ensure that you have the latest
+[Windows Server updates](https://support.microsoft.com/en-us/topic/windows-server-2022-update-history-e1caa597-00c5-4ab9-9f3e-8212fe80b2ee)
+installed.
 
-Alternatively, you can use the PowerShell script [install-kb5012637.ps1](vsphere_ci/scripts/install-kb5012637.ps1) to 
-programmatically download and install the required patch:
+Alternatively, you can use the PowerShell script [install-updates.ps1](vsphere_ci/scripts/install-updates.ps1) to
+programmatically download and install the updates:
 
 ```powershell
-    ./install-kb5012637.ps1
+    ./install-updates.ps1
 ```
+Note that you will have to perform multiple updates and reboots to ensure all the updates are applied.
 
 ## 6. Generalize the virtual machine installation
 

docs/vsphere_ci/README.md

@@ -1,23 +1,22 @@
 # Building the vSphere Windows VM Golden Image programmatically
 
 This document focuses on building Windows VM golden image in vSphere 6.7 and later, to be used in CI but can be 
-used as example for other vSphere environments. We propose [Packer](https://github.com/hashicorp/packer) to automate 
+used as example for other vSphere environments. We use [Packer](https://github.com/hashicorp/packer) to automate the
 installation and configuration of the Windows VM golden image. 
 
 Packer generates a vSphere VM template that can be converted to a virtual machine. After converting the 
 template to virtual machine, the `machine-api` can use this newly created virtual machine (the golden image) in Power-Off state for 
 subsequent VM cloning. 
 
 The above golden image name is the one we use in the Machine Set's `providerSpec.template`. The following steps need 
-to be executed from the bastion host or any instance that has access to the vSphere environment.
-
+to be executed with a sshuttle opened to the vSphere bastion host.
 
 ## Installing Packer
 
-In case of our vSphere environment, we need to install Packer 1.6.6 on the bastion host. The installation steps are:
+Install Packer 1.8.3 on the host where you will be building image. The installation steps are:
 
 - Download [Packer](https://www.packer.io/downloads)
-    - `curl -o packer.zip https://releases.hashicorp.com/packer/1.6.6/packer_1.6.6_linux_amd64.zip`
+    - `curl -o packer.zip https://releases.hashicorp.com/packer/1.8.3/packer_1.8.3_linux_amd64.zip`
 - Unzip using zip utility on the host
     - `gunzip -S .zip packer.zip`
 - Add execution permission to the Packer binary
@@ -27,42 +26,54 @@ In case of our vSphere environment, we need to install Packer 1.6.6 on the basti
 
 ## Prerequisite files
 
-Please ensure the `scripts` directory is present in the location where you are running Packer from on the 
-bastion host and has the following files:
+Please ensure the `scripts` directory is present in the location where you are
+running Packer from and has the following files:
 
     - authorized_keys
-    - autounattend.xml
     - install-vm-tools.cmd
     - configure-vm-tools.ps1
     - install-openssh.ps1
     - install-firewall-rules.ps1
-    - install-kb5012637.ps1
+    - install-updates.ps1
+
+In addition the `answer-file` directory is present at the same level as the `scripts` directory and has the following
+files:
+
+    - autounattend.xml
+    - unattend.xml
 
 The [authorized_keys](scripts/authorized_keys) file must contain a public key, where the private key 
 associated with this public key is what will be used by WMCO to configure VMs created from Windows VM. After 
 deploying WMCO, this private key will be provided by the user in the form of a Secret.
 
-The [autounattend.xml](scripts/autounattend.xml) file must be edited to update the value of 
-`WindowsPassword` with a user provided password. The `ProductKey` must also be updated with a proper value.
-autounattend.xml specifies that the following steps should occur:
+The [autounattend.xml](scripts/autounattend.xml) file automates the Windows installation and  must be edited to update the
+value of `WindowsPassword` with a user provided password. autounattend.xml specifies that the following steps should
+occur after the basic install:
 
 - Runs `install-vm-tools.cmd` script which installs VMWare tools
 - Runs `configure-vm-tools.ps1` script which configures VMWare tools
 - Runs `install-openssh.ps1` script which installs and configures OpenSSH Server
-- Runs `install-firewall-rules.ps1` script which configures the firewall rules
-- Runs `install-kb5012637.ps1` script which installs a required Windows OS-level container networking patch
-
-The above [autounattend.xml](scripts/autounattend.xml) script is different from the [unattend.xml](../unattend.xml)
-as this script does Windows OS installation as well.
 
+Packer takes over after after the initial install and runs [provisioners](https://www.packer.io/docs/provisioners) that
+performs the following:
+- Runs `install-firewall-rules.ps1` script which configures the firewall rules
+- Runs `install-updates.ps1` script which installs the latest updates
+- Reboot to apply the updates
+- Runs `install-updates.ps1` script again to ensure we are installing all updates as some Windows updates requires
+  reboots
+- Reboot again to apply the updates
+- Pauses to wait for the VM to coalesce
+
+Packer then shutdown the VM via sysprep which uses the [answer-files/unattend.xml](unattend.xml). `unattend.xml` is used to
+generalize the VM that is created from the resulting template.
 ## Packer build configuration file
 
 Packer needs a build file which specifies how the virtual machine template should be built. You can find a [reference 
-build file](build.json) in the repository.
+build file](windows-server-2022.pkr.hcl) in the repository.
 
 ### Variables
 
-In order to use the provided [reference build file](build.json) as a valid configuration with Packer, you **must** 
+In order to use the provided [reference build file](windows-server-2022.pkr.hcl) as a valid configuration with Packer, you **must**
 adjust the following variables:
 
 - `<vmtools-iso-path>` Path where VMWare Tools ISO is available in vSphere datacenter
@@ -71,7 +82,7 @@ adjust the following variables:
 - `<vm-template-folder>` Name of the folder where the VM templates will be created by Packer
 - `<vm-template-name>` Name of the VM template that will be created by Packer
 - `<vm-elevated-password>` Password for the Windows virtual machine Administrator user,
-  must match with the password entered in the [autounattend.xml](scripts/autounattend.xml) script
+   must match with the password entered in the [autounattend.xml](answer-files/autounattend.xml) script
 - `<vsphere-cluster>` Name of the vSphere cluster
 - `<vsphere-datacenter>` Name of the vSphere datacenter
 - `<vsphere-datastore>` Name of the vSphere datastore
@@ -83,21 +94,21 @@ adjust the following variables:
 
 ## Building with Packer
 
-Packer relies on a [build file](build.json) for virtual machine template creation.
+Packer relies on a [build file](windows-server-2022.pkr.hcl) for virtual machine template creation.
 
 To build:
 ```bash
-  packer build build.json
+  packer build windows-server-2022.pkr.hcl
 ```
 
 To forcefully rebuild the template:
 ```bash
-  packer build -force build.json
+  packer build -force windows-server-2022.pkr.hcl
 ```
 
 To enable detailed logging:
 ```bash
-  PACKER_LOG=1 packer build build.json
+  PACKER_LOG=1 packer build windows-server-2022.pkr.hcl
 ```
 
 ### What to do during the Packer build
@@ -115,8 +126,11 @@ and hit `Enter` on the keyboard. This should start the OS setup as intended.
 Packer mounts the Windows iso and starts the VM. 
 - All the files in `floppy_files` section of your build file will be copied to the floppy disk of the mounted iso 
  which is represented as `a:\` drive in the Windows VM
-- [autounattend.xml](scripts/autounattend.xml) is a special file in Windows which gets automatically executed once the
-VM starts. You can specify all the commands that needs to executed on first boot.
+- [autounattend.xml](answer-files/autounattend.xml) is a special file in Windows which automates the Windows installation
+  once the VM starts. You can specify the commands in the `FirstLogonCommands` section and they will be executed on the
+  first boot of the VM. These steps should be restricted to basic ones that setup the VM for communication with Packer.
+- Rest of the Windows configuration and setup are performed by the provisioners in
+  [windows-server-2022.pkr.hcl](windows-server-2022.pkr.hcl).
 
 ## Using the virtual machine template
 
@@ -125,5 +139,5 @@ the folder `<vm-template-folder>` following the [Variables](#variables). The lat
 golden image, as described in [the documentation](../vsphere-golden-image.md#9-using-the-virtual-machine-template).
 
 ## References
-- [Sample autounattend](https://github.com/guillermo-musumeci/packer-vsphere-iso-windows/blob/master/win2019.base/win2019.base.json)
-- [Packer unattended windows installs](https://www.packer.io/guides/automatic-operating-system-installs/autounattend_windows)
+- [Sample Packer Windows Server 2022 build](https://github.com/StefanZ8n/packer-ws2022/blob/main/ws2022.pkr.hcl)
+- [Packer Unattended Installation for Windows](https://www.packer.io/guides/automatic-operating-system-installs/autounattend_windows)

docs/vsphere_ci/scripts/autounattend.xml → docs/vsphere_ci/answer-files/autounattend.xml

@@ -62,15 +62,17 @@
             </ImageInstall>
             <UserData>
                 <AcceptEula>true</AcceptEula>
+                <!-- Product Key from https://learn.microsoft.com/en-us/windows-server/get-started/kms-client-activation-keys#windows-server-2022 -->
+                <!-- This allows us to get around having to intervene and skip entering the key during the packer build -->
                 <ProductKey>
-                    <Key>XXXXX-XXXXX-XXXXX-XXXXX-XXXXX</Key>
+                    <Key>VDYBN-27WPP-V4HQT-9VMD4-VMK7H</Key>
                 </ProductKey>
             </UserData>
         </component>
     </settings>
     <settings pass="specialize">
         <component name="Microsoft-Windows-Shell-Setup" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
-            <TimeZone>Romance Standard Time</TimeZone>
+            <TimeZone>UTC</TimeZone>
             <ComputerName>winhost</ComputerName>
         </component>
     </settings>
@@ -113,24 +115,14 @@
                     <Description>Install and configure OpenSSH Server</Description>
                     <CommandLine>cmd.exe /c powershell.exe -File a:\install-openssh.ps1 "a:\authorized_keys"</CommandLine>
                 </SynchronousCommand>
-                <SynchronousCommand wcm:action="add">
-                    <Order>6</Order>
-                    <Description>Configure firewall rules</Description>
-                    <CommandLine>cmd.exe /c powershell.exe -File a:\install-firewall-rules.ps1</CommandLine>
-                </SynchronousCommand>
-                <SynchronousCommand wcm:action="add">
-                    <Order>7</Order>
-                    <Description>Install Windows Patch KB5012637</Description>
-                    <CommandLine>cmd.exe /c powershell.exe -File a:\install-kb5012637.ps1</CommandLine>
-                </SynchronousCommand>
             </FirstLogonCommands>
             <OOBE>
                 <HideEULAPage>true</HideEULAPage>
                 <HideLocalAccountScreen>true</HideLocalAccountScreen>
                 <HideOEMRegistrationScreen>true</HideOEMRegistrationScreen>
                 <HideOnlineAccountScreens>true</HideOnlineAccountScreens>
                 <HideWirelessSetupInOOBE>true</HideWirelessSetupInOOBE>
-                <NetworkLocation>Home</NetworkLocation>
+                <NetworkLocation>Work</NetworkLocation>
                 <ProtectYourPC>1</ProtectYourPC>
             </OOBE>
 

docs/vsphere_ci/answer-files/unattend.xml

@@ -0,0 +1,58 @@
+<?xml version="1.0" encoding="utf-8"?>
+<unattend xmlns="urn:schemas-microsoft-com:unattend">
+    <settings pass="generalize">
+        <component name="Microsoft-Windows-Security-SPP" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
+            <SkipRearm>1</SkipRearm>
+        </component>
+        <component name="Microsoft-Windows-PnpSysprep" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
+            <PersistAllDeviceInstalls>false</PersistAllDeviceInstalls>
+            <DoNotCleanUpNonPresentDevices>false</DoNotCleanUpNonPresentDevices>
+        </component>
+    </settings>
+    <settings pass="specialize">
+        <component name="Microsoft-Windows-Shell-Setup" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
+            <TimeZone>UTC</TimeZone>
+        </component>
+    </settings>
+   <settings pass="oobeSystem">
+      <component name="Microsoft-Windows-International-Core" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
+         <InputLocale>en-US</InputLocale>
+         <SystemLocale>en-US</SystemLocale>
+         <UILanguage>en-US</UILanguage>
+         <UILanguageFallback>en-US</UILanguageFallback>
+         <UserLocale>en-US</UserLocale>
+      </component>
+      <component name="Microsoft-Windows-Shell-Setup" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
+         <OOBE>
+            <HideEULAPage>true</HideEULAPage>
+            <HideLocalAccountScreen>true</HideLocalAccountScreen>
+            <HideOEMRegistrationScreen>true</HideOEMRegistrationScreen>
+            <HideOnlineAccountScreens>true</HideOnlineAccountScreens>
+            <HideWirelessSetupInOOBE>true</HideWirelessSetupInOOBE>
+            <ProtectYourPC>1</ProtectYourPC>
+         </OOBE>
+         <UserAccounts>
+            <AdministratorPassword>
+               <Value>WindowsPassword</Value>
+               <PlainText>true</PlainText>
+            </AdministratorPassword>
+         </UserAccounts>
+            <AutoLogon>
+                <Password>
+                    <Value>WindowsPassword</Value>
+                    <PlainText>true</PlainText>
+                </Password>
+                <LogonCount>1</LogonCount>
+                <Username>Administrator</Username>
+                <Enabled>true</Enabled>
+            </AutoLogon>
+         <FirstLogonCommands>
+            <SynchronousCommand wcm:action="add">
+               <CommandLine>powershell.exe -File C:\rename-computer.ps1</CommandLine>
+               <Order>1</Order>
+            </SynchronousCommand>
+         </FirstLogonCommands>
+      </component>
+   </settings>
+   <cpi:offlineImage cpi:source="wim:c:/dev/source/sources/install.wim#Windows Server 2022 SERVERDATACENTER" xmlns:cpi="urn:schemas-microsoft-com:cpi" />
+</unattend>