feat: add support for iso_url in builder configuration (#249)

Add support for `iso_url` in the builder configuration blocks.

Signed-off-by: Ryan Johnson <johnsonryan@vmware.com>

README.md

@@ -349,17 +349,7 @@ The directory structure of the repository.
 ├── ansible
 │   ├── roles
 │   │   └── <role>
-│   │       ├── defaults
-│   │       │   └── main.yml
-│   │       ├── handlers
-│   │       │   └── main.yml
-│   │       ├── meta
-│   │       │   └── main.yml
-│   │       ├── tasks
-│   │       │   └── main.yml
-│   │       │   └── *.yml
-│   │       └── vars
-│   │           └── main.yml
+│   │       └── *.yml
 │   ├── ansible.cfg
 │   └── main.yml
 ├── artifacts
@@ -397,7 +387,7 @@ The directory structure of the repository.
 The files are distributed in the following directories.
 
 * **`ansible`** - contains the Ansible roles to prepare Linux machine image builds.
-* **`artifacts`** - contains the OVF artifacts created by the builds, if enabled.
+* **`artifacts`** - contains the OVF artifacts exported by the builds, if enabled.
 * **`builds`** - contains the templates, variables, and configuration files for the machine image builds.
 * **`scripts`** - contains the scripts to initialize and prepare Windows machine image builds.
 * **`manifests`** - manifests created after the completion of the machine image builds.
@@ -407,13 +397,20 @@ The files are distributed in the following directories.
 >
 > When forking the project for upstream contribution, please be mindful not to make changes that may expose your sensitive information, such as passwords, keys, certificates, etc.
 
-### Step 2 - Download the Guest Operating Systems ISOs
+### Step 2 - Guest Operating Systems ISOs
 
-1. Download the x64 guest operating system [.iso][iso] images.
+The project supports configuring the ISO from either a datastore or URL source. By default, the project uses the datastore source.
+
+Follow the steps below to configure either option.
+#### Using a Datastore Source
+
+If you are using a datastore to store your guest operating system [`.iso`][iso] files, you must download and upload these to a datastore path.
+
+1. Download the x64 guest operating system `.iso` files.
 
     Linux Distributions:
 
-    * VMware Photon OS 4 Server
+    * VMware Photon OS 4
         * [Download][download-linux-photon-server-4] the 4.0 Rev2 release of the **FULL** `.iso` image. (_e.g._ `photon-4.0-xxxxxxxxx.iso`)
     * Debian 11
         * [Download][download-linux-debian-11] the latest **netinst** release `.iso` image. (_e.g._ `debian-11.x.0-amd64-netinst.iso`)
@@ -453,9 +450,9 @@ The files are distributed in the following directories.
     * Microsoft Windows 11
     * Microsoft Windows 10
 
-1. Obtain the checksum type (_e.g.,_ `sha256`, `md5`, etc.) and checksum value for each guest operating system `.iso` image from the vendor. This will be use in the build input variables.
+1. Obtain the checksum type (_e.g.,_ `sha256`, `md5`, etc.) and checksum value for each guest operating system `.iso` from the vendor. This will be use in the build input variables.
 
-1. [Upload][vsphere-upload] your guest operating system `.iso` images to the ISO datastore and paths that will be used in your variables.
+1. [Upload][vsphere-upload] or your guest operating system `.iso` files to the datastore and update the configuration variables, leaving the `iso_url` variable as `null`.
 
     **Example**: `config/common.pkvars.hcl`
 
@@ -466,12 +463,25 @@ The files are distributed in the following directories.
     **Example**: `builds/<type>/<build>/*.auto.pkvars.hcl`
 
     ```hcl
+    iso_url            = null
     iso_path           = "iso/linux/photon"
     iso_file           = "photon-4.0-xxxxxxxxx.iso"
     iso_checksum_type  = "md5"
     iso_checksum_value = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
     ```
 
+#### Using a URL Source
+
+If you are using a URL source to obtain your guest operating system [`.iso`][iso] files, you must update the input variables to use the URL source.
+
+Update the `iso_url` variable to download the `.iso` from a URL. The `iso_url` variable takes presedence over any other `iso_*` variables.
+
+  **Example**: `builds/<type>/<build>/*.auto.pkvars.hcl`
+
+```hcl
+iso_url = "https://artifactory.rainpole.io/iso/linux/photon/4.0/x86_64/photon-4.0-xxxxxxxxx.iso"
+```
+
 ### Step 3 - Configure Service Account Privileges in vSphere
 
 Create a custom vSphere role with the required privileges to integrate HashiCorp Packer with VMware vSphere. A service account can be added to the role to ensure that Packer has least privilege access to the infrastructure. Clone the default **Read-Only** vSphere role and add the following privileges:
@@ -660,7 +670,8 @@ The content of the public key, `build_key`, is added the key to the `.ssh/author
 
 > **Warning**
 >
-> Replace the default public keys and passwords.
+> Replace the default public keys and passwords.  
+>
 > By default, both Public Key Authentication and Password Authentication are enabled for Linux distributions. If you wish to disable Password Authentication and only use Public Key Authentication, comment or remove the portion of the associated Ansible `configure` role.
 
 ##### Ansible Variables
@@ -767,7 +778,7 @@ Edit the `config/proxy.pkvars.hcl` file to configure the following:
 
 ```hcl
 communicator_proxy_host     = "proxy.rainpole.io"
-communicator_proxy_port     = 1080
+communicator_proxy_port     = 8080
 communicator_proxy_username = "rainpole"
 communicator_proxy_password = "<plaintext_password>"
 ```

builds/linux/almalinux/8/linux-almalinux.auto.pkrvars.hcl

@@ -28,6 +28,7 @@ vm_disk_thin_provisioned = true
 vm_network_card          = "vmxnet3"
 
 // Removable Media Settings
+iso_url            = null
 iso_path           = "iso/linux/almalinux"
 iso_file           = "AlmaLinux-8.6-x86_64-dvd.iso"
 iso_checksum_type  = "sha256"

builds/linux/almalinux/8/linux-almalinux.pkr.hcl

@@ -86,6 +86,7 @@ source "vsphere-iso" "linux-almalinux" {
   notes                = local.build_description
 
   // Removable Media Settings
+  iso_url      = var.iso_url
   iso_paths    = local.iso_paths
   iso_checksum = local.iso_checksum
   http_content = var.common_data_source == "http" ? local.data_source_content : null

builds/linux/almalinux/8/variables.pkr.hcl

@@ -230,6 +230,11 @@ variable "common_iso_datastore" {
   description = "The name of the source vSphere datastore for ISO images. (e.g. 'sfo-w01-cl01-nfs01')"
 }
 
+variable "iso_url" {
+  type        = string
+  description = "The URL source of the ISO image. (e.g. 'https://artifactory.rainpole.io/.../os.iso')"
+}
+
 variable "iso_path" {
   type        = string
   description = "The path on the source vSphere datastore for ISO image. (e.g. 'iso'/linux/almalinux')"

builds/linux/almalinux/9/linux-almalinux.auto.pkrvars.hcl

@@ -28,6 +28,7 @@ vm_disk_thin_provisioned = true
 vm_network_card          = "vmxnet3"
 
 // Removable Media Settings
+iso_url            = null
 iso_path           = "iso/linux/almalinux"
 iso_file           = "AlmaLinux-9.0-x86_64-dvd.iso"
 iso_checksum_type  = "sha256"

builds/linux/almalinux/9/linux-almalinux.pkr.hcl

@@ -86,6 +86,7 @@ source "vsphere-iso" "linux-almalinux" {
   notes                = local.build_description
 
   // Removable Media Settings
+  iso_url      = var.iso_url
   iso_paths    = local.iso_paths
   iso_checksum = local.iso_checksum
   http_content = var.common_data_source == "http" ? local.data_source_content : null

builds/linux/almalinux/9/variables.pkr.hcl

@@ -230,6 +230,11 @@ variable "common_iso_datastore" {
   description = "The name of the source vSphere datastore for ISO images. (e.g. 'sfo-w01-cl01-nfs01')"
 }
 
+variable "iso_url" {
+  type        = string
+  description = "The URL source of the ISO image. (e.g. 'https://artifactory.rainpole.io/.../os.iso')"
+}
+
 variable "iso_path" {
   type        = string
   description = "The path on the source vSphere datastore for ISO image. (e.g. 'iso'/linux/almalinux')"

builds/linux/centos/7/linux-centos.auto.pkrvars.hcl

@@ -28,6 +28,7 @@ vm_disk_thin_provisioned = true
 vm_network_card          = "vmxnet3"
 
 // Removable Media Settings
+iso_url            = null
 iso_path           = "iso/linux/centos"
 iso_file           = "CentOS-7-x86_64-DVD-2009.iso"
 iso_checksum_type  = "sha256"

builds/linux/centos/7/linux-centos.pkr.hcl

@@ -86,6 +86,7 @@ source "vsphere-iso" "linux-centos" {
   notes                = local.build_description
 
   // Removable Media Settings
+  iso_url      = var.iso_url
   iso_paths    = local.iso_paths
   iso_checksum = local.iso_checksum
   http_content = var.common_data_source == "http" ? local.data_source_content : null

builds/linux/centos/7/variables.pkr.hcl

@@ -230,6 +230,11 @@ variable "common_iso_datastore" {
   description = "The name of the source vSphere datastore for ISO images. (e.g. 'sfo-w01-cl01-nfs01')"
 }
 
+variable "iso_url" {
+  type        = string
+  description = "The URL source of the ISO image. (e.g. 'https://artifactory.rainpole.io/.../os.iso')"
+}
+
 variable "iso_path" {
   type        = string
   description = "The path on the source vSphere datastore for ISO image. (e.g. 'iso/linux/centos')"

builds/linux/centos/8-stream/linux-centos-stream.auto.pkrvars.hcl

@@ -28,6 +28,7 @@ vm_disk_thin_provisioned = true
 vm_network_card          = "vmxnet3"
 
 // Removable Media Settings
+iso_url            = null
 iso_path           = "iso/linux/centos"
 iso_file           = "CentOS-Stream-8-x86_64-latest-dvd1.iso"
 iso_checksum_type  = "sha256"

builds/linux/centos/8-stream/linux-centos-stream.pkr.hcl

@@ -86,6 +86,7 @@ source "vsphere-iso" "linux-centos-stream" {
   notes                = local.build_description
 
   // Removable Media Settings
+  iso_url      = var.iso_url
   iso_paths    = local.iso_paths
   iso_checksum = local.iso_checksum
   http_content = var.common_data_source == "http" ? local.data_source_content : null

builds/linux/centos/8-stream/variables.pkr.hcl

@@ -230,6 +230,11 @@ variable "common_iso_datastore" {
   description = "The name of the source vSphere datastore for ISO images. (e.g. 'sfo-w01-cl01-nfs01')"
 }
 
+variable "iso_url" {
+  type        = string
+  description = "The URL source of the ISO image. (e.g. 'https://artifactory.rainpole.io/.../os.iso')"
+}
+
 variable "iso_path" {
   type        = string
   description = "The path on the source vSphere datastore for ISO image. (e.g. 'iso/linux/centos')"

builds/linux/centos/9-stream/linux-centos-stream.auto.pkrvars.hcl

@@ -28,6 +28,7 @@ vm_disk_thin_provisioned = true
 vm_network_card          = "vmxnet3"
 
 // Removable Media Settings
+iso_url            = null
 iso_path           = "iso/linux/centos"
 iso_file           = "CentOS-Stream-9-latest-x86_64-dvd1.iso"
 iso_checksum_type  = "sha256"

builds/linux/centos/9-stream/linux-centos-stream.pkr.hcl

@@ -86,6 +86,7 @@ source "vsphere-iso" "linux-centos-stream" {
   notes                = local.build_description
 
   // Removable Media Settings
+  iso_url      = var.iso_url
   iso_paths    = local.iso_paths
   iso_checksum = local.iso_checksum
   http_content = var.common_data_source == "http" ? local.data_source_content : null

builds/linux/centos/9-stream/variables.pkr.hcl

@@ -230,6 +230,11 @@ variable "common_iso_datastore" {
   description = "The name of the source vSphere datastore for ISO images. (e.g. 'sfo-w01-cl01-nfs01')"
 }
 
+variable "iso_url" {
+  type        = string
+  description = "The URL source of the ISO image. (e.g. 'https://artifactory.rainpole.io/.../os.iso')"
+}
+
 variable "iso_path" {
   type        = string
   description = "The path on the source vSphere datastore for ISO image. (e.g. 'iso/linux/centos')"

builds/linux/debian/11/linux-debian.auto.pkrvars.hcl

@@ -28,6 +28,7 @@ vm_disk_thin_provisioned = false
 vm_network_card          = "vmxnet3"
 
 // Removable Media Settings
+iso_url            = null
 iso_path           = "iso/linux/debian"
 iso_file           = "debian-11.4.0-amd64-netinst.iso"
 iso_checksum_type  = "sha256"

builds/linux/debian/11/linux-debian.pkr.hcl

@@ -20,9 +20,9 @@ packer {
 //  Defines the local variables.
 
 locals {
-  build_by      = "Built by: HashiCorp Packer ${packer.version}"
-  build_date    = formatdate("YYYY-MM-DD hh:mm ZZZ", timestamp())
-  build_version = formatdate("YY.MM", timestamp())
+  build_by          = "Built by: HashiCorp Packer ${packer.version}"
+  build_date        = formatdate("YYYY-MM-DD hh:mm ZZZ", timestamp())
+  build_version     = formatdate("YY.MM", timestamp())
   build_description = "Version: v${local.build_version}\nBuilt on: ${local.build_date}\n${local.build_by}"
   iso_paths         = ["[${var.common_iso_datastore}] ${var.iso_path}/${var.iso_file}"]
   iso_checksum      = "${var.iso_checksum_type}:${var.iso_checksum_value}"
@@ -85,8 +85,9 @@ source "vsphere-iso" "linux-debian" {
   notes                = local.build_description
 
   // Removable Media Settings
-  iso_paths    = local.iso_paths
-  iso_checksum = local.iso_checksum
+  iso_url        = var.iso_url
+  iso_paths      = local.iso_paths
+  iso_checksum   = local.iso_checksum
   http_content   = var.common_data_source == "http" ? local.data_source_content : null
   floppy_content = var.common_data_source == "disk" ? local.data_source_content : null
 
@@ -133,7 +134,7 @@ source "vsphere-iso" "linux-debian" {
       skip_import = var.common_content_library_skip_export
     }
   }
-  
+
   // OVF Export Settings
   dynamic "export" {
     for_each = var.common_ovf_export_enabled == true ? [1] : []

builds/linux/debian/11/variables.pkr.hcl

@@ -230,6 +230,11 @@ variable "common_iso_datastore" {
   description = "The name of the source vSphere datastore for ISO images. (e.g. 'sfo-w01-cl01-nfs01')"
 }
 
+variable "iso_url" {
+  type        = string
+  description = "The URL source of the ISO image. (e.g. 'https://artifactory.rainpole.io/.../os.iso')"
+}
+
 variable "iso_path" {
   type        = string
   description = "The path on the source vSphere datastore for ISO image. (e.g. 'iso/linux/debian')"

builds/linux/photon/4/linux-photon.auto.pkrvars.hcl

@@ -25,6 +25,7 @@ vm_disk_thin_provisioned = true
 vm_network_card          = "vmxnet3"
 
 // Removable Media Settings
+iso_url            = null
 iso_path           = "iso/linux/photon"
 iso_file           = "photon-4.0-c001795b8.iso"
 iso_checksum_type  = "md5"
@@ -36,4 +37,4 @@ vm_boot_wait  = "2s"
 
 // Communicator Settings
 communicator_port    = 22
-communicator_timeout = "30m"
+communicator_timeout = "30m"

builds/linux/photon/4/linux-photon.pkr.hcl

@@ -83,6 +83,7 @@ source "vsphere-iso" "linux-photon" {
   notes                = local.build_description
 
   // Removable Media Settings
+  iso_url      = var.iso_url
   iso_paths    = local.iso_paths
   iso_checksum = local.iso_checksum
   http_content = var.common_data_source == "http" ? local.data_source_content : null

builds/linux/photon/4/variables.pkr.hcl

@@ -212,6 +212,11 @@ variable "common_iso_datastore" {
   description = "The name of the source vSphere datastore for ISO images. (e.g. 'sfo-w01-cl01-nfs01')"
 }
 
+variable "iso_url" {
+  type        = string
+  description = "The URL source of the ISO image. (e.g. 'https://artifactory.rainpole.io/.../os.iso')"
+}
+
 variable "iso_path" {
   type        = string
   description = "The path on the source vSphere datastore for ISO image. (e.g. 'iso/linux/photon')"

builds/linux/rhel/7/linux-rhel.auto.pkrvars.hcl

@@ -28,6 +28,7 @@ vm_disk_thin_provisioned = true
 vm_network_card          = "vmxnet3"
 
 // Removable Media Settings
+iso_url            = null
 iso_path           = "iso/linux/rhel"
 iso_file           = "rhel-server-7.9-x86_64-dvd.iso"
 iso_checksum_type  = "sha256"
@@ -39,4 +40,4 @@ vm_boot_wait  = "2s"
 
 // Communicator Settings
 communicator_port    = 22
-communicator_timeout = "30m"
+communicator_timeout = "30m"

builds/linux/rhel/7/linux-rhel.pkr.hcl

@@ -88,6 +88,7 @@ source "vsphere-iso" "linux-rhel" {
   notes                = local.build_description
 
   // Removable Media Settings
+  iso_url      = var.iso_url
   iso_paths    = local.iso_paths
   iso_checksum = local.iso_checksum
   http_content = var.common_data_source == "http" ? local.data_source_content : null

builds/linux/rhel/7/variables.pkr.hcl

@@ -244,6 +244,11 @@ variable "common_iso_datastore" {
   description = "The name of the source vSphere datastore for ISO images. (e.g. 'sfo-w01-cl01-nfs01')"
 }
 
+variable "iso_url" {
+  type        = string
+  description = "The URL source of the ISO image. (e.g. 'https://artifactory.rainpole.io/.../os.iso')"
+}
+
 variable "iso_path" {
   type        = string
   description = "The path on the source vSphere datastore for ISO image. (e.g. 'iso/linux/rhel')"

builds/linux/rhel/8/linux-rhel.auto.pkrvars.hcl

@@ -28,6 +28,7 @@ vm_disk_thin_provisioned = true
 vm_network_card          = "vmxnet3"
 
 // Removable Media Settings
+iso_url            = null
 iso_path           = "iso/linux/rhel"
 iso_file           = "rhel-8.6-x86_64-dvd.iso"
 iso_checksum_type  = "sha256"

builds/linux/rhel/8/linux-rhel.pkr.hcl

@@ -88,6 +88,7 @@ source "vsphere-iso" "linux-rhel" {
   notes                = local.build_description
 
   // Removable Media Settings
+  iso_url      = var.iso_url
   iso_paths    = local.iso_paths
   iso_checksum = local.iso_checksum
   http_content = var.common_data_source == "http" ? local.data_source_content : null