vmware: CL support for OVF Environments

[alekssaul]

Versions

• Tectonic version (release or commit hash):
  1.7.1
• Terraform version (terraform version):
  0.9.6
• Platform (aws|azure|openstack|metal):
  vmware

What happened?

Per Scott S. VMware Install failed against CL 1465.6.0 image as the CL image did not abide by not using DHCP provision as expected . Opening an issue for tracking.

edit: based on further digging it seems like new version of ignition (> 0.16) requires ignition configuration to be passed via OVF Environment interface instead of guestinfo

What you expected to happen?

Terraform inject static IP via ignition/systemd-networkd. CL Image should have been configured with static IP instead of DHCP
Tectonic on VMware with CL 1465.6.0 image to be provisioned

How to reproduce it (as minimally and precisely as possible)?

Use Tectonic on VMware guide with CL 1465.6.0 OVA

Anything else we need to know?

Issue created for tracking purposes. Will be investigating further.

[snsumner]

I've confirmed within my own VMware lab environment that this version is ignoring the static IPs provided and using DHCP.

[snsumner]

Error applying plan:

4 error(s) occurred:

• module.workers.vsphere_virtual_machine.node[0]: 1 error(s) occurred:

• ssh: handshake failed: ssh: unable to authenticate, attempted methods [publickey none], no supported methods remain

• module.masters.vsphere_virtual_machine.node[0]: 1 error(s) occurred:

• ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

• module.masters.vsphere_virtual_machine.node[1]: 1 error(s) occurred:

• ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

• module.workers.vsphere_virtual_machine.node[1]: 1 error(s) occurred:

• ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

[alekssaul]

/cc: @crawford to keep me honest. + @robszumski due to Tectonic on VMware impact

Tectonic for VMware is using guestinfo.coreos.config.data to pass ignition data into CoreOS CL OVA template. With the change in; coreos/ignition@cb09b1b ignition will detect that OVF environment is in use and ignore guestinfo.coreos.config.data under the assumption that user either interactively (or via OVF spec) deployed the virtual machine.

Tectonic / Terraform workflow is stuck in the middle where OVF template is used, but no data is passed via OVF interface. Resulting in ignition not getting configuration data.

I can see 3 options to resolve:

1. have the end user work around manually. When end-user imports CL image into vSphere, end user will have to manually un-check "Enable vApp Options" button. I've tested this with 1465.6.0 release with success.
   

2. have ignition attempt to fall back (or default) to guestinfo.coreos.config.data if env.Properties["guestinfo.coreos.config.data"], is empty. (Somewhere here; https://github.com/coreos/ignition/blob/v0.16.0/internal/providers/vmware/vmware_amd64.go#L68-L69)

3. Add support to upstream Terraform to enable passing OVF spec into VM creation action and modify Tectonic Terraform to use the OVF Spec instead of guestinfo.coreos.config.data

IMHO quickest resolution is (option 1) to add documentation to disable "Enable vApp Options" into Step 2/3 of https://github.com/coreos/tectonic-installer/blob/master/Documentation/install/vmware/vmware-terraform.md#prerequsities. Option 3 would be a good long term goal.

[snsumner]

I confirmed in my ESXi 6.5u1 lab environment that workaround #1 to uncheck vApps option did resolve the issue.

[crawford]

@alekssaul What are the contents of the "ovfenv" variable when you un-check that option?

[amarruedo]

@alekssaul regarding option 1 mentioned in:

1. have the end user work around manually. When end-user imports CL image into vSphere, end user will have to manually un-check "Enable vApp Options" button. I've tested this with 1465.6.0 release with success.

I cannot un-check "Enable vApp Options" button since that option is missing on my machines (we don't have appropriate license to enable DSR on our cluster). Does this mean that I cannot deploy CoreOS virtual machines on vSphere using Terraform? (I've tried with different versions of terraform-vsphere-provider, using custom_configuration_parameters and the newer extra_configs to pass guestinfo.coreos.config.data, but it is always ignored).