This repository has been archived by the owner on Oct 16, 2020. It is now read-only.
Ignition not using guestinfo.coreos.config.data on ESXi #1627
Comments
Nice debugging. That is a strange result. My initial guess is that the guestinfo isn't available early enough for Ignition to read. We can test this by forcing Ignition to run a second time (this is going to break other things, but we'll do it just to test). After the first boot, run the following to reset the disk to the initial state and then reboot: Ignition should then run a second time. If my theory is correct, it will work on this second pass. |
|
I am able to reproduce this issue on my ESXi environment too. And I tried the sgdisk command and reboot, it still didn't work. |
|
Can you attach the logs for Ignition and coreos-cloudinit from that test? |
|
|
I don't see Ignition's logs after the second boot. To be clear, what we are trying to test is whether or not the VMware hypervisor is taking so long to set the guestvariables that Ignition is unable to read them (since it runs first) but coreos-cloudinit is able (because it runs later). The test sequence should be:
That last step is what I am most interested in seeing. If Ignition is able to read the config the second time around, it means that the hypervisor is to blame. If it's still unable to read the config, but coreos-cloudinit is able, then there is something wrong with Ignition. |
|
Closing due to inactivity. |
|
I have the exact same issue with esxi+ovftool+ignition, but it works as expected if I'm using cloud-config. I believe it's caused by the use of --X:injectOvfEnv to ovftool. When this flag is activated all guestinfo properties are located in guestinfo.ovfEnv and you need to parse XML to get the value of guestinfo.coreos.config.data. This is supported in https://github.com/coreos/coreos-cloudinit, but not in https://github.com/coreos/ignition which tries to read the property directly: https://github.com/coreos/ignition/blob/master/internal/providers/vmware/vmware_amd64.go#L38 InjectOvfEnv seems to be required when you are running ovftool without vCenter:
Source: https://www.vmware.com/support/developer/ovf/ovf400/ovftool-400-userguide.pdf |
|
Okay, so apparently there is another way of doing this with ovftool which works with Ignition. Instead of using --X:injectOvfEnv, you can set guestinfo properties directly with --X:guest. This works for me with ovftool 4.2 and ESXi 6.5: |
|
Awesome findings. I'll have to add this to our documentation. |
|
Has anyone done this with terraform? I am using the custom_configuration_parameters on the VM in terraform but on first boot ignition is not run. If I do the disk reset and reboot I see ignition getting run. Was just curious if anyone had been able to get ignition to work with terraform. |
|
I've documented how we use ignition with ovftool over here: http://anton.lindstrom.io/coreos-on-esxi/ |
|
This was actually just fixed by coreos/ignition#384. Once we tag a new version of Ignition and include it in the OS, OVF guest variables will work out of the box. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Issue Report
Using ovftool to deploy the coreos ova to an ESXi instance, it seems that passing an Ignition configuration via the guestinfo properties is not recognised by Ignition
--prop:guestinfo.coreos.config.data="${IGNITION}"
--prop:guestinfo.coreos.config.data.encoding=base64 \
Bug
CoreOS Version
NAME=CoreOS
ID=coreos
VERSION=1122.2.0
VERSION_ID=1122.2.0
BUILD_ID=2016-09-06-1449
PRETTY_NAME="CoreOS 1122.2.0 (MoreOS)"
ANSI_COLOR="1;32"
HOME_URL="https://coreos.com/"
BUG_REPORT_URL="https://github.com/coreos/bugs/issues"
Environment
ESXi
Version:1.4.0
Build number:3959074
ESXi version:6.0.0
ESXi build number:4192238
Expected Behavior
Ignition configures the network interface
Actual Behavior
Ignition reports
failed to fetch config: not a config (empty
Reproduction Steps
2 files attached that are used to deploy the ova
igntion_config
build_server.sh
coreos.zip
Other Information
A base64 encoded version of the ignition config is created by
IGNITION="
base64 ignition_config"The coreos image has been deployed using ovftool with the command ...
ovftool --X:injectOvfEnv --allowExtraConfig
--diskMode=thin
--datastore=${VM_DATASTORE}
--name=${VM_NAME}
--network="${VM_NETWORK}"
--viMemoryResource=${VM_MEMORY}
--prop:guestinfo.coreos.config.data="${IGNITION}"
--prop:guestinfo.coreos.config.data.encoding=base64
--X:waitForIp --powerOn
${DEPLOY_OVA} vi://${ESXI_USERNAME}:${ESXI_PASSWORD}@${ESXI_HOST}
Logging into the deployed coreos
Oct 21 09:00:29 localhost ignition[238]: Ignition v0.8.0
Oct 21 09:00:29 localhost ignition[238]: config successfully fetched
Oct 21 09:00:29 localhost ignition[238]: failed to fetch config: not a config (empty)
Oct 21 09:00:29 localhost ignition[238]: not a config (empty): ignoring user-provided config
Oct 21 09:00:31 localhost ignition[403]: Ignition v0.8.0
Oct 21 09:00:31 localhost ignition[403]: config successfully fetched
Oct 21 09:00:31 localhost ignition[403]: failed to fetch config: not a config (empty)
Oct 21 09:00:31 localhost ignition[403]: not a config (empty): ignoring user-provided config
The text was updated successfully, but these errors were encountered: