Skip to content
This repository has been archived by the owner. It is now read-only.

Guestinfo configuration applied in wrong order #2630

theit8514 opened this issue Nov 8, 2019 · 4 comments

Guestinfo configuration applied in wrong order #2630

theit8514 opened this issue Nov 8, 2019 · 4 comments


Copy link

@theit8514 theit8514 commented Nov 8, 2019

Issue Report


When applying a guestinfo configuration in VMware, CoreOS attempts to connect to the guestinfo.coreos.config.url before configuring the interface in guestinfo.interface.0.ip.0.address. If it matters, this host is getting an IPv6 configuration via autoconfig but not an IPv4 DHCP address.

Container Linux Version

$ cat /etc/os-release
NAME="Container Linux by CoreOS"
PRETTY_NAME="Container Linux by CoreOS 2247.6.0 (Rhyolite)"


What hardware/cloud provider/hypervisor is being used to run Container Linux?
VMware ESXi 6.7 U2

Expected Behavior

CoreOS should configure the network prior to fetching guestinfo.coreos.config.url or retry the fetch after the network has been configured if the previous fetch failed.

Actual Behavior

coreos-cloudinit[659]: 2019/11/08 07:10:48 Unable to fetch data: Get https://host/coreos/path/user_data: dial tcp: lookup host: no such host

Reproduction Steps

  1. Deploy VMware OVA, configuring the URL and static IP on a VM Network that has no DHCP.
  2. Boot the VM.
  3. CoreOS attempts to download the URL 15 times, then configures the static IP. The URL is not fetched or loaded.

Other Information

vApp environment xml:

<?xml version="1.0" encoding="UTF-8"?>
      <Kind>VMware ESXi</Kind>
      <Vendor>VMware, Inc.</Vendor>
         <Property oe:key="" oe:value=""/>
         <Property oe:key="" oe:value=""/>
         <Property oe:key="guestinfo.coreos.config.url" oe:value="https://host/coreos/path/user_data"/>
         <Property oe:key="guestinfo.dns.server.0" oe:value=""/>
         <Property oe:key="guestinfo.dns.server.1" oe:value=""/>
         <Property oe:key="guestinfo.hostname" oe:value="docker-node2"/>
         <Property oe:key="guestinfo.interface.0.dhcp" oe:value="no"/>
         <Property oe:key="guestinfo.interface.0.ip.0.address" oe:value=""/>
         <Property oe:key="guestinfo.interface.0.mac" oe:value="00:50:56:96:5a:ca"/>
         <Property oe:key="" oe:value=""/>
         <Property oe:key="guestinfo.interface.0.role" oe:value="public"/>
         <Property oe:key="guestinfo.interface.0.route.0.destination" oe:value=""/>
         <Property oe:key="guestinfo.interface.0.route.0.gateway" oe:value=""/>
      <ve:Adapter ve:mac="00:50:56:96:5a:ca" ve:network="VM Network" ve:unitNumber="7"/>
Copy link

@lucab lucab commented Nov 8, 2019

@theit8514 thanks for the report. The configuration you are trying to apply is for cloud-init, which is known to have such race issues and has been superseded by Ignition.

The recommendation is to follow and stick to (plus .encoding) to pass your Ignition configuration.

Copy link

@theit8514 theit8514 commented Nov 9, 2019

Thanks for the reply. The only thing I didn't state from the above is that these settings were generated by the CoreOS OVA file.

I have tried switching over to the but until just now I had no luck getting it to work. Typing out the process here, I realized that I was base64 encoding the cloud-config instead of the ignition transpiled file. Now cloudinit recognizes an ignition file with this log entry: Detected an Ignition config. Exiting...

I am still seeing a race condition with my config: The IP address is not pingable and Ignition is attempting to fetch the config but failing now due to systemd-resolved "connection refused". It also seems to be stuck in an infinite boot loop waiting for ignition to complete.

   "systemd" : {},
   "passwd" : {},
   "storage" : {},
   "ignition" : {
      "security" : {
         "tls" : {}
      "timeouts" : {},
      "config" : {
         "replace" : {
            "verification" : {},
            "source" : "https://host/coreos/path/user_data"
      "version" : "2.2.0"
   "networkd" : {
      "units" : [
            "name" : "",
            "contents" : "[Match]\nMACAddress=00:50:56:96:5a:ca\n\n[Network]\nDNS=\nDNS=\nAddress=\nGateway=\n"

Screenshot from 2019-11-09 10-38-38

I have tried both config>replace and config>append.

Copy link

@theit8514 theit8514 commented Nov 9, 2019

I found this document stating that my use case is not supported and to use kernel ip= parameters:

Attempting to do so (ip= still fails with the above config. Has a similar error trying to resolve the domain with systemd-resolved and attempts to download infinitely.

Copy link

@lucab lucab commented Nov 11, 2019

@theit8514 at an high level, you are expecting Ignition to fetch and apply a configuration to change the initramfs (i.e. the environment where Ignition is running), in order to influence how Ignition fetches its configuration. This is a fundamental chicken-egg problem which (by design) cannot be solved by Ignition alone.

There are have a few ways out of that:

  • set up DHCP on the network for the main interface of the VM. This will result in a proper network environment in the initramfs, which can be used by Ignition to perform remote fetches.
  • pass the whole configuration via This will completely avoid network fetches in the initramfs.
  • setting up static network via kernel arguments (as in your last comment). This should work but is a bit unwieldy compared to the other ways, and it needs to be performed at boot-loading time on first-boot.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
None yet

No branches or pull requests

2 participants