Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3 region invalid if ec2-metadata fails on first attempt #826

Closed
wernerb opened this issue Jul 10, 2019 · 5 comments

Comments

@wernerb
Copy link

commented Jul 10, 2019

Bug

When using S3 source, and network isn't ready for its first metadata retrieval, the regionHint is nil and so defaults to us-east-1. This happens 1 in 5 times when the networkadapter is a little slow to attach.

The impact is large for us as we make heavy use of security rules, we use prefixLists from our VPC to ensure only outgoing s3 connections to eu-central-1 are allowed, which in this case fails because us-east-1 is erronously chosen

Operating System Version

stable

Ignition Version

v0.32.0

Environment

AWS with security rules egress for specific S3 endpoints (https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-prefix-lists.html)

Expected Behavior

Client retries a few times and succesfully sets regionHint to the correct region after retrieving metadata.

Actual Behavior

Timeouts happen on ec2 metadata, and finally do go through but regionHint is nil so subsequent s3 calls fail as they go to the wrong region

Reproduction Steps

0: In a different region than us-east-1 (e.g., eu-central-1), add egress security group rules to only allow cidrs from your region:

{
  "ip_prefix": "52.219.72.0/22",
  "region": "eu-central-1",
  "service": "S3"
}
{
  "ip_prefix": "52.219.44.0/22",
  "region": "eu-central-1",
  "service": "S3"
}
{
  "ip_prefix": "52.92.68.0/22",
  "region": "eu-central-1",
  "service": "S3"
}
{
  "ip_prefix": "54.231.192.0/20",
  "region": "eu-central-1",
  "service": "S3"
}

1: Use userdata like:

{
    "ignition": {
        "config": {
            "replace": [{
                "source": "s3://ourbucket?versionId=version"
            }]
        },
        "version": "2.2.0"
    }
}
  1. Launch individual instances in eu-central-1 a few times and check System log. 1 in 5 will error with a timeout as in the log attached

Partial Log:

Here we see at 9th attempt userdata is available

[   25.121134] ignition[395]: GET http://169.254.169.254/2009-04-04/user-data: attempt #9
[   25.130785] ignition[395]: GET result: OK

but we see that the region is set to us-east-1 as 52.216 belongs to US-EAST-1 according to amazon. In the code we see that regionHint if nil is set to us-east-1`

[  145.459009] ignition[395]: failed to fetch config: RequestError: send request failed
caused by: dial tcp 52.216.109.237:443: i/o timeout
[  145.468215] ignition[395]: failed to acquire config: RequestError: send request failed
caused by: dial tcp 52.216.109.237:443: i/o timeout

Full log

[    1.092988] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
[    1.093954] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
[    1.094889] ACPI: PCI Interrupt Link [LNKD] (IRQs *5 10 11)
[    1.120426] xen:balloon: Initialising balloon driver
[    1.146389] pci 0000:00:02.0: vgaarb: setting as boot VGA device
[    1.146636] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
[    1.146657] pci 0000:00:02.0: vgaarb: bridge control possible
[    1.147644] vgaarb: loaded
[    1.148702] pps_core: LinuxPPS API ver. 1 registered
[    1.149642] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    1.150653] PTP clock support registered
[    1.151673] PCI: Using ACPI for IRQ routing
[    1.153527] HPET: 3 timers in total, 0 timers will be used for per-cpu timer
[    1.153658] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[    1.154643] hpet0: 3 comparators, 64-bit 62.500000 MHz counter
[    1.157665] clocksource: Switched to clocksource xen
[    1.173300] VFS: Disk quotas dquot_6.6.0
[    1.176243] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    1.180858] pnp: PnP ACPI init
[    1.182741] system 00:00: [mem 0x00000000-0x0009ffff] could not be reserved
[    1.187428] system 00:01: [io  0x08a0-0x08a3] has been reserved
[    1.191718] system 00:01: [io  0x0cc0-0x0ccf] has been reserved
[    1.195311] system 00:01: [io  0x04d0-0x04d1] has been reserved
[    1.199290] system 00:07: [io  0x10c0-0x1141] has been reserved
[    1.203336] system 00:07: [io  0xb044-0xb047] has been reserved
[    1.248965] pnp: PnP ACPI: found 8 devices
[    1.260832] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[    1.267495] NET: Registered protocol family 2
[    1.271503] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes)
[    1.279598] TCP established hash table entries: 32768 (order: 6, 262144 bytes)
[    1.287339] TCP bind hash table entries: 32768 (order: 7, 524288 bytes)
[    1.295635] TCP: Hash tables configured (established 32768 bind 32768)
[    1.302411] UDP hash table entries: 2048 (order: 4, 65536 bytes)
[    1.307371] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes)
[    1.312322] NET: Registered protocol family 1
[    1.315965] NET: Registered protocol family 44
[    1.320720] pci 0000:00:01.0: PIIX3: Enabling Passive Release
[    1.326362] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[    1.330665] pci 0000:00:01.0: Activating ISA DMA hang workarounds
[    1.336473] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[    1.872360] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[    1.876830] software IO TLB: mapped [mem 0xec000000-0xf0000000] (64MB)
[    1.881359] RAPL PMU: API unit is 2^-32 Joules, 3 fixed counters, 655360 ms ovfl timer
[    1.886707] RAPL PMU: hw unit of domain pp0-core 2^-14 Joules
[    1.889837] RAPL PMU: hw unit of domain package 2^-14 Joules
[    1.893126] RAPL PMU: hw unit of domain dram 2^-16 Joules
[    1.897028] Initialise system trusted keyrings
[    1.900109] workingset: timestamp_bits=39 max_order=20 bucket_order=0
[    2.152561] Key type asymmetric registered
[    2.156596] Asymmetric key parser 'x509' registered
[    2.160860] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 249)
[    2.167762] io scheduler noop registered
[    2.171289] io scheduler deadline registered
[    2.175694] io scheduler cfq registered (default)
[    2.179957] io scheduler mq-deadline registered
[    2.183498] io scheduler kyber registered
[    2.186435] io scheduler bfq registered
[    2.189803] ioatdma: Intel(R) QuickData Technology Driver 4.00
[    2.196967] xen:grant_table: Grant tables using version 1 layout
[    2.200063] Grant table initialized
[    2.201887] Cannot get hvm parameter CONSOLE_EVTCHN (18): -22!
[    2.205384] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    2.237483] 00:06: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[    2.242901] i8042: PNP: PS/2 Controller [PNP0303:PS2K,PNP0f13:PS2M] at 0x60,0x64 irq 1,12
[    2.249864] serio: i8042 KBD port at 0x60,0x64 irq 1
[    2.252778] serio: i8042 AUX port at 0x60,0x64 irq 12
[    2.258056] rtc_cmos 00:02: registered as rtc0
[    2.258064] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0
[    2.260589] rtc_cmos 00:02: alarms up to one day, 114 bytes nvram, hpet irqs
[    2.270221] NET: Registered protocol family 10
[    2.273391] Segment Routing with IPv6
[    2.275694] NET: Registered protocol family 17
[    2.278440] Key type dns_resolver registered
[    2.281102] mce: Using 2 MCE banks
[    2.283126] sched_clock: Marking stable (1747258367, 535834786)->(2633474216, -350381063)
[    2.288139] registered taskstats version 1
[    2.290482] Loading compiled-in X.509 certificates
[    2.326136] Loaded X.509 cert 'CoreOS, Inc: Module signing key for 4.19.50-coreos-r1: 14702b0bbd56293cc751b403543deb85705a38be'
[    2.332830] ima: No TPM chip found, activating TPM-bypass!
[    2.336184] ima: Allocated hash algorithm: sha1
[    2.339849] xenbus_probe_frontend: Device with no driver: device/vbd/51712
[    2.348437] xenbus_probe_frontend: Device with no driver: device/vif/0
[    2.352255] rtc_cmos 00:02: setting system clock to 2019-07-03 14:44:26 UTC (1562165066)
[    2.378526] Freeing unused kernel image memory: 40656K
[    2.385718] Write protecting the kernel read-only data: 18432k
[    2.390494] Freeing unused kernel image memory: 2016K
[    2.394032] Freeing unused kernel image memory: 804K
[    2.397539] Run /init as init process
[    2.406061] systemd[1]: systemd 241-23-g05e654e+ running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK -SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT -GNUTLS -ACL +XZ +LZ4 +SECCOMP +BLKID -ELFUTILS +KMOD -IDN2 -IDN +PCRE2 default-hierarchy=legacy)
[    2.431926] systemd[1]: Detected virtualization xen.
[    2.436792] systemd[1]: Detected architecture x86-64.
[    2.442737] systemd[1]: Running in initial RAM disk.

Welcome to [0;34mdracut-045 (Initramfs)[0m!

[    2.457052] systemd[1]: No hostname configured.
[    2.461212] systemd[1]: Set hostname to <localhost>.
[    2.465763] systemd[1]: Initializing machine ID from random generator.
[    2.543571] systemd[1]: File /usr/lib/systemd/system/systemd-journald.service:12 configures an IP firewall (IPAddressDeny=any), but the local system does not support BPF/cgroup based firewalling.
[    2.561872] systemd[1]: Proceeding WITHOUT firewalling in effect! (This warning is only shown for the first loaded unit using IP firewalling.)
[    2.590354] systemd[1]: Reached target Timers.
[[0;32m  OK  [0m] Reached target [0;1;39mTimers[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mSlices[0m.
[[0;32m  OK  [0m] Listening on [0;1;39mJournal Socket[0m.
[[0;32m  OK  [0m] Created slice [0;1;39msystem-systemd\x2dfsck.slice[0m.
[[0;32m  OK  [0m] Started [0;1;39mDispatch Password …ts to Console Directory Watch[0m.
[[0;32m  OK  [0m] Listening on [0;1;39mJournal Socket (/dev/log)[0m.
         Starting [0;1;39mApply Kernel Variables[0m...
[[0;32m  OK  [0m] Reached target [0;1;39mSwap[0m.
         Starting [0;1;39mdracut cmdline hook[0m...
[[0;32m  OK  [0m] Listening on [0;1;39mudev Kernel Socket[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mPaths[0m.
[[0;32m  OK  [0m] Listening on [0;1;39mudev Control Socket[0m.
[[0;32m  OK  [0m] Listening on [0;1;39mJournal Audit Socket[0m.
         Starting [0;1;39mJournal Service[0m...
[[0;32m  OK  [0m] Reached target [0;1;39mLocal Encrypted Volumes[0m.
         Starting [0;1;39mCreate list of re…odes for the current kernel[0m...
[[0;32m  OK  [0m] Reached target [0;1;39mSockets[0m.
[[0;32m  OK  [0m] Started [0;1;39mApply Kernel Variables[0m.
[[0;32m  OK  [0m] Started [0;1;39mdracut cmdline hook[0m.
[[0;32m  OK  [0m] Started [0;1;39mCreate list of req… nodes for the current kernel[0m.
         Starting [0;1;39mCreate Static Device Nodes in /dev[0m...
         Starting [0;1;39mdracut pre-udev hook[0m...
[[0;32m  OK  [0m] Started [0;1;39mCreate Static Device Nodes in /dev[0m.
[    2.787964] device-mapper: uevent: version 1.0.3
[    2.790576] device-mapper: ioctl: 4.39.0-ioctl (2018-04-03) initialised: dm-devel@redhat.com
[[0;32m  OK  [0m] Started [0;1;39mdracut pre-udev hook[0m.
         Starting [0;1;39mudev Kernel Device Manager[0m...
[    2.908718] tsc: Refined TSC clocksource calibration: 2300.001 MHz
[    2.912347] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x212735f0517, max_idle_ns: 440795237604 ns
[[0;32m  OK  [0m] Started [0;1;39mJournal Service[0m.
[    2.987129] audit: type=1130 audit(1562165067.134:2): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-journald comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
[    2.451340] systemd-sysctl[152]: Couldn't write 'fq_codel' to 'net/core/default_qdisc', ignoring: No such file or directory
[    2.468985] dracut-cmdline[153]: dracut-dracut-045
[    2.471382] dracut-cmdline[153]: Using kernel command line parameters: rootflags=rw mount.usrflags=ro BOOT_IMAGE=/coreos/vmlinuz-a mount.usr=/dev/mapper/usr verity.usr=PARTUUID=7130c94a-213a-4e5a-8e26-6cce9662f132 rootflags=rw mount.usrflags=ro consoleblank=0 root=LABEL=ROOT console=ttyS0,115200n8 coreos.first_boot=detected coreos.oem.id=ec2 modprobe.blacklist=xen_fbfront net.ifnames=0 nvme_core.io_timeout=4294967295 verity.usrhash=b93b667c140dfcdd681488b18e4e609b3778cc8ea51419c4b7fc04ddd63c0508
[    2.512547] systemd-udevd[241]: Network interface NamePolicy= disabled on kernel command line, ignoring.
[    2.519952] systemd-udevd[241]: Specified group 'render' unknown
[[0;32m  OK  [0m] Started [0;1;39mudev Kernel Device Manager[0m.
[    3.066031] audit: type=1130 audit(1562165067.213:3): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-udevd comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
         [    2.532882] systemd[1]: Started udev Kernel Device Manager.
[    2.549186] systemd[1]: Starting dracut pre-trigger hook...Starting [0;1;39mdracut pre-trigger hook[0m...
         Starting [0;1;39mNetwork Service[0m...

[    2.558141] systemd[1]: Starting Network Service...
[    2.568935] dracut-pre-trigger[250]: rd.md=0: removing MD RAID activation
[[0;32m  OK  [0m] Started [0;1;39mdracut pre-trigger hook[0m.
         Starting [0;1;39mudev [    3.143690] audit: type=1130 audit(1562165067.289:4): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=dracut-pre-trigger comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
Coldplug all Devices[0m...
[    2.604148] systemd[1]: Started dracut pre-trigger hook.
[    2.626832] systemd[1]: Starting udev Coldplug all Devices...
[    2.630159] systemd-udevd[241]: Network interface NamePolicy= disabled on kernel command line, ignoring.
[    2.644165] systemd-udevd[241]: Specified group 'render' unknown
[[0;32m  OK  [0m] Started [0;1;39mudev Coldplug all Devices[0m.
         Starting [0;1;39mdracu[    3.259948] audit: type=1130 audit(1562165067.406:5): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-udev-trigger comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
t initqueue hook[0m...
[    2.719726] systemd[1]: Started udev Coldplug all Devices.
[    2.741422] systemd[1]: Starting dracut initqueue hook...
[[    2.813967] systemd-networkd[256]: Enumeration completed
[    2.814373] systemd[1]: Started Network Service.
[0;32m  OK  [0m] Started [0;1;39mNetwork Service[0m.
[    3.357457] audit: type=1130 audit(1562165067.504:6): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-networkd comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
         Starting [0;1;39mNetwork Name Resolution[0m...
[    2.822971] systemd[1]: Starting Network Name Resolution...
[    3.464319] SCSI subsystem initialized
[    3.480720] cryptd: max_cpu_qlen set to 1000
[    3.511572] AVX2 version of gcm_enc/dec engaged.
[    3.519505] AES CTR mode by8 optimization enabled
[    3.531285] xen_netfront: Initialising Xen virtual ethernet driver
[    3.543191] Invalid max_queues (4), will use default max: 2.
[    3.022789] systemd-udevd[286]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
[    3.032494] systemd-networkd[256]: lo: Configured
[    3.589751] scsi host0: ata_piix
[    3.063693] systemd-resolved[309]: Positive Trust Anchors:
[    3.067192] systemd-resolved[309]: . IN DS 19036 8 2 49aac11d7b6f6446702e54a1607371607a1a41855200fd2ce1cdde32f24e8fb5
[    3.073167] systemd-resolved[309]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d
[    3.078896] systemd-resolved[309]: Negative trust anchors: 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa 23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa 27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa 31.172[    3.635065] scsi host1: ata_piix
.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa corp home internal intranet lan [    3.641276] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc100 irq 14
local private te[    3.647653] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc108 irq 15
st
[    3.121290] systemd-resolved[309]: Defaulting to hostname 'linux'.
[    3.127036] systemd[1]: Started Network Name Resolution.
[    3.169129] systemd-udevd[318]: Using default interface naming scheme 'v238'.
[    3.172516] systemd-udevd[318]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
[[0;32m  OK  [0m] Started [0;1;39mNetwork Name Resolution[0m.
[    3.719377] A link change request failed with some changes committed already. Interface eth0 may have been left with an inconsistent configuration, please check.
[    3.719714] audit: type=1130 audit(1562165067.865:7): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-resolved comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
[    3.757008] blkfront: xvda: barrier or flush: disabled; persistent grants: disabled; indirect descriptors: enabled;
[    3.224295] systemd[1]: Reached target Network.
[    3.230172] systemd-networkd[256]: eth0: Gained carrier
[[0;32m  OK  [0m] Reached target [0;1;39mNetwork[0m.
[    3.237308] systemd-networkd[256]: eth0: Could not bring up interface: Invalid argument
[[    3.240917] systemd[1]: Reached target Host and Network Name Lookups.
[0;32m  OK  [0m] Reached target [0;1;39mHost and Network Name Lookups[0m.
[    3.788481]  xvda: xvda1 xvda2 xvda3 xvda4 xvda6 xvda7 xvda9
[    3.283292] systemd-udevd[311]: Process 'cdrom_id --lock-media /dev/xvda' failed with exit code 1.
[[    3.337207] systemd[1]: Found device /dev/disk/by-label/ROOT.
[0;32m  OK  [0m] Found device [0;1;39m/dev/disk/by-label/ROOT[0m.
[[    3.351581] systemd[1]: Started dracut initqueue hook.
[0;32m  OK  [0m] Started [0;1;39mdracut initqueue hook[0m.
[    3.898745] audit: type=1130 audit(1562165068.046:8): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=dracut-initqueue comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
[    3.370037] systemd[1]: Found device /dev/disk/by-label/OEM.
[[0;32m  OK  [0m] Found device [0;1;39m/dev/disk/by-label/OEM[0m.
[[0;32m  OK  [0m] Found device [0;1;39m/dev/disk/by-…a-213a-4e5a-8e26-6cce9662f132[0m.
[    3.401862] systemd[1]: Found device /dev/disk/by-partuuid/7130c94a-213a-4e5a-8e26-6cce9662f132.
[[0;32m  OK  [0m] Found device [0;1;39m/dev/disk/by-label/EFI-SYSTEM[0m.
[    3.413672]          Starting [0;1;39mIgnition (setup)[0m...
[[0;32m  OK  [0m] Reached target [0;1;39mRemote File Systems (Pre)[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mRemote File Systems[0m.
         Starting [0;1;39[    3.980559] audit: type=1130 audit(1562165068.125:9): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=dracut-pre-mount comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
mdracut pre-mount hook[0m...
[[0;32m  OK  [0m] Reached target [0;1;39mInitrd Root Device[0m.
[[0;32m  OK  [0m] Started [0;1;39mdracut pre-mount hook[0m.
systemd[1]: Found device /dev/disk/by-label/EFI-SYSTEM.
[    3.468238] systemd[1]: Starting Ignition (setup)...
[    3.470609] systemd[1]: Reached target Remote File Systems (Pre).
[    3.473876] systemd[1]: Reached target Remote File Systems.
[    3.476964] systemd[1]: Starting dracut pre-mount hook...
[    3.480240] systemd[1]: Reached target Initrd Root Device.
[    3.483006] systemd[1]: Started dracut pre-mount hook.
[    4.069731] EXT4-fs (xvda6): mounted filesystem with ordered data mode. Opts: (null)
[    3.548933] systemd[1]: mnt-oem.mount: Succeeded.
[[0;32m  OK  [0m] Started [0;1;39mIgnition (setup)[0m.[    3.560943] systemd[1]: ignition-setup.service: Succeeded.
[    3.561987] systemd[1]: Started Ignition (setup).

[    4.102772] audit: type=1130 audit(1562165068.250:10): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=ignition-setup comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
         Starting [0;1;39mIgnition (disks)[0m...
[    3.581664] systemd[1]: Starting Ignition (disks)...
[    3.586116] ignition[395]: Ignition v0.32.0
[    3.916531] ignition[395]: reading system config file "/usr/lib/ignition/base.ign"
[    3.921563] ignition[395]: no config URL provided
[    3.925796] ignition[395]: reading system config file "/usr/lib/ignition/user.ign"
[    3.932807] ignition[395]: no config at "/usr/lib/ignition/user.ign"
[    3.939873] ignition[395]: GET http://169.254.169.254/2009-04-04/user-data: attempt #1
[    3.956480] ignition[395]: GET error: Get http://169.254.169.254/2009-04-04/user-data: dial tcp 169.254.169.254:80: connect: network is unreachable
[    4.117515] ignition[395]: GET http://169.254.169.254/2009-04-04/user-data: attempt #2
[    4.121919] ignition[395]: GET error: Get http://169.254.169.254/2009-04-04/user-data: dial tcp 169.254.169.254:80: connect: network is unreachable
[    4.517930] ignition[395]: GET http://169.254.169.254/2009-04-04/user-data: attempt #3
[    4.524727] ignition[395]: GET error: Get http://169.254.169.254/2009-04-04/user-data: dial tcp 169.254.169.254:80: connect: network is unreachable
[    4.741990] systemd-networkd[256]: eth0: Gained IPv6LL
[    5.318824] ignition[395]: GET http://169.254.169.254/2009-04-04/user-data: attempt #4
[    5.325110] ignition[395]: GET error: Get http://169.254.169.254/2009-04-04/user-data: dial tcp 169.254.169.254:80: connect: network is unreachable
[    6.919257] ignition[395]: GET http://169.254.169.254/2009-04-04/user-data: attempt #5
[    6.927294] ignition[395]: GET error: Get http://169.254.169.254/2009-04-04/user-data: dial tcp 169.254.169.254:80: connect: network is unreachable
[[0m[0;31m*     [0m] (1 of 2) A start job is running for Ignition (disks) (6s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (1 of 2) A start job is running for Ignition (disks) (7s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for Ignition (disks) (7s / no limit)
[   10.119710] ignition[395]: GET http://169.254.169.254/2009-04-04/user-data: attempt #6
[   10.126331] ignition[395]: GET error: Get http://169.254.169.254/2009-04-04/user-data: dial tcp 169.254.169.254:80: connect: network is unreachable
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for /dev/mapper/usr (8s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (2 of 2) A start job is running for /dev/mapper/usr (8s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for /dev/mapper/usr (9s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for Ignition (disks) (9s / no limit)
M
[K[     [0;31m*[0m] (1 of 2) A start job is running for Ignition (disks) (10s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for Ignition (disks) (10s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for /dev/mapper/usr (11s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (2 of 2) A start job is running for /dev/mapper/usr (11s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for /dev/mapper/usr (12s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for Ignition (disks) (12s / no limit)
[   15.120114] ignition[395]: GET http://169.254.169.254/2009-04-04/user-data: attempt #7
[   15.135479] ignition[395]: GET error: Get http://169.254.169.254/2009-04-04/user-data: dial tcp 169.254.169.254:80: connect: network is unreachable
M
[K[[0;1;31m*[0m[0;31m*    [0m] (1 of 2) A start job is running for Ignition (disks) (13s / no limit)
M
[K[[0m[0;31m*     [0m] (1 of 2) A start job is running for Ignition (disks) (13s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for /dev/mapper/usr (14s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (2 of 2) A start job is running for /dev/mapper/usr (14s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for /dev/mapper/usr (15s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for Ignition (disks) (15s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (1 of 2) A start job is running for Ignition (disks) (16s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for Ignition (disks) (16s / no limit)
M
[K[     [0;31m*[0m] (2 of 2) A start job is running for /dev/mapper/usr (17s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (2 of 2) A start job is running for /dev/mapper/usr (17s / no limit)
[   20.120516] ignition[395]: GET http://169.254.169.254/2009-04-04/user-data: attempt #8
[   20.133674] ignition[395]: GET error: Get http://169.254.169.254/2009-04-04/user-data: dial tcp 169.254.169.254:80: connect: network is unreachable
[   20.188175] systemd-networkd[256]: eth0: DHCPv4 address 10.14.27.92/24 via 10.14.27.1
[   20.205312] systemd-networkd[256]: eth0: IPv6 successfully enabled
[   20.211734] systemd-networkd[256]: eth0: Configured
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for /dev/mapper/usr (18s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for Ignition (disks) (18s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (1 of 2) A start job is running for Ignition (disks) (19s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for Ignition (disks) (19s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for /dev/mapper/usr (20s / no limit)
M
[K[[0m[0;31m*     [0m] (2 of 2) A start job is running for /dev/mapper/usr (20s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for /dev/mapper/usr (21s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for Ignition (disks) (21s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (1 of 2) A start job is running for Ignition (disks) (22s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for Ignition (disks) (22s / no limit)
[   25.121134] ignition[395]: GET http://169.254.169.254/2009-04-04/user-data: attempt #9
[   25.130785] ignition[395]: GET result: OK
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for /dev/mapper/usr (23s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (2 of 2) A start job is running for /dev/mapper/usr (23s / no limit)
M
[K[     [0;31m*[0m] (2 of 2) A start job is running for /dev/mapper/usr (24s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for Ignition (disks) (24s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (1 of 2) A start job is running for Ignition (disks) (25s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for Ignition (disks) (25s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for /dev/mapper/usr (26s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (2 of 2) A start job is running for /dev/mapper/usr (26s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for /dev/mapper/usr (27s / no limit)
M
[K[[0m[0;31m*     [0m] (1 of 2) A start job is running for Ignition (disks) (27s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (1 of 2) A start job is running for Ignition (disks) (28s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for Ignition (disks) (28s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for /dev/mapper/usr (29s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (2 of 2) A start job is running for /dev/mapper/usr (29s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for /dev/mapper/usr (30s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for Ignition (disks) (30s / no limit)
M
[K[     [0;31m*[0m] (1 of 2) A start job is running for Ignition (disks) (31s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for Ignition (disks) (31s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for /dev/mapper/usr (32s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (2 of 2) A start job is running for /dev/mapper/usr (32s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for /dev/mapper/usr (33s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for Ignition (disks) (33s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (1 of 2) A start job is running for Ignition (disks) (34s / no limit)
M
[K[[0m[0;31m*     [0m] (1 of 2) A start job is running for Ignition (disks) (34s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for /dev/mapper/usr (35s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (2 of 2) A start job is running for /dev/mapper/usr (35s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for /dev/mapper/usr (36s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for Ignition (disks) (36s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (1 of 2) A start job is running for Ignition (disks) (37s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for Ignition (disks) (37s / no limit)
M
[K[     [0;31m*[0m] (2 of 2) A start job is running for /dev/mapper/usr (38s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (2 of 2) A start job is running for /dev/mapper/usr (38s / no limit)
[   41.377720] random: crng init done
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for /dev/mapper/usr (39s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for Ignition (disks) (39s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (1 of 2) A start job is running for Ignition (disks) (40s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for Ignition (disks) (40s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for /dev/mapper/usr (41s / no limit)
M
[K[[0m[0;31m*     [0m] (2 of 2) A start job is running for /dev/mapper/usr (41s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for /dev/mapper/usr (42s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for Ignition (disks) (42s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (1 of 2) A start job is running for Ignition (disks) (43s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for Ignition (disks) (43s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for /dev/mapper/usr (44s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (2 of 2) A start job is running for /dev/mapper/usr (44s / no limit)
M
[K[     [0;31m*[0m] (2 of 2) A start job is running for /dev/mapper/usr (45s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for Ignition (disks) (45s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (1 of 2) A start job is running for Ignition (disks) (46s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for Ignition (disks) (46s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for /dev/mapper/usr (47s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (2 of 2) A start job is running for /dev/mapper/usr (47s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for /dev/mapper/usr (48s / no limit)
M
[K[[0m[0;31m*     [0m] (1 of 2) A start job is running for Ignition (disks) (48s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (1 of 2) A start job is running for Ignition (disks) (49s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for Ignition (disks) (49s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for /dev/mapper/usr (50s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (2 of 2) A start job is running for /dev/mapper/usr (50s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for /dev/mapper/usr (51s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for Ignition (disks) (51s / no limit)
M
[K[     [0;31m*[0m] (1 of 2) A start job is running for Ignition (disks) (52s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for Ignition (disks) (52s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for /dev/mapper/usr (53s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (2 of 2) A start job is running for /dev/mapper/usr (53s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for /dev/mapper/usr (54s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for Ignition (disks) (54s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (1 of 2) A start job is running for Ignition (disks) (55s / no limit)
M
[K[[0m[0;31m*     [0m] (1 of 2) A start job is running for Ignition (disks) (55s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for /dev/mapper/usr (56s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (2 of 2) A start job is running for /dev/mapper/usr (56s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for /dev/mapper/usr (57s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for Ignition (disks) (57s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (1 of 2) A start job is running for Ignition (disks) (58s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for Ignition (disks) (58s / no limit)
M
[K[     [0;31m*[0m] (2 of 2) A start job is running for /dev/mapper/usr (59s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (2 of 2) A start job is running for /dev/mapper/usr (59s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for /dev/mapper/usr (1min / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for Ignition (disks) (1min / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (1 of 2) A start job is running for…ition (disks) (1min 1s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for…ition (disks) (1min 1s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for…ev/mapper/usr (1min 2s / no limit)
M
[K[[0m[0;31m*     [0m] (2 of 2) A start job is running for…ev/mapper/usr (1min 2s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for…ev/mapper/usr (1min 3s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for…ition (disks) (1min 3s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (1 of 2) A start job is running for…ition (disks) (1min 4s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for…ition (disks) (1min 4s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for…ev/mapper/usr (1min 5s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (2 of 2) A start job is running for…ev/mapper/usr (1min 5s / no limit)
M
[K[     [0;31m*[0m] (2 of 2) A start job is running for…ev/mapper/usr (1min 6s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for…ition (disks) (1min 6s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (1 of 2) A start job is running for…ition (disks) (1min 7s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for…ition (disks) (1min 7s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for…ev/mapper/usr (1min 8s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (2 of 2) A start job is running for…ev/mapper/usr (1min 8s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for…ev/mapper/usr (1min 9s / no limit)
M
[K[[0m[0;31m*     [0m] (1 of 2) A start job is running for…ition (disks) (1min 9s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (1 of 2) A start job is running for…tion (disks) (1min 10s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for…tion (disks) (1min 10s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 11s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 11s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for…v/mapper/usr (1min 12s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for…tion (disks) (1min 12s / no limit)
M
[K[     [0;31m*[0m] (1 of 2) A start job is running for…tion (disks) (1min 13s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for…tion (disks) (1min 13s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for…v/mapper/usr (1min 14s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 14s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 15s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for…tion (disks) (1min 15s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (1 of 2) A start job is running for…tion (disks) (1min 16s / no limit)
M
[K[[0m[0;31m*     [0m] (1 of 2) A start job is running for…tion (disks) (1min 16s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 17s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 17s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 18s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for…tion (disks) (1min 18s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (1 of 2) A start job is running for…tion (disks) (1min 19s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for…tion (disks) (1min 19s / no limit)
M
[K[     [0;31m*[0m] (2 of 2) A start job is running for…v/mapper/usr (1min 20s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (2 of 2) A start job is running for…v/mapper/usr (1min 20s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for…v/mapper/usr (1min 21s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for…tion (disks) (1min 21s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (1 of 2) A start job is running for…tion (disks) (1min 22s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for…tion (disks) (1min 22s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 23s / no limit)
M
[K[[0m[0;31m*     [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 23s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 24s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for…tion (disks) (1min 24s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (1 of 2) A start job is running for…tion (disks) (1min 25s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for…tion (disks) (1min 25s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for…v/mapper/usr (1min 26s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (2 of 2) A start job is running for…v/mapper/usr (1min 26s / no limit)
M
[K[     [0;31m*[0m] (2 of 2) A start job is running for…v/mapper/usr (1min 27s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for…tion (disks) (1min 27s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (1 of 2) A start job is running for…tion (disks) (1min 28s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for…tion (disks) (1min 28s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 29s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 29s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 30s / no limit)
M
[K[[0m[0;31m*     [0m] (1 of 2) A start job is running for…tion (disks) (1min 30s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (1 of 2) A start job is running for…tion (disks) (1min 31s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for…tion (disks) (1min 31s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 32s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 32s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for…v/mapper/usr (1min 33s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for…tion (disks) (1min 33s / no limit)
M
[K[     [0;31m*[0m] (1 of 2) A start job is running for…tion (disks) (1min 34s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for…tion (disks) (1min 34s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for…v/mapper/usr (1min 35s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 35s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 36s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for…tion (disks) (1min 36s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (1 of 2) A start job is running for…tion (disks) (1min 37s / no limit)
M
[K[[0m[0;31m*     [0m] (1 of 2) A start job is running for…tion (disks) (1min 37s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 38s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 38s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 39s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for…tion (disks) (1min 39s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (1 of 2) A start job is running for…tion (disks) (1min 40s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for…tion (disks) (1min 40s / no limit)
M
[K[     [0;31m*[0m] (2 of 2) A start job is running for…v/mapper/usr (1min 41s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (2 of 2) A start job is running for…v/mapper/usr (1min 41s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for…v/mapper/usr (1min 42s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for…tion (disks) (1min 42s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (1 of 2) A start job is running for…tion (disks) (1min 43s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for…tion (disks) (1min 43s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 44s / no limit)
M
[K[[0m[0;31m*     [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 44s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 45s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for…tion (disks) (1min 45s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (1 of 2) A start job is running for…tion (disks) (1min 46s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for…tion (disks) (1min 46s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for…v/mapper/usr (1min 47s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (2 of 2) A start job is running for…v/mapper/usr (1min 47s / no limit)
M
[K[     [0;31m*[0m] (2 of 2) A start job is running for…v/mapper/usr (1min 48s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for…tion (disks) (1min 48s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (1 of 2) A start job is running for…tion (disks) (1min 49s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for…tion (disks) (1min 49s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 50s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 50s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 51s / no limit)
M
[K[[0m[0;31m*     [0m] (1 of 2) A start job is running for…tion (disks) (1min 51s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (1 of 2) A start job is running for…tion (disks) (1min 52s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for…tion (disks) (1min 52s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 53s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 53s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for…v/mapper/usr (1min 54s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for…tion (disks) (1min 54s / no limit)
M
[K[     [0;31m*[0m] (1 of 2) A start job is running for…tion (disks) (1min 55s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for…tion (disks) (1min 55s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for…v/mapper/usr (1min 56s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 56s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 57s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for…tion (disks) (1min 57s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (1 of 2) A start job is running for…tion (disks) (1min 58s / no limit)
M
[K[[0m[0;31m*     [0m] (1 of 2) A start job is running for…tion (disks) (1min 58s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 59s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (2 of 2) A start job is running for…v/mapper/usr (1min 59s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for /dev/mapper/usr (2min / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for Ignition (disks) (2min / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (1 of 2) A start job is running for…ition (disks) (2min 1s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for…ition (disks) (2min 1s / no limit)
M
[K[     [0;31m*[0m] (2 of 2) A start job is running for…ev/mapper/usr (2min 2s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (2 of 2) A start job is running for…ev/mapper/usr (2min 2s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for…ev/mapper/usr (2min 3s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for…ition (disks) (2min 3s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (1 of 2) A start job is running for…ition (disks) (2min 4s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for…ition (disks) (2min 4s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for…ev/mapper/usr (2min 5s / no limit)
M
[K[[0m[0;31m*     [0m] (2 of 2) A start job is running for…ev/mapper/usr (2min 5s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for…ev/mapper/usr (2min 6s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for…ition (disks) (2min 6s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (1 of 2) A start job is running for…ition (disks) (2min 7s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for…ition (disks) (2min 7s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for…ev/mapper/usr (2min 8s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (2 of 2) A start job is running for…ev/mapper/usr (2min 8s / no limit)
M
[K[     [0;31m*[0m] (2 of 2) A start job is running for…ev/mapper/usr (2min 9s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for…ition (disks) (2min 9s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (1 of 2) A start job is running for…tion (disks) (2min 10s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for…tion (disks) (2min 10s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for…v/mapper/usr (2min 11s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (2 of 2) A start job is running for…v/mapper/usr (2min 11s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for…v/mapper/usr (2min 12s / no limit)
M
[K[[0m[0;31m*     [0m] (1 of 2) A start job is running for…tion (disks) (2min 12s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (1 of 2) A start job is running for…tion (disks) (2min 13s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for…tion (disks) (2min 13s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for…v/mapper/usr (2min 14s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (2 of 2) A start job is running for…v/mapper/usr (2min 14s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for…v/mapper/usr (2min 15s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for…tion (disks) (2min 15s / no limit)
M
[K[     [0;31m*[0m] (1 of 2) A start job is running for…tion (disks) (2min 16s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for…tion (disks) (2min 16s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for…v/mapper/usr (2min 17s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (2 of 2) A start job is running for…v/mapper/usr (2min 17s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for…v/mapper/usr (2min 18s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for…tion (disks) (2min 18s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (1 of 2) A start job is running for…tion (disks) (2min 19s / no limit)
M
[K[[0m[0;31m*     [0m] (1 of 2) A start job is running for…tion (disks) (2min 19s / no limit)
M
[K[[0;1;31m*[0m[0;31m*    [0m] (2 of 2) A start job is running for…v/mapper/usr (2min 20s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (2 of 2) A start job is running for…v/mapper/usr (2min 20s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for…v/mapper/usr (2min 21s / no limit)
M
[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for…tion (disks) (2min 21s / no limit)
M
[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (1 of 2) A start job is running for…tion (disks) (2min 22s / no limit)
M
[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for…tion (disks) (2min 22s / no limit)
M
[K[     [0;31m*[0m] (2 of 2) A start job is running for…v/mapper/usr (2min 23s / no limit)
[  145.459009] ignition[395]: failed to fetch config: RequestError: send request failed
caused by: dial tcp 52.216.109.237:443: i/o timeout
[  145.468215] ignition[395]: failed to acquire config: RequestError: send request failed
caused by: dial tcp 52.216.109.237:443: i/o timeout
M
[K[[0;1;31mFAILED[0m] Failed to start [0;1;39mIgnition (disks)[0m.
[KSee 'systemctl status ignition-disks.service' for details.
[[0;1;33mDEPEND[0m] Dependency failed for [0;1;39mPopu…orcx store to satisfy profile[0m.
[[0;1;33mDEPEND[0m] Dependency failed for [0;1;39mInitrd Default Target[0m.
[[0;1;33mDEPEND[0m] Dependency failed for [0;1;39mIgnition (files)[0m.
[  146.037548] kauditd_printk_skb: 1 callbacks suppressed
[  146.037549] audit: type=1130 audit(1562165210.185:12): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=ignition-disks comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=failed'
[  145.501931] [[0;32m  OK  [0m] Reached target [0;1;39mLocal File Systems (Pre)[0m.
         Starting [0[  146.071730] audit: type=1131 audit(1562165210.214:13): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=dracut-pre-mount comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
;1;39mVerity Setup for /dev/mapper/usr[0m...
         Starting [0;1;39mSetup Virtual Console[0m...
         Stopping [0;1;39mNetwork Service[0m...
[[0;32m  OK  [0m] Stopped [0;1;39mdracut pre-mount hook[0m.
         Starting [0;1;39mFile System Check on /dev/disk/by-label/ROOT[0m...
[[0;32m  OK  [0m] Stopped [0;1;39mdracut initqueue hook[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mdracut pre-trigger hook[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mdracut pre-udev hook[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mdracut cmdline hook[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mDispatch Password …ts to Console Directory Watch[0m.
[  146.166917] audit: type=1131 audit(1562165210.271:14): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=dracut-initqueue comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
[  146.166920] audit: type=1131 audit(1562165210.271:15): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=dracut-pre-trigger comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
[  146.166922] audit: type=1131 audit(1562165210.282:16): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=dracut-pre-udev comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
[  146.166924] audit: type=1131 audit(1562165210.282:17): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=dracut-cmdline comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
systemd[1]: ignition-disks.service: Main process exited, code=exited, status=1/FAILURE
[[0;32m  OK  [0m[  145.753497] ignition[395]: Ignition failed: RequestError: send request failed
caused by: dial tcp 52.216.109.237:443: i/o timeout
[  145.753906] systemd[1]: ignition-disks.service: Failed with result 'exit-code'.
[  145.755125] systemd[1]: Failed to start Ignition (disks).
[  145.755923] systemd-fsck[407]: ROOT: clean, 23902/1498496 files, 365971/1489915 blocks
[  145.756541] systemd[1]: Dependency failed for Populate torcx store to satisfy profile.
[  145.757404] systemd[1]: Dependency failed for Initrd Default Target.
[  145.758076] systemd[1]: initrd.target: Job initrd.target/start failed with result 'dependency'.
[  145.758663] systemd[1]: initrd.target: Triggering OnFailure= dependencies.
[  145.759427] systemd[1]: torcx-profile-populate.service: Job torcx-profile-populate.service/start failed with result 'dependency'.
[  145.761026] systemd[1]: Dependency failed for Ignition (files).
[  145.761769] systemd[1]: ignition-files.service: Job ignition-files.service/start failed with result 'dependency'.
[  145.803538] systemd[1]: Reached target Local File Systems (Pre).
[  145.804499] systemd[1]: Starting Verity Setup for /dev/mapper/usr...
[  145.805522] systemd[1]: Starting Setup Virtual Console...
[  145.806647] systemd[1]: Stopping Network Service...
[  145.807609] systemd[1]: dracut-pre-mount.service: Succeeded.
[  145.808639] systemd[1]: Stopped dracut pre-mount hook.
[  145.809698] systemd[1]: Starting File System Check on /dev/disk/by-label/ROOT...
[  145.810659] systemd[1]: dracut-initqueue.service: Succeeded.
[  145.811593] systemd[1]: Stopped dracut initqueue hook.
[  145.812533] systemd[1]: dracut-pre-trigger.service: Succeeded.
[  145.813443] systemd[1]: Stopped dracut pre-trigger hook.
[  145.814389] systemd[1]: dracut-pre-udev.service: Succeeded.
[  145.815589] systemd[1]: Stopped dracut pre-udev hook.
[  145.816692] systemd[1]: dracut-cmdline.service: Succeeded.
[  145.817735] systemd[1]: Stopped dracut cmdline hook.
[  145.818778] systemd[1]: systemd-ask-password-console.path: Succeeded.
[  145.819886] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.
[  145.820883] systemd[1]: Started File System Check on /dev/disk/by-label/ROOT.
] Started [0;1;39mFile System Check on /dev/disk/by-label/ROOT[0m.
[  146.431801] audit: type=1130 audit(1562165210.579:18): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-fsck-root comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
[[0;32m  OK  [0m] Started [0;1;39mSetup Virtual Console[0m.
         Startin[  146.470932] audit: type=1130 audit(1562165210.616:19): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-vconsole-setup comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
g [0;1;39mEmergency Shell[0m..[  146.499018] audit: type=1131 audit(1562165210.616:20): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-vconsole-setup comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
.
         Mounting [0;1;39m/sysroot[0m...
[  145.921840] systemd[1]: systemd-vconsole-setup.service: Succeeded.
[  145.997564] systemd[1]: Started Setup Virtual Console.
[  146.001182] systemd[1]: Starting Emergency Shell...
[  146.005532] systemd[1]: Mounting /sysroot...
[[0;32m  OK  [0m] Started [0;1;39mVerity Setup for /dev/mapper/usr[0m.
[  1[  146.563513] audit: type=1130 audit(1562165210.707:21): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=verity-setup comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
46.024491] systemd[1]: Started Verity Setup for /dev/mapper/usr.
[  146.062134] systemd[1]: Found device /dev/mapper/usr.
[[0;32m  OK  [0m] Found device [0;1;39m/dev/mapper/usr[0m.
[  146.063588] systemd[1]: Condition check resulted in File System Check on /dev/mapper/usr being skipped.
[[0;32m  OK  [0m] Mounted [0;1;39m/sysroot[0m.
[  146.085612] systemd[1]: Mounted /sysroot.
         Mounting [0;1;39m/sysroot/usr[0m...
[[0;32m  OK  [0m] Reached target [0;1;39mInitrd Root File System[0m.
[  146.093947] systemd[1]: Mounting /sysroot/usr...
[  146.094359] systemd[1]: Reached target Initrd Root File System.
[[0;32m  OK  [0m] Mounted [0;1;39m/sysroot/usr[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mLocal File Systems[0m.
         Starting [0;1;39mRoot filesystem setup[0m...
[  146.127959] systemd[1]: Mounted /sysroot/usr.
[  146.128742] systemd[1]: Reached target Local File Systems.
[  146.129358] systemd[1]: Starting Root filesystem setup...
[[0;32m  OK  [0m] Started [0;1;39mRoot filesystem setup[0m.
         Starting [0;1;39mReload Configuration from the Real Root[0m...
[  146.206303] systemd[1]: initrd-setup-root.service: Succeeded.
[  146.207967] systemd[1]: Started Root filesystem setup.
[  146.208913] systemd[1]: Starting Reload Configuration from the Real Root...
[  146.214208] systemd[1]: Reloading.
[  146.313791] systemd-networkd[256]: lo: Lost carrier
[  146.322029] systemd-networkd[256]: eth0: Lost carrier
[  146.322259] systemd-networkd[256]: eth0: DHCP lease lost
[  146.330134] systemd-networkd[256]: eth0: IPv6 successfully enabled
[[0;32m  OK  [0m] Stopped [0;1;39mNetwork Service[0m.[  146.352017] systemd[1]: systemd-networkd.service: Succeeded.
[  146.357729] systemd[1]: Stopped Network Service.

[  146.380798] systemd[1]: initrd-parse-etc.service: Succeeded.
[  146.381449] systemd[1]: Started Reload Configuration from the Real Root.
[[0;32m  OK  [0m] Started [0;1;39mReload Configuration from the Real Root[0m.
[  146.385352] systemd[1]: Reached target Initrd File Systems.
[[0;32m  OK  [0m] Reached target [0;1;39mInitrd File Systems[0m.
[  146.388543] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...
         Starting [0;1;39mCleaning Up and Shutting Down Daemons[0m...
[  146.396087] systemd[1]: Requested transaction contradicts existing jobs: Transaction for initrd-switch-root.target/start is destructive (emergency.target has 'start' job queued, but 'stop' is included in transaction).
[  146.397820] systemctl[557]: Failed to start initrd-switch-root.target: Transaction for initrd-switch-root.target/start is destructive (emergency.target has 'start' job queued, but 'stop' is included in transaction).
[  146.398138] systemctl[557]: See system logs and 'systemctl status initrd-switch-root.target' for details.
[[0;1;31mFAILED[0m] Failed to start [0;1;39mCleaning Up and Shutting Down Daemons[0m.
See 'systemctl status initrd-cleanup.service' for details.
[  146.400933] systemd[1]: initrd-cleanup.service: Main process exited, code=exited, status=4/NOPERMISSION
[  146.401364] systemd[1]: initrd-cleanup.service: Failed with result 'exit-code'.
[  146.402410] systemd[1]: Failed to start Cleaning Up and Shutting Down Daemons.
[  146.403406] systemd[1]: initrd-cleanup.service: Triggering OnFailure= dependencies.
         Starting [0;1;39mSetup Virtual Console[0m...
[  146.412949] systemd[1]: Starting Setup Virtual Console...
[[0;32m  OK  [0m] Started [0;1;39mSetup Virtual Console[0m.
[  146.456206] systemd[1]: systemd-vconsole-setup.service: Succeeded.
[  146.457013] systemd[1]: Started Setup Virtual Console.
cat: unrecognized option '-------------------------------------------------------------------------------'
Try 'cat --help' for more information.
[0;1;31m[0mIgnition v0.32.0
[0;1;31m[0mreading system config file "/usr/lib/ignition/base.ign"
[0;1;31m[0mparsing config with SHA512: 865c03baa79b8c74023d13a0b3666474fa06a165421a1e05731b76e0f557d42c5c89d4870a0b9c4182ad7d4d8209de20dca9c9da63d637e0410fbd60314cac6c
[0;1;31m[0mparsed url from cmdline: ""
[0;1;31m[0mno config URL provided
[0;1;31m[0mreading system config file "/usr/lib/ignition/user.ign"
[0;1;31m[0mno config at "/usr/lib/ignition/user.ign"
[0;1;31m[0mGET http://169.254.169.254/2009-04-04/user-data: attempt #1
[0;1;31m[0mGET error: Get http://169.254.169.254/2009-04-04/user-data: dial tcp 169.254.169.254:80: connect: network is unreachable
[0;1;31m[0mGET http://169.254.169.254/2009-04-04/user-data: attempt #2
[0;1;31m[0mGET error: Get http://169.254.169.254/2009-04-04/user-data: dial tcp 169.254.169.254:80: connect: network is unreachable
[0;1;31m[0mGET http://169.254.169.254/2009-04-04/user-data: attempt #3
[0;1;31m[0mGET error: Get http://169.254.169.254/2009-04-04/user-data: dial tcp 169.254.169.254:80: connect: network is unreachable
[0;1;31m[0mGET http://169.254.169.254/2009-04-04/user-data: attempt #4
[0;1;31m[0mGET error: Get http://169.254.169.254/2009-04-04/user-data: dial tcp 169.254.169.254:80: connect: network is unreachable
[0;1;31m[0mGET http://169.254.169.254/2009-04-04/user-data: attempt #5
[0;1;31m[0mGET error: Get http://169.254.169.254/2009-04-04/user-data: dial tcp 169.254.169.254:80: connect: network is unreachable
[0;1;31m[0mGET http://169.254.169.254/2009-04-04/user-data: attempt #6
[0;1;31m[0mGET error: Get http://169.254.169.254/2009-04-04/user-data: dial tcp 169.254.169.254:80: connect: network is unreachable
[0;1;31m[0mGET http://169.254.169.254/2009-04-04/user-data: attempt #7
[0;1;31m[0mGET error: Get http://169.254.169.254/2009-04-04/user-data: dial tcp 169.254.169.254:80: connect: network is unreachable
[0;1;31m[0mGET http://169.254.169.254/2009-04-04/user-data: attempt #8
[0;1;31m[0mGET error: Get http://169.254.169.254/2009-04-04/user-data: dial tcp 169.254.169.254:80: connect: network is unreachable
[0;1;31m[0mGET http://169.254.169.254/2009-04-04/user-data: attempt #9
[0;1;31m[0mGET result: OK
[0;1;31m[0mparsing config with SHA512: dfb2f1a0fdefda476574aa21f0f3a9ee05b289cbf99ed50680c914d1722e36db3058f09838c335bc0c154985743717b75ca319973c309951c2db1d64128013db
[0;1;31m[0mfailed to fetch config: RequestError: send request failed
caused by: dial tcp 52.216.109.237:443: i/o timeout
[0;1;31m[0mfailed to acquire config: RequestError: send request failed
caused by: dial tcp 52.216.109.237:443: i/o timeout
[0;1;31m[0mIgnition failed: RequestError: send request failed
caused by: dial tcp 52.216.109.237:443: i/o timeout
Press Enter for emergency shell or wait 5 minutes for reboot.                 
Press Enter for emergency shell or wait 4 minutes 45 seconds for reboot.      ```

@ajeddeloh ajeddeloh added the kind/bug label Jul 15, 2019

@ajeddeloh

This comment has been minimized.

Copy link
Contributor

commented Jul 15, 2019

Thanks for the report, I'll take a look at this. Sorry for the slow response.

@wernerb

This comment has been minimized.

Copy link
Author

commented Jul 23, 2019

@ajeddeloh thank you for the fix! Is there a way for me to test it in an coreos ami? or is a release for a new ignition version not far away? :)

@ajeddeloh

This comment has been minimized.

Copy link
Contributor

commented Jul 23, 2019

It should be in alpha next week.

ajeddeloh added a commit to ajeddeloh/coreos-overlay that referenced this issue Jul 23, 2019

sys-apps/ignition: patch ignition for ignition/826
Fix issue where Ignition tries to get the AWS region too early and
fails, defaulting to us-east-1.

ref: coreos/ignition#826
@ajeddeloh

This comment has been minimized.

Copy link
Contributor

commented Jul 30, 2019

Forgot to close this. Reopen if this week's alpha does not fix it.

@ajeddeloh ajeddeloh closed this Jul 30, 2019

@wernerb

This comment has been minimized.

Copy link
Author

commented Aug 12, 2019

I have not seen this bug in the alpha for 30 instance launches, it seems to be working. 👍 Thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.