Multiple runs of igniton

[gappan]

Issue Report

Bug

Container Linux Version

$ cat /etc/os-release
NAME="Container Linux by CoreOS"
ID=coreos
VERSION=1353.8.0
VERSION_ID=1353.8.0
BUILD_ID=2017-05-30-2322
PRETTY_NAME="Container Linux by CoreOS 1353.8.0 (Ladybug)"
ANSI_COLOR="38;5;75"
HOME_URL="https://coreos.com/"
BUG_REPORT_URL="https://issues.coreos.com"

Environment

What hardware/cloud provider/hypervisor is being used to run Container Linux?
AWS - Xen

We tried to have a sane default image of coreos(with certain settings like ntp configured using ignition) and use that as our base image for our applications.
Came across this issue #1291 which mentions that deleting /etc/machine-id will trigger systemd first boot action. Hence we deleted the /etc/machine-id and saved the image.
We planed to use the above saved base core os image and on top of that we wanted to execute its own specific ignition configs but the coreos-cloudinit never ran the ignition files and it was throwing Detected an Ignition config. Exiting....
Is it not possible to run ignition twice?

[crawford]

Ignition is designed to only run once. We wanted to make it very clear that this is a provisioning tool and not a general configuration management tool.

Your use case however, seems pretty reasonable. In order to trick Ignition into running a second time, you'll want to touch /boot/coreos/first_boot in addition to removing /etc/machine-id. Ignition will run if that file is present. Once Ignition successfully completes, it will remove the flag.

[euank]

Can you detail what you mean by "sane default image of coreos"?

Perhaps we should apply those changes to the distro so you won't have to do whatever modifications these are.

[gappan]

@crawford tried touching the file /boot/coreos/first_bootwith -rwxr-xr-x. 1 root root . Ignition did not trigger and also this file still existed after the boot. Also as earlier removed the /etc/machine-id as well. Anything I am missing here?

@euank: It is more specific to our org. Eg: configuring the ntp server in timesyncd. Reason being, AWS user-data file has a size limit. Hence we wanted to have a base image which could have all the general configurations required for our apps. And create our apps on top of that base image and use ignition for provisioning our apps.

[bgilbert]

The /boot/coreos/first_boot trick only works with images initially provisioned with Container Linux 1367.5.0 or above. On older systems you can get the same result by running sgdisk -U 00000000-0000-0000-0000-000000000001 /dev/xvda (and also removing /etc/machine-id), but please consider updating to a current Container Linux release instead.

Ignition's config append and replace facilities are intended to address exactly this kind of size limit. There are a couple possibilities:

1. Upload your entire Ignition config to S3, and pass a stub Ignition config in userdata that uses ignition.config.replace.
2. Continue to pass your application configs in userdata, and use ignition.config.append to include your base Ignition config from a well-known URL.

[gappan]

Updated my Container Linux release and it worked ! Thanks a bunch for the replies.

[seh]

After removing the first_boot file and rebooting, Ignition runs again, but it seems that systemd units enabled on this second boot don’t get started.

The documentation here mentions the special enabling behavior on first boot. Is deleting that aforementioned file and tricking Ignition into running again not enough to summon this “first boot” behavior again?

[cgonyeo]

@seh as mentioned in the docs you link, you'll need to invoke systemctl preset-all to cause systemd presets to be reevaluated, which will probably enable/disable your units. This will reset the enabled status to the default for all units on the system though, so you can also run systemctl preset <unit-name> to have the presets for a specific unit evaluated.