Roll back an update and use of /proc/cmdline

[iaguis]

We're trying to implement a roll back scheme so if you deploy a bogus tree you roll back to the previous one.

To do that we create a dracut service that checks if the last boot succeeded and if it didn't, it swaps the bootloader configuration.

At the time we roll back, since we're already booted, /proc/cmdline will have the old tree and ostree-prepare-root will use that causing the bogus tree to boot again. To fix it we bind-mount a file with the new ostree= value on /proc/cmdline.

Since this is pretty hacky, I wanted to start a discussion on how to do it in a cleaner way, passing the information of the tree to boot in another way. In addition, note that ostree admin also relies on the information in /proc/cmdline.

In the future, if we move this logic to the bootloader (instead of doing it in the initramfs) we won't have this problem but we're not quite there yet.

[cgwalters]

We should include ostree admin rollback in ostree as a demo command I think. There's an implementation in rpm-ostree: https://github.com/projectatomic/rpm-ostree/blob/master/src/daemon/rpmostreed-transaction-types.c#L308

I guess your last point is important - you're trying to do a rollback from inside the initramfs? And crucially how do you detect "bogus tree"?

[iaguis]

I guess your last point is important - you're trying to do a rollback from inside the initramfs? And crucially how do you detect "bogus tree"?

Yes. The idea is to have a "boot-flags" file that records if the last boot of a tree is successful or not and the tries left for it.

When doing a deployment, we set successful=0 and tries_left > 0. Then, on reboot, a dracut service checks these flags to decide whether to just boot (if successful=1), boot and decrease tries_left (if successful=0 and tries_left > 0) or to roll back (if successful=0 and tries_left=0).

We mark a tree as successful when the user is able to log in to a gnome session.

[cgwalters]

It seems like the simplest might be to not try to change the bootloader configuration in the initramfs, but to teach ostree-switch-root how to parse the /boot/loader config. Basically when booting, if a service fails, then do: touch /ostree/deploy/$os/deploy/$x.broken
Then in the initramfs, look for the .broken file, and if it's found, look for the next deployment in the list.

The slightly ugly part is that we probably don't want to drag libostree into the initramfs...but the alternative is duplicating the bootloader parsing code.

[cgwalters]

See also coreos/rpm-ostree#177