deploy: Add support for providing config overlay for /etc

[cgwalters]

Related:

• deploy: Add --no-merge #2081
• https://mail.gnome.org/archives/ostree-list/2020-February/msg00000.html
• consider making config changes truly transactional on RHCOS openshift/machine-config-operator#1190

And many more.

Basically let's support update systems also providing configuration in a transactional way.

ostree admin deploy --etc-overlay=configlayer-0 --etc-overlay=configlayer-1 <osref>
would take the defaults from the new osref tree's /etc and overlay the provided config layers on top.

And
ostree admin deploy --etc=<etcref> <osref>
would just use whatever is in <etcref> for /etc entirely, avoiding any defaults.

[cgwalters]

Although particularly #2081 we kind of support this today if one does a locked+staged deployment. Hmm, it looks like we never quite promoted the rpm-ostree logic for deployment locking into ostree.

[cgwalters]

Here's a random idea: What if we supported /etc.d? Then it wouldn't be ostree specific at all and could be implemented by other systems. Then we could say e.g. Ignition would write to /etc.d/ignition. If a user wanted to add some manual state it'd be /etc.d/custom/systemd/system/myservice.conf. Tooling like Ansible could write to /etc.d/myplaybook and /etc.d/myotherplaybook etc.

Maybe we make /etc a FUSE filesystem that dynamically unions things, and files that come from an underlying layer would be read-only. Writes would go to /etc.d/local or something.

[Bob131]

Building upon the previous suggestion, it might be interesting to try and support a (mostly) ephemeral /etc, something like a union mount of /usr/etc and a tmpfs with some magic to try and store things like /etc/machine-id in a place it can persist. The use-case I had in mind is a system that tries to avoid the configuration management problem by just generating non-default configs in /etc on every boot.

[cgwalters]

Also, what about secrets? ostree was never designed to store secrets. I think anyone who is doing this and transporting them would want to ensure that the data is encrypted in transit (i.e. TLS) and ideally at rest too. Maybe something like eCryptfs or integration with fs-crypt?

[AdrianVovk]

@cgwalters What kind of secrets do you expect OSTree to be transporting over a network? Not sure I'm following here

[cgwalters]

@cgwalters What kind of secrets do you expect OSTree to be transporting over a network? Not sure I'm following here

A good example would be someone who is using ostree to do builds for devices that need to connect to one or more WiFi networks, and they want to bake in the credentials necessary for that.

Another example is requiring a secret key or token in order to access OS updates at all.

[cgwalters]

As of lately, my focus is on "ostree native containers" work over here: https://github.com/ostreedev/ostree-rs-ext

This helps solve this problem because there is a new first-class way to add configuration changes into the transaction by having them be part of a derived image.