cmd-build-fast: New command to iterate on content quickly

[cgwalters]

Part of the implementation of
#1341

This is useful for me hacking on ostree/rpm-ostree. Here we
don't need to regenerate the initramfs (usually), we just want
new ostree binaries in /usr. But we do want the OS to be
set up as a "proper" OSTree commit, i.e. not rpm-ostree usroverlay.

Here this completes a qemu image (overlaid on previous) in 13 seconds,
compared to 147 seconds for a full build; 11x faster and we're
not writing O(sizeof(coreos)) bytes to my SSD, but
O(sizeof(change) + sizeof(metadata(coreos)).

With this a workflow looks like:

$ cd /src/ostree
$ make && make install DESTDIR=/path/to/cosa/overrides/rootfs
$ cd /path/to/cosa
$ cosa build-fast
$ cosa run --qemu-image tmp/fastbuild/fastbuild-qemu.qcow2

(Or, instead of cosa run, cosa kola run --qemu-image tmp/fastbuild/fastbuild-qemu.qcow2 -E /src/ostree ext.ostree.*)

[cgwalters]

OK whee I just had my first "oh wait this doesn't work because I used build-fast" moment - I was working on a new systemd unit in the target root stored in the overlays and updated the .preset file and tried to hack things by rsync -rlv src/config/overlay.d/foo/ overrides/rootfs - but we only running the systemctl preset-all in the postprocess, which this skips. So my unit was silently not enabled.

But...eh, we can expand on this as needed.

[cgwalters]

OK I now updated this to run in another mode that more clearly replaces rpm-ostree's make vmoverlay - if you e.g.
cd /path/to/ostree.git && env COSA_DIR=/path/to/fcos cosa build-fast you will end up with the qemu image in the ostree working directory, and you don't need to think about the make install DESTDIR= part. It's a bit like having per-project coreos-assembler branches.

xref coreos/rpm-ostree#1958

[cgwalters]

Bigger picture, this PR is trying to add to coreos-assembler the concept of a "project source directory" that doesn't currently exist aside from the src/config.

I worked on this PR in parallel with this script in ostree:
https://github.com/ostreedev/ostree/pull/2048/files#diff-893bbb0a1a54c74c3f41b00641d40909

Because again an important goal is "edit $project (ostree, igntition, etc.); build fast dev qemu image, run project tests" while still supporting "run project tests" outside of building the project.

Tying these two things together for example, if this lands I'd probably change runkola.sh in ostree's tests to glob for fastbuild*.qcow2 or so.

I dunno though. Maybe it is cleaner to have something like:

$ ln -sr /path/to/ostree overrides/src

Then you always live inside the cosa workdir, and it becomes much more obvious how to work on multiple projects.

[jlebon]

Cool stuff! I like the project dir approach.

ISTM like we'll want the OSTree parts of logic in rpm-ostree instead at some point, e.g. rpm-ostree compose overlay or something? Then we get things like initrd regeneration and the /etc move thing more easily. (This would be helpful too for eventually implementing coreos/rpm-ostree#442.)

[jlebon]

This is minor/an aside, though in non-trivial shell scripts like this I find it conceptually easier to just always stay in the same directory and do anything that really needs a cd in a subshell.

[cgwalters]

But everything after that is in the project dir?

[jlebon]

Hmm, it seemed to me like all the paths used were absolute in the ${projectdir} case.

[jlebon]

/lgtm

[openshift-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cgwalters, jlebon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [cgwalters,jlebon]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[cgwalters]

ISTM like we'll want the OSTree parts of logic in rpm-ostree instead at some point, e.g. rpm-ostree compose overlay or something?

Definitely, I have a local branch for that with some hacks but...it has a similar problem to trying to do this in the main coreos-assembler path in that it's super messy to add a bunch of "skip this, skip that, hack this" steps into the main code flow.