tests: Rework tests/installed → tests/kola

[cgwalters]

Previously we made an effort to use the Fedora Standard Test Interface.
This effort was not very successful; the primary thing that
it really died on is Ansible just didn't support rebooting
very well. I think that's since gotten better, but even
then, Ansible wasn't the best thing for a test framework
for us anyways.

In the meantime Fedora CoreOS happened emphasizing Ignition
and not "post-hoc reconciliation" models like Ansible over
ssh.

And, coreos-assembler happened too.

Furthermore, we really need to test OSTree's interaction
with Ignition as we've invented several special things there.

Then most recently, I've been working on having
cosa/kola support running externally defined tests:
coreos/coreos-assembler#1215

There's a lot of things to clean up after this but at least this
works for me:

$ cd /srv/fcos
$ cosa kola run -- --parallel 4 --output-dir tmp/kola -E ~/src/github/ostreedev/ostree/ 'ext.ostree.*'

NOTE: This does not drop ostree binaries into the target. See:
coreos/coreos-assembler#1252 (comment)

This drops our dependency on Python in the installed tests, and
also fixes a few bugs that came up.

I disabled the itest-bare-user-root.sh one because it's
entangled with the shell script infrastructure for the unit tests.

[cgwalters]

OK, a major update here, see the commit message for details. All tests passing here. Out with the Ansible, in with the kola!

[cgwalters]

Updated to s/deps/data/. I guess we could wait a bit till the cosa container is rebuilt and try running this in our CI...but OTOH I wouldn't want to block if that fails necessarily because even having this in master is super useful for me because at least anyone running locally can run tests more easily.

[cgwalters]

Interesting, so that Travis failure is:

ERROR: tests/test-concurrency.py
================================
n_committers 4 n_pruners 0
Creating tree tree0
Creating tree tree1
PID 30315 ostree --repo=repo commit --fsync=0 -b tree0 --tree=dir=tree0
PID 30316 ostree --repo=repo commit --fsync=0 -b tree0 --tree=dir=tree0
PID 30317 ostree --repo=repo commit --fsync=0 -b tree1 --tree=dir=tree1
PID 30318 ostree --repo=repo commit --fsync=0 -b tree1 --tree=dir=tree1
PID 30316 exited OK
PID 30318 exited OK
PID 30317 exited OK
PID 30315 exited OK
n_committers 6 n_pruners 3
Creating tree tree0
Creating tree tree1
Creating tree tree2
PID 30321 ostree --repo=repo commit --fsync=0 -b tree0 --tree=dir=tree0
PID 30322 ostree --repo=repo commit --fsync=0 -b tree0 --tree=dir=tree0
PID 30323 ostree --repo=repo commit --fsync=0 -b tree1 --tree=dir=tree1
PID 30324 ostree --repo=repo commit --fsync=0 -b tree1 --tree=dir=tree1
PID 30325 ostree --repo=repo commit --fsync=0 -b tree2 --tree=dir=tree2
PID 30326 ostree --repo=repo commit --fsync=0 -b tree2 --tree=dir=tree2
error: opendir(staging-89ed2f5d-09e9-4dc1-8360-9d7011bc2edd-anBXJW): No such file or directory
PID 30327: ostree --repo=repo prune --refs-only
PID 30328: ostree --repo=repo prune --refs-only
PID 30329: ostree --repo=repo prune --refs-only
process 30322 exited with code 1
PID 30326 exited OK
PID 30324 exited OK
PID 30321 exited OK
PID 30323 exited OK
PID 30325 exited OK
PID 30328 exited OK
PID 30327 exited OK
PID 30329 exited OK
A child process exited abnormally
1..2
014ccd06e67eacefc6a965c01a3fe3aa953cbac93cd17ab49b7cc35325b9e8b9
67eddf38725595bf7a23c692101b6edded083c8fbc5b099ce7be5afa6f56f90c
00867bcf06c15f34b3b1ce4d62eeb13747ec470c1b52bf74f5834475fa8875d9
0fbf3241a95691c836497e5c1ae0422d67fe9e71ba8fade043c64c58970a5b0d
ok no concurrent prunes
PASS: tests/test-concurrency.py 1 no concurrent prunes
d62b8c33421f8dadd6844646cb46e7f7bf8e9dfe79db01c6829ac282fa8d3a3a
83798fcef5f1893b7848103e039021de22eaeb8615cb05e9bfa3b4b16352605a
10461655e44945cd0c923c6451de43c5cf0ad0ef11f62f5c24aabb7e9a6a1485
7a566b367f6a250f2e1433e42ad4f84120b2d6ce87856f9f3f7bddacc690689f
Total objects: 72
No unreachable objects
fd51a8b23108e97ec7d57dea2a13b9ada796e0998b1acd364b94e900aaf4d80b
Total objects: 73
No unreachable objects
Total objects: 73
No unreachable objects
ERROR: tests/test-concurrency.py - too few tests run (expected 2, got 1)
ERROR: tests/test-concurrency.py - exited with status 1

Worth investigating at some point.

[cgwalters]

/override continuous-integration/travis-ci/pr

[openshift-ci-robot]

@cgwalters: Overrode contexts on behalf of cgwalters: continuous-integration/travis-ci/pr

In response to this:

/override continuous-integration/travis-ci/pr

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[jlebon]

This is definitely a nice improvement over the Ansible stuff! I still need to think about how this contrasts to the rpm-ostree approach and the pros and cons of each. I think it would simplify a lot of things, but complicate some other things. But happy to see better coverage here! Looking forward to having this hooked up to CI soon.

LGTM overall, just one minor comment that stuck out, but not really a blocker.

[jlebon]

/lgtm

[openshift-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cgwalters, jlebon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [cgwalters,jlebon]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment