Fix data volumes

[drpaneas]

The documentation proposes to use bind mounts and the problem with those is that the geekotest user ends up with a different UID compared to the same user on the host machine.

uid=496(geekotest) #webui container
uid=496(systemd-bus-proxy) #worker container
uid=496(nscd) #host OS

As a result there are complications where openQA tries to access this data (Save, Upload, etc). This can be fixed by this solution or by using persistent named data volumes.

In the following example I am using a data volume, called Assets and another one called Tests.

Working Example

Start the webui container:

docker run -d --name openqa_webui -p 80:80 -p 873:873 -p 443:443 -v Assets:/var/lib/openqa/share/factory -v Tests:/var/lib/openqa/share/tests binarysequence/openqa-webui

Create the default fake authentication keys: curl -X POST http://localhost:80/login
Notice: for this one, I would recommend to install curl in the image and do it from there

Start the worker container:

docker run -d --privileged --name openqa_worker --link openqa_webui:openqa-webui --volumes-from openqa_webui binarysequence/openqa-worker-x86_64

Fetch the source code of the tests:

docker exec -it --user geekotest openqa_webui /var/lib/openqa/script/fetchneedles

Now, let's try to clone a job which requires to download an iso, write a qcow image and share it with among the two containers:

docker exec -it --user geekotest openqa_webui /var/lib/openqa/script/clone_job.pl --host localhost --from https://openqa.opensuse.org 579016

Debugging

As you can see, this data is now accessible between the two containers via the --volumes-from parameter and also in the host system:

# docker exec -it openqa_webui ls -l /var/lib/openqa/share/tests/ | tail -n 1
drwxr-xr-x 1 geekotest nogroup 194 Jan 11 10:19 opensuse
# docker exec -it openqa_worker ls -l /var/lib/openqa/share/tests/ | tail -n 1
drwxr-xr-x 1 systemd-bus-proxy nobody 194 Jan 11 10:19 opensuse
# ls -l /var/lib/docker/volumes/Tests/_data | tail -n 1
drwxr-xr-x 1 nscd nobody 194 Jan 11 11:19 opensuse

# docker exec -it openqa_webui ls -l /var/lib/openqa/share/factory/{iso,hdd} | grep '-'
-rw-r--r-- 1 geekotest nogroup 1341506560 Jan 11 10:50 opensuse-Tumbleweed-x86_64-20180109-textmode@64bit.qcow2
-rw-r--r-- 1 geekotest nogroup 4577034240 Jan 10 14:49 openSUSE-Tumbleweed-DVD-x86_64-Snapshot20180109-Media.iso
# docker exec -it openqa_worker ls -l /var/lib/openqa/share/factory/{iso,hdd} | grep '-'
-rw-r--r-- 1 systemd-bus-proxy nobody 1341506560 Jan 11 10:50 opensuse-Tumbleweed-x86_64-20180109-textmode@64bit.qcow2
-rw-r--r-- 1 systemd-bus-proxy nobody 4577034240 Jan 10 14:49 openSUSE-Tumbleweed-DVD-x86_64-Snapshot20180109-Media.iso
# ls -l /var/lib/docker/volumes/Assets/_data/{iso,hdd} | grep '-'
-rw-r--r-- 1 nscd nobody 1341506560 Jan 11 11:50 opensuse-Tumbleweed-x86_64-20180109-textmode@64bit.qcow2
-rw-r--r-- 1 nscd nobody 4577034240 Jan 10 15:49 openSUSE-Tumbleweed-DVD-x86_64-Snapshot20180109-Media.iso

However, volumes need manually deletion. You cannot clean them up just be removing the container. It's an extra step, which is implemented this way because of insurances purposes. The point is that this data it's important; at least much more important than the container itself. It's a location for the container to store data that outlive the executable -- that is why we need to name them.