tests, spread: add Arch Linux to CI systems

[bboozzoo]

This is an early PR adding support for Arch Linux in the CI pipeline. Hopefully this will allow to catch the cross-distro incompatibilities sooner.

Related forum topic: https://forum.snapcraft.io/t/integrate-arch-linux-into-ci-pipeline/2904

[zyga]

LGMT with one remark.

[zyga]

Can we please call this build_arch_pkg, people may find this confusing as package is a generic term.

[bboozzoo]

renamed

[zyga]

We are still running on the Linode kernel: 5,0,0,-;Linux version 4.9.50-x86_64-linode86 (maker@build.linode.com) (gcc version 4.9.2 (Debian 4.9.2-10) ) #1 SMP Thu Sep 14 19:28:20 UTC 2017

[sergiocazzolato]

Change looks promising, question inline.

[sergiocazzolato]

why you are changing this?

[bboozzoo]

We need to do a system upgrade in Arch in order to get a consistent system state. I figured that moving this operation into project prepare was the best fit (see 7773050 for implementation). However, there is a slight complication, in order to do a reboot properly, we use the REBOOT command which is only defined at the top level spread prepare|restore.. and so on, so it's not visible in prepare-restore.sh which is executed in a separate shell. To work around this problem, I've opted for sourcing the script instead. 😞

Other options are to move the upgrade piece to prepare block, or use some special exit code in prepare-restore.sh to indicate that reboot is needed.

[mvo5]

Could you please add a comment (copy/paste this here is probably fine) about the details why this is done in the way you do it?

[sergiocazzolato]

perhaps you could use MATCH here

[bboozzoo]

MATCH is only defined in the shell that runs the script defined in spread.yaml or task.yaml. If pkgdb.sh is used in separate shell, the function may not be defined.

[zyga]

I have a solution for missing MATCH that came out of my refactoring. I have it ready but I'll only post it on Monday to let you guys rest too :)

[mvo5]

This margin is too small to contain it ;)?

[zyga]

Looks good, nice to see more tests enabled!

[mvo5]

We probably need more than a single worker here.

[mvo5]

Thanks a driving this, really great to see arch in the CI. Some questions/suggestion inline.

[sergiocazzolato]

why you are changing this?

[bboozzoo]

We need to do a system upgrade in Arch in order to get a consistent system state. I figured that moving this operation into project prepare was the best fit (see 7773050 for implementation). However, there is a slight complication, in order to do a reboot properly, we use the REBOOT command which is only defined at the top level spread prepare|restore.. and so on, so it's not visible in prepare-restore.sh which is executed in a separate shell. To work around this problem, I've opted for sourcing the script instead. 😞

Other options are to move the upgrade piece to prepare block, or use some special exit code in prepare-restore.sh to indicate that reboot is needed.

[mvo5]

Could you please add a comment (copy/paste this here is probably fine) about the details why this is done in the way you do it?

[sergiocazzolato]

perhaps you could use MATCH here

[bboozzoo]

MATCH is only defined in the shell that runs the script defined in spread.yaml or task.yaml. If pkgdb.sh is used in separate shell, the function may not be defined.

[zyga]

I have a solution for missing MATCH that came out of my refactoring. I have it ready but I'll only post it on Monday to let you guys rest too :)

[mvo5]

This margin is too small to contain it ;)?

[mvo5]

Hm, is this really the only way? It looks really hard to read, what are the inputs/outputs here?

[bboozzoo]

We need to comment out this whole pkgver section: https://github.com/bboozzoo/snapd/blob/a3a463199cc12b4dbb70e92339f7c45c518317eb/packaging/arch/PKGBUILD#L33 then
update source=("git+ssh://...") to point to tar.gz, rename package_snapd-git() to package_snapd() and rename the pacakge to snapd.

[mvo5]

Hm, I get the feeling this may hide a bug in our code? Shouldn't we fail gracefully if this dir does not exists? cc @zyga

[zyga]

I think so though perhaps some tests are overzealous. What did you see @bboozzoo ?

[bboozzoo]

DIRECTORY/alternatives fails because it tries to get a inode of this directory in the host's filesystem and fails as there is no such directory on Arch.

[codecov-io]

Codecov Report

Merging #4285 into master will decrease coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #4285      +/-   ##
==========================================
- Coverage   78.05%   78.03%   -0.02%     
==========================================
  Files         450      450              
  Lines       30899    30899              
==========================================
- Hits        24118    24113       -5     
- Misses       4772     4776       +4     
- Partials     2009     2010       +1

Impacted Files	Coverage Δ
overlord/hookstate/hookmgr.go	71.67% <0%> (-1.16%)	⬇️
overlord/ifacestate/helpers.go	63.48% <0%> (-0.83%)	⬇️
cmd/snap-seccomp/main.go	52.6% <0%> (-0.53%)	⬇️
interfaces/builtin/system_observe.go	100% <0%> (ø)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 87e197c...be14ed4. Read the comment docs.

[bboozzoo]

Another Arch curiosity:

2017-11-29 12:35:04 Error preparing linode:arch-2017.07.01:tests/main/interfaces-dbus : 
...
And the provider dbus loop is started
+ . /home/gopath/src/github.com/snapcore/snapd/tests/lib/dbus.sh
+ start_dbus_unit /var/lib/snapd/snap/bin/test-snapd-dbus-provider.provider
+ local executable=/var/lib/snapd/snap/bin/test-snapd-dbus-provider.provider
+ dbus-launch
dbus-launch: error while loading shared libraries: libX11.so.6: cannot open shared object file: No such file or directory

Turns out dbus does not depend on libx11 😕