tests: reboot the node when restoring after a test involving lxd

[bboozzoo]

LXD makes changes to the mount namespace of the host. Make sure to reboot, so
that the next test runs with a clean state.

cc @zyga

[zyga]

+1 on the concept. I think it still needs fixing because ubuntu-core-* systems reboot up front. I'm afraid you have to store $SPREAD_REBOOT in prepare and decide upon the current value in restore. Otherwise you will either reboot or not depending on the history of other tests and the backend/system preparation.

[bboozzoo]

AFAIK SPREAD_REBOOT starts anew for each script.

[zyga]

Ah, indeed. I read the docs now and you can either do comparison on SPREAD_REBOOT or just use REBOOT with a numeric argument.

[zyga]

This looks good but fails in practice:

2019-07-30 12:03:22 Error restoring google:ubuntu-16.04-32:tests/main/lxd : 
-----
+ [[ 0 == 0 ]]
++ wc -l
++ find /home/gopath -name 'snapd_*.deb'
+ [[ 2 -eq 0 ]]
+ lxd.lxc stop my-ubuntu --force
+ lxd.lxc delete my-ubuntu
+ snap remove lxd
lxd removed
+ REBOOT
<REBOOT>

I wonder if this has to be in the execute section and doesn't really work outside of it.

[bboozzoo]

Apparently using REBOOT in spread task restore fails on 16.04 only. Needs further investigation.

...
travis_time:start:92dfd379
2019-07-31 10:47:41 Error restoring google:ubuntu-16.04-64:tests/main/lxd : 
-----
+ [[ 0 == 0 ]]
++ wc -l
++ find /home/gopath -name 'snapd_*.deb'
+ [[ 2 -eq 0 ]]
+ lxd.lxc stop my-ubuntu --force
+ lxd.lxc delete my-ubuntu
+ snap remove lxd
lxd removed
+ REBOOT
<REBOOT>
-----
travis_time:end:92dfd379:start=1564570040510960505,finish=1564570061665405026,duration=21154444521
travis_fold:end:fold-92dfd379
...
2019-07-31 11:17:39 Successful tasks: 2643
2019-07-31 11:17:39 Aborted tasks: 0
2019-07-31 11:17:39 Failed task restore: 3
    - google:ubuntu-16.04-32:tests/main/lxd
    - google:ubuntu-16.04-64:tests/main/lxd
    - google:ubuntu-16.04-64:tests/regression/lp-1815869
error: unsuccessful run

[bboozzoo]

Needs a fix in spread. I've implemented a fix in a private branch so far bboozzoo/spread@2ebf0c3 will try to propose it upstream.

[sergiocazzolato]

Nice fix, thanks for this!

[sergiocazzolato]

I set the PR as blocked until spread is fixed.

[bboozzoo]

Deprecated by #7435