tests: install most important snaps

[sergiocazzolato]

This branch adds a tests which installs a set of snaps form different channels.

The basic idea is to test that most important snaps can be installed and there is not any failure in the process. This test will be executed nightly with spread cron.

The test autodetects the channel and mode for each snap.

[zyga]

I'm not super keen on having this in the main suite where it blocks all PRs.

Ideally (at least for me, obviously) this would be a nightly suite that we use as a go/no-go test for releases.

[mvo5]

I think this is what is happening, note that the test is makred with "manual: true" below. However I think we should mention this in either the description or the summay.

[zyga]

Aha, I missed that.

[sergiocazzolato]

Added details explaning that

[zyga]

Who will maintain this list over time?

[sergiocazzolato]

Well, I'll do until we implement a way to get automatically that list. I'll be working on that.

[zyga]

Let's use MATCH here

[mvo5]

Thanks for this PR! Some nitpicks and suggestions inside but no blockers.

[zyga]

I'm not super keen on having this in the main suite where it blocks all PRs.

Ideally (at least for me, obviously) this would be a nightly suite that we use as a go/no-go test for releases.

[mvo5]

I think this is what is happening, note that the test is makred with "manual: true" below. However I think we should mention this in either the description or the summay.

[zyga]

Aha, I missed that.

[sergiocazzolato]

Added details explaning that

[mvo5]

(nitpick, not really important as we use bash in spread anyway but I can't help it, years of ensuring maintainer scripts work with dash (instead of bash) have tained me :)) - if you want to write this in portable "sh" instead of bash you could simpliy write it as.

CHANNELS="stable candidate beta edge"
for CHANNEL in $CHANNELS; do
...

[sergiocazzolato]

sure, change done, thanks for reviewing

[mvo5]

(nitpick) I find it harder to read the postfix || && than using "if ...; then", I would prefer converting it. I assume this is for snaps that are not actually in the store (anymore) ? If so, maybe we can also have an echo like "skippping test for $SNAP" or something? And if spread grows a warning feaure we can emit a proper warning here :)

[sergiocazzolato]

ok, I'll add the echo.

[codecov-io]

Codecov Report

Merging #3692 into master will decrease coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3692      +/-   ##
==========================================
- Coverage   75.81%   75.79%   -0.02%     
==========================================
  Files         402      402              
  Lines       34745    34745              
==========================================
- Hits        26341    26336       -5     
- Misses       6529     6533       +4     
- Partials     1875     1876       +1

Impacted Files	Coverage Δ
wrappers/binaries.go	72.72% <0%> (-6.82%)	⬇️
cmd/snap/cmd_aliases.go	93.33% <0%> (-1.67%)	⬇️
interfaces/sorting.go	98.71% <0%> (-1.29%)	⬇️
overlord/ifacestate/helpers.go	62.33% <0%> (ø)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6cc76e4...bc6e0fb. Read the comment docs.

[sergiocazzolato]

@zyga @mvo changes ready, just failing some tests because the refresh-all tests are failing on autopkgtest