tests,packaging: add package build support for openSUSE #3406

Merged
merged 31 commits into from Jun 9, 2017

Conversation

Projects
None yet
5 participants
Contributor

morphis commented May 29, 2017

This is currently based on #3365 so you will see the changes for Fedora here too until that PR is merged.

Simon Fels added some commits May 29, 2017

codecov-io commented May 29, 2017

Codecov Report

Merging #3406 into master will increase coverage by <.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3406      +/-   ##
==========================================
+ Coverage   77.26%   77.27%   +<.01%     
==========================================
  Files         373      373              
  Lines       25687    25687              
==========================================
+ Hits        19848    19849       +1     
+ Misses       4096     4095       -1     
  Partials     1743     1743
Impacted Files Coverage Δ
interfaces/sorting.go 93.33% <0%> (ø) ⬆️
cmd/snap/cmd_aliases.go 96% <0%> (+2%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fab55b8...61b4899. Read the comment docs.

Simon Fels added some commits May 29, 2017

Simon Fels added some commits Jun 2, 2017

Looks good, some questions inline but quite happy about it!

packaging/fedora/snap-mgmt.sh
@@ -18,7 +32,7 @@ systemctl_stop() {
fi
}
-if [ "$1" = "purge" ]; then
+purge() {
@mvo5

mvo5 Jun 7, 2017

Collaborator

I added a forum topic that we need to look into sharing code between this and the debian postrm (https://forum.snapcraft.io/t/share-code-between-debian-postrm-purge-and-snap-mgmt-sh/915)

@morphis

morphis Jun 7, 2017

Contributor

Thanks!

spread.yaml
- # poor-man's "quiet"
- apt-get update >& "$tf" || ( cat "$tf"; exit 1 )
- apt-get install -y xdelta3 curl >& "$tf" || ( cat "$tf"; exit 1 )
+ # NOTE: We can't use tests/lib/pkgdb.sh here as it doesn't exist at
@mvo5

mvo5 Jun 7, 2017

Collaborator

Thanks for this comment! I was wondering exactly this and you answered it :-D

@morphis

morphis Jun 7, 2017

Contributor

:-)

tests/lib/apt.sh
@@ -13,6 +13,28 @@ install_build_snapd(){
mv sources.list.back /etc/apt/sources.list
apt update
else
- distro_install_local_package "$GOHOME"/snapd_*.deb
+ packages=
+ case "$SPREAD_SYSTEM" in
@mvo5

mvo5 Jun 7, 2017

Collaborator

Idle wondering if the name for this file (apt.sh) is still appropriate - maybe it should just get merged with pkgdb.sh - but that is a different PR I guess :)

@morphis

morphis Jun 7, 2017

Contributor

Done

tests/lib/pkgdb.sh
+ dnf -y -q upgrade
+ ;;
+ opensuse-*)
+ zypper -q update -y
@mvo5

mvo5 Jun 7, 2017

Collaborator

Funny how similar all the tools are in their CLI interface. I guess apt had a lot of influence here.

tests/lib/prepare.sh
@@ -96,29 +101,53 @@ prepare_classic() {
if snap --version |MATCH unknown; then
echo "Package build incorrect, 'snap --version' mentions 'unknown'"
snap --version
- apt-cache policy snapd
+ case "$SPREAD_SYSTEM" in
@mvo5

mvo5 Jun 7, 2017

Collaborator

Could we abstract this to pkgdb.sh as well maybe? Feels a bit ugly to have this test here.

@morphis

morphis Jun 7, 2017

Contributor

Done.

Simon Fels added some commits Jun 7, 2017

@morphis morphis referenced this pull request Jun 7, 2017

Closed

tests: apt autoclean #3437

Simon Fels added some commits Jun 7, 2017

tests/main/ack/task.yaml
@@ -1,5 +1,4 @@
summary: Check snap ack
-systems: [-ubuntu-core-16-arm-*]
@morphis

morphis Jun 8, 2017

Contributor

Hm, that shouldn't happen. Let me fix that.

@zyga

zyga Jun 8, 2017

Contributor

This is still broken

@morphis

morphis Jun 8, 2017

Contributor

Hm, I've pushed a change for that.

Some comments for your consideration.

@@ -171,7 +173,7 @@ install -m 644 -D packaging/opensuse-42.2/permissions %buildroot/%{_sysconfdir}/
install -m 644 -D packaging/opensuse-42.2/permissions.paranoid %buildroot/%{_sysconfdir}/permissions.d/snapd.paranoid
# Install the systemd units
make -C data/systemd install DESTDIR=%{buildroot} SYSTEMDSYSTEMUNITDIR=%{_unitdir}
-for s in snapd.autoimport.service snapd.system-shutdown.service; do
+for s in snapd.autoimport.service snapd.system-shutdown.service snap-repair.timer snap-repair.service; do
@zyga

zyga Jun 8, 2017

Contributor

I was under the impression that the repair functionality is only applicable to core systems. I would not recommend using it in any classic distribution where existing packaging systems are sufficient to recover from any issue.

@morphis

morphis Jun 8, 2017

Contributor

That is exactly why we're rm these system units here after they got installed by data/systemd/Makefile :-)

@zyga

zyga Jun 8, 2017

Contributor

Ohhh, I didn't notice the line below :D

spread.yaml
@@ -317,7 +322,7 @@ restore: |
suites:
tests/main/:
summary: Full-system tests for snapd
- systems: [-fedora-*]
+ systems: [-fedora-*, -opensuse-*]
@zyga

zyga Jun 8, 2017

Contributor

Can you add a comment explaining why those are disabled?

@morphis

morphis Jun 8, 2017

Contributor

Sure.

@@ -336,14 +341,17 @@ suites:
if [[ "$SPREAD_SYSTEM" != ubuntu-core-16-* ]]; then
. $TESTSLIB/pkgdb.sh
distro_purge_package snapd
- distro_purge_package snap-confine
+ if [[ "$SPREAD_SYSTEM" != opensuse-* ]]; then
@zyga

zyga Jun 8, 2017

Contributor

This is where I would argue that the following solution is cleaner:

# NOTE: snap-confine never existed on openSUSE
distro-purge-package {debian,fedora}:snap-confine`
@morphis

morphis Jun 8, 2017

Contributor

It's always a tradeoff. Purging not existing packages is a very rare case where as installing packages is the very regular one. So limiting us to a minimal set of these is enough here.

spread.yaml
fi
tests/completion/:
summary: completion tests
# ppc64el disabled because of https://bugs.launchpad.net/snappy/+bug/1655594
- systems: [-ubuntu-core-*, -ubuntu-*-ppc64el, -fedora-*]
+ systems: [-ubuntu-core-*, -ubuntu-*-ppc64el, -fedora-*, -opensuse-*]
@zyga

zyga Jun 8, 2017

Contributor

Why are those tests disabled on Fedora and openSUSE? Can you please add a comment.

spread.yaml
@@ -373,7 +381,7 @@ suites:
tests/regression/:
summary: Regression tests for snapd
- systems: [-fedora-*]
+ systems: [-fedora-*, -opensuse-*]
@zyga

zyga Jun 8, 2017

Contributor

Can you please document why those are disabled? Ideally we'd run them and skip the parts that involve confinement.

@morphis

morphis Jun 8, 2017

Contributor

Sure, that is what will happen in a later PR.

spread.yaml
@@ -392,7 +400,7 @@ suites:
tests/upgrade/:
summary: Tests for snapd upgrade
- systems: [-ubuntu-core-16-*, -fedora-*]
+ systems: [-ubuntu-core-16-*, -fedora-*, -opensuse-*]
@zyga

zyga Jun 8, 2017

Contributor

Can you please document those too?

spread.yaml
@@ -408,7 +416,7 @@ suites:
tests/unit/:
summary: Suite to run unit tests (non-go and different go runtimes)
- systems: [-ubuntu-core-16-*, -fedora-*]
+ systems: [-ubuntu-core-16-*, -fedora-*, -opensuse-*]
@zyga

zyga Jun 8, 2017

Contributor

Can you please document those too? I thought that we can run unit tests on openSUSE now, is this not the case?

@morphis

morphis Jun 8, 2017

Contributor

We can run unit tests on openSUSE but as explained before this PR just adds build support for openSUSE and doesn't enable any spread tests (in real we have to enable at least a single one tests/main/help to get the build running). Enabling tests will happen with later PRs step by step.

spread.yaml
@@ -434,7 +442,7 @@ suites:
tests/nightly/:
summary: Suite for nightly, expensive, tests
manual: true
- systems: [-fedora-*]
+ systems: [-fedora-*, -opensuse-*]
@zyga

zyga Jun 8, 2017

Contributor

Can you please document those too?

spread.yaml
@@ -453,7 +461,7 @@ suites:
tests/nested/:
summary: Tests for nested images
- systems: [-fedora-*]
+ systems: [-fedora-*, -opensuse-*]
@zyga

zyga Jun 8, 2017

Contributor

Can you please document those too?

@@ -31,13 +31,24 @@ fedora_name_package() {
esac
}
+opensuse_name_package() {
@zyga

zyga Jun 8, 2017

Contributor

Yeah, I also wonder why those function names are backwards.

@morphis

morphis Jun 8, 2017

Contributor

Why are they backwards` It's were distro=opensuse verb=to name and what=package in this case.

@zyga

zyga Jun 8, 2017

Contributor

It just looks odd as nothing else uses this style. If anything I'd call it opensuse_package_name (object.object.object style)

tests/lib/pkgdb.sh
+ packages="${GOHOME}/snapd_*.deb"
+ ;;
+ fedora-*|opensuse-*)
+ packages="${GOHOME}/snap-confine*.rpm ${GOPATH}/snapd*.rpm"
@zyga

zyga Jun 8, 2017

Contributor

snap-confine is not a package for openSUSE, I think you have to split this

@morphis

morphis Jun 8, 2017

Contributor

We could but don't have to, Zypper wont fail interestingly. However let me split these.

tests/lib/pkgdb.sh
+
+ # Perform per distribution post-installation steps if necessary
+ case "$SPREAD_SYSTEM" in
+ opensuse-*)
@zyga

zyga Jun 8, 2017

Contributor

We could query systemd preset policy instead of maintaining a switch statement.

@morphis

morphis Jun 8, 2017

Contributor

Good idea.

@morphis

morphis Jun 8, 2017

Contributor

Done

@@ -66,6 +66,42 @@ fedora_build_rpm() {
rm $GOPATH/*.src.rpm
}
+opensuse_build_rpm() {
+ release=$(echo "$SPREAD_SYSTEM" | awk '{split($0,a,"-");print a[2]}')
@zyga

zyga Jun 8, 2017

Contributor

Could this just come from /etc/os-release? I'm not sure why we're encoding it in SPREAD_SYSTEM

@morphis

morphis Jun 8, 2017

Contributor

It's a reliable identifier we own. /etc/os-release isn't really owned by us. Would like to keep these influence out of the picture when we're doing spread specific things like setting up the environment.

+ sed -i -e "s/^Version:.*$/Version: $version/g" packaging/opensuse-$release/snapd.spec
+
+ mkdir -p /tmp/pkg/snapd-$version
+ cp -rav * /tmp/pkg/snapd-$version/
@zyga

zyga Jun 8, 2017

Contributor

Did you consider using 'osc build' to do a local build of the package?

@morphis

morphis Jun 8, 2017

Contributor

A osc build isn't possible without a login. Also I have code to unite the rpm build for Fedora and openSUSE under a single build_rpm function.

tests/lib/prepare.sh
EOF
+ systemctl daemon-reload
@zyga

zyga Jun 8, 2017

Contributor

Can you document why those are invoked here.

Simon Fels added some commits Jun 8, 2017

One thing is still broken (typo in syntax)

@@ -171,7 +173,7 @@ install -m 644 -D packaging/opensuse-42.2/permissions %buildroot/%{_sysconfdir}/
install -m 644 -D packaging/opensuse-42.2/permissions.paranoid %buildroot/%{_sysconfdir}/permissions.d/snapd.paranoid
# Install the systemd units
make -C data/systemd install DESTDIR=%{buildroot} SYSTEMDSYSTEMUNITDIR=%{_unitdir}
-for s in snapd.autoimport.service snapd.system-shutdown.service; do
+for s in snapd.autoimport.service snapd.system-shutdown.service snap-repair.timer snap-repair.service; do
@zyga

zyga Jun 8, 2017

Contributor

I was under the impression that the repair functionality is only applicable to core systems. I would not recommend using it in any classic distribution where existing packaging systems are sufficient to recover from any issue.

@morphis

morphis Jun 8, 2017

Contributor

That is exactly why we're rm these system units here after they got installed by data/systemd/Makefile :-)

@zyga

zyga Jun 8, 2017

Contributor

Ohhh, I didn't notice the line below :D

tests/main/ack/task.yaml
@@ -1,5 +1,4 @@
summary: Check snap ack
-systems: [-ubuntu-core-16-arm-*]
@morphis

morphis Jun 8, 2017

Contributor

Hm, that shouldn't happen. Let me fix that.

@zyga

zyga Jun 8, 2017

Contributor

This is still broken

@morphis

morphis Jun 8, 2017

Contributor

Hm, I've pushed a change for that.

zyga approved these changes Jun 8, 2017

LGTM

Simon Fels added some commits Jun 9, 2017

@mvo5 mvo5 merged commit e2b5530 into snapcore:master Jun 9, 2017

4 of 7 checks passed

xenial-amd64 autopkgtest running
Details
xenial-i386 autopkgtest running
Details
xenial-ppc64el autopkgtest running
Details
artful-amd64 autopkgtest finished (success)
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
yakkety-amd64 autopkgtest finished (success)
Details
zesty-amd64 autopkgtest finished (success)
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment