tests: document nested suites (#13879)

* tests: document nightly/sbuild

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests: document nested/classic/hotplug

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests: fix typo: verifying

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests: fix typo: established

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests: fix typo: finished

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests: fix typo: missing space

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests: document nested/classic/snapshots-with-core-refresh-revert

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests: document nested/core/core-revert

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests: document nested/core/image-build

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests: document nested/manual/cloud-init-*-vuln

The two tests are related, one checks what happens on a system that
boots up for the first time with the fix, while the other the fix is
applied on update.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests: document nested/manual/cmdline-remove-append

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests: document nested/manual/core-early-config

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests: document nested/manual/core20-new-snapd-does-not-break-old-initrd

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests: document nested/manual/core20-validation-sets

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests: fix typo: bionic

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests: document nested/manual/gadget-connections

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* Update tests/nested/classic/hotplug/task.yaml

Co-authored-by: Graham Morrison <graham.morrison@canonical.com>

* Update tests/nested/classic/hotplug/task.yaml

Co-authored-by: Graham Morrison <graham.morrison@canonical.com>

* Update tests/nested/classic/hotplug/task.yaml

Co-authored-by: Graham Morrison <graham.morrison@canonical.com>

* Update tests/nested/core/core-revert/task.yaml

Co-authored-by: Graham Morrison <graham.morrison@canonical.com>

* Update tests/nested/core/core-revert/task.yaml

Co-authored-by: Graham Morrison <graham.morrison@canonical.com>

* Update tests/nested/core/core-revert/task.yaml

Co-authored-by: Graham Morrison <graham.morrison@canonical.com>

* Update tests/nested/core/core-revert/task.yaml

Co-authored-by: Graham Morrison <graham.morrison@canonical.com>

* Update tests/nested/core/core20-kernel-failover/task.yaml

Co-authored-by: Graham Morrison <graham.morrison@canonical.com>

* Update tests/nested/core/core20-kernel-failover/task.yaml

Co-authored-by: Graham Morrison <graham.morrison@canonical.com>

* Update tests/nested/manual/cmdline-remove-append/task.yaml

Co-authored-by: Graham Morrison <graham.morrison@canonical.com>

* Update tests/nested/manual/core-early-config/task.yaml

Co-authored-by: Graham Morrison <graham.morrison@canonical.com>

* Update tests/nested/manual/core-early-config/task.yaml

Co-authored-by: Graham Morrison <graham.morrison@canonical.com>

* Update tests/nested/manual/core20-new-snapd-does-not-break-old-initrd/task.yaml

Co-authored-by: Graham Morrison <graham.morrison@canonical.com>

* Update tests/nested/manual/core20-new-snapd-does-not-break-old-initrd/task.yaml

Co-authored-by: Graham Morrison <graham.morrison@canonical.com>

* Update tests/nested/manual/devmode-snaps-can-run-other-snaps/task.yaml

Co-authored-by: Graham Morrison <graham.morrison@canonical.com>

* Update tests/nested/manual/gadget-connections/task.yaml

Co-authored-by: Graham Morrison <graham.morrison@canonical.com>

* Update tests/nightly/sbuild/task.yaml

Co-authored-by: Graham Morrison <graham.morrison@canonical.com>

* Update tests/nested/manual/devmode-snaps-can-run-other-snaps/task.yaml

Co-authored-by: Graham Morrison <graham.morrison@canonical.com>

* Update task.yaml

---------

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
Co-authored-by: Sergio Cazzolato <sergiocazzolato@gmail.com>
Co-authored-by: Graham Morrison <graham.morrison@canonical.com>
Co-authored-by: Sergio Cazzolato <sergio.cazzolato@canonical.com>

tests/nested/classic/hotplug/task.yaml

@@ -1,5 +1,19 @@
 summary: Create ubuntu classic image, install snapd and test hotplug feature
 
+details: |
+    Snapd contains a little-used subsystem for reacting to dynamic device
+    reconfiguration events, and in response, maintains a set of system-provided
+    slots for certain snapd interfaces.
+
+    One of the first interfaces to support this feature was the serial-port
+    interface. Using this, snapd was able to detect serial ports that were not declared
+    statically in the gadget snap, but instead were enumerated on startup or
+    discovered dynamically when attached to the system over the USB bus.
+
+    Simulating and testing this sort of interaction requires a nested virtual machine, where
+    qemu can be scripted to dynamically attach devices to the running virtual
+    machine.
+
 prepare: |
     echo "Install snapd.deb in the nested vm"
     remote.push "${GOHOME}"/snapd_*.deb
@@ -83,7 +97,7 @@ execute: |
     remote.exec "sudo snap connect serial-port-hotplug:serial-port :qemuusbserial"
     check_slot_connected qemuusbserial
 
-    echo "Veryfing serial-port permissions of the snap"
+    echo "Verifying serial-port permissions of the snap"
     verify_apparmor_profile "/dev/ttyUSB0"
     remote.exec "/snap/bin/serial-port-hotplug.consumer write-1" | MATCH "Access to /dev/ttyUSB0 ok"
     remote.exec "/snap/bin/serial-port-hotplug.consumer write-2" | MATCH "Access to /dev/ttyUSB1 failed"
@@ -119,7 +133,7 @@ execute: |
     check_slot_connected qemuusbserial
     check_slot_not_gone qemuusbserial
 
-    echo "Veryfing serial-port permissions of the snap, the first device is now expected on ttyUSB1"
+    echo "Verifying serial-port permissions of the snap, the first device is now expected on ttyUSB1"
     check_slot_device_path qemuusbserial "/dev/ttyUSB1"
     verify_apparmor_profile "/dev/ttyUSB1"
     remote.exec "/snap/bin/serial-port-hotplug.consumer write-3" | MATCH "Access to /dev/ttyUSB0 failed"

tests/nested/classic/snapshots-with-core-refresh-revert/task.yaml

@@ -1,5 +1,12 @@
 summary: test snapshots work when core snap is refreshed and reverted
 
+details: |
+    Snapd can create and restore snapshots of snap application data. This test
+    checks if refreshing and reverting the core snap breaks this capability. The
+    test allows comparison of the locally-built snapd with the core snap from a
+    given channel in the store, making it useful for ensuring the data format is
+    stable or compatible across snapd updates.
+
 prepare: |
     echo "Configure hosts file"
     # shellcheck disable=SC2016

tests/nested/core/core-revert/task.yaml

@@ -1,5 +1,13 @@
 summary: core revert test
 
+details: |
+    Older versions of snapd used to change the way seccomp profiles are handled, which could cause
+    problems when future snapd was reverted to old snapd, causing old profiles to
+    fail to parse and load. This was addressed with the introduction of a system
+    key and eventually with the switch to pre-compiled seccomp profiles. This
+    test uses a virtual machine to ensure that the core snap can be safely
+    reverted in case this is required for another reason.
+
 systems: [ubuntu-18.04-64]
 
 kill-timeout: 30m
@@ -35,7 +43,7 @@ execute: |
     remote.exec "sudo snap refresh --${NESTED_CORE_REFRESH_CHANNEL} core" || true
 
     if ! tests.nested wait-for ssh; then
-        echo "ssh not stablished, exiting..."
+        echo "ssh connection not established, exiting..."
         exit 1
     fi
 
@@ -50,7 +58,7 @@ execute: |
     remote.exec "sudo snap revert core" || true
 
     if ! tests.nested wait-for ssh; then
-        echo "ssh not stablished, exiting..."
+        echo "ssh connection not established, exiting..."
         exit 1
     fi
 

tests/nested/core/core20-kernel-failover/task.yaml

@@ -1,8 +1,8 @@
 summary: Check that a broken kernel snap automatically rolls itself back
 
 details: |
-  Check that when it is triggered an installation of a broken kernel (there are 7 different
-  kernel corruptions), the install change sinished with error and the initial kernel revision
+  Check the trigger for a broken kernel installation (there are 7 different
+  kernel corruptions) adds an install change error and that the initial kernel revision
   remains installed and we don't have leftover bootenv.
 
 # TODO:UC20: write equivalent test for base snap failover

tests/nested/core/hotplug/task.yaml

@@ -86,7 +86,7 @@ execute: |
     remote.exec "sudo snap connect serial-port-hotplug:serial-port :qemuusbserial"
     check_slot_connected qemuusbserial
 
-    echo "Veryfing serial-port permissions of the snap"
+    echo "Verifying serial-port permissions of the snap"
     verify_apparmor_profile "/dev/ttyUSB0"
 
     echo "Unplugging the device"
@@ -119,7 +119,7 @@ execute: |
     check_slot_connected qemuusbserial
     check_slot_not_gone qemuusbserial
 
-    echo "Veryfing serial-port permissions of the snap, the first device is now expected on ttyUSB1"
+    echo "Verifying serial-port permissions of the snap, the first device is now expected on ttyUSB1"
     check_slot_device_path qemuusbserial "/dev/ttyUSB1"
     verify_apparmor_profile "/dev/ttyUSB1"
 

tests/nested/core/image-build/task.yaml

@@ -1,5 +1,8 @@
 summary: create ubuntu-core image and execute the suite in a nested qemu instance
 
+details: |
+    This test runs several spread tests against a core16 image running in qemu.
+
 systems: [ubuntu-16.04-64]
 
 execute: |

tests/nested/manual/cloud-init-never-used-not-vuln/task.yaml

@@ -2,6 +2,12 @@ summary: |
     Test that cloud-init is no longer vulnerable on Ubuntu Core with the fix for
     CVE-2020-11933 in place.
 
+details: |
+    CVE-2020-11933 allowed anyone to present the device with cloud-info
+    meta-data, for example on a removable media, and reboot the machine to gain
+    elevated privileges. The test ensures that once snapd snap is refreshed to a
+    version which contains the fix, cloud-init gets disabled.
+
 systems: [ubuntu-18.04-64, ubuntu-16.04-64]
 
 environment:
@@ -69,7 +75,7 @@ execute: |
     remote.wait-for snap-command
     remote.exec "sudo snap wait system seed.loaded"
 
-    echo "Prepare snapd snapto install with the fix"
+    echo "Prepare snapd snap to install with the fix"
     # if we are not building from current, then we need to prep the snapd snap
     # to install with the fix, this simulates/verifies that devices in the field
     # without the fix will actually be fixed after they refresh

tests/nested/manual/cloud-init-nocloud-not-vuln/task.yaml

@@ -2,6 +2,12 @@ summary: |
     Test that cloud-init is no longer vulnerable on Ubuntu Core with the fix for
     CVE-2020-11933 in place with a system that used NoCloud configuration.
 
+details: |
+    CVE-2020-11933 allowed anyone to present the device with cloud-info
+    meta-data, for example on a removable media, and reboot the machine to gain
+    elevated privileges. The test ensures that cloud-init only runs on the first
+    boot.
+
 systems: [ubuntu-18.04-64, ubuntu-16.04-64]
 
 environment:

tests/nested/manual/cmdline-remove-append/task.yaml

@@ -1,5 +1,11 @@
 summary: Check that gadget.yaml can remove/add kernel command lines
 
+details: |
+  Gadget snaps have influence over the boot process, and subsequently, 
+  can control the kernel command line - a critical resource in many ways. This
+  test verifies that an updated gadget snap can effectively add or remove kernel
+  command line arguments.
+
 systems: [ubuntu-2*]
 
 prepare: |

tests/nested/manual/core-early-config/task.yaml

@@ -1,5 +1,10 @@
 summary: Test that config defaults are applied early when image is created.
 
+details: |
+    Snapd gadget snaps have influence over the default configuration of seeded
+    snaps. The test verifies that a gadget snap can effectively apply default
+    configuration settings to a system that is booted for the first time.
+
 # core18 specific test (and nested vm is derived from host system)
 systems: [ubuntu-18.04-64]
 

tests/nested/manual/core20-new-snapd-does-not-break-old-initrd/task.yaml

@@ -1,16 +1,23 @@
 summary: verify that new snapd's do not break old snap-bootstrap/kernel initrds
 
+details: |
+  The snapd secure boot implementation is complex, and some of the elements that live in the
+  snapd.git repository end up as a part of the signed kernel package's initrd
+  file. This inadvertently allows a skew to occur, where a future snapd is booted with a past
+  initrd.
+
+  The test exercises two scenarios:
+  
+  1. We start with stable kernel + stable snapd -> refresh to new snapd
+  2. We start with stable kernel + new snapd
+
+  In both cases we then trigger a reseal operation and reboot to make sure that
+  the old snap-bootstrap/initrd in the stable kernel can still unlock the
+  encrypted partitions
+
 # ubuntu-22.04-64: enable on uc22 once pc-kernel is on 22/candidate channel
 systems: [ubuntu-20.04-64]
 
-# we have two variants here:
-#
-# 1. we start with stable kernel + stable snapd -> refresh to new snapd
-# 2. we start with stable kernel + new snapd
-#
-# and then in both cases we then trigger a reseal operation and reboot to make
-# sure that the old snap-bootstrap/initrd in the stable kernel can still unlock
-# the encrypted partitions
 
 environment:
   NESTED_CUSTOM_MODEL: $TESTSLIB/assertions/ubuntu-core-{VERSION}-amd64.model

tests/nested/manual/core20-validation-sets/task.yaml

@@ -1,5 +1,11 @@
 summary: Verify that validation-sets are working and correctly tracked after seeding
 
+details: |
+    Snapd offers a way to ensure a good set of snaps is installed in a given
+    system, ensuring that refreshes move between one good set and another. This
+    system is known as validation sets. The test verifies that a validation set
+    is effective immediately after seeding.
+
 systems: [ubuntu-20.04-64]
 
 environment:

tests/nested/manual/devmode-snaps-can-run-other-snaps/task.yaml

@@ -2,8 +2,8 @@ summary: |
   Test that devmode confined snaps can execute other snaps.
 
 details: |
-  For xenial the test covers running core and non-core based devmode snaps from core based strict snap and
-  and for boinic the test covers running a non-core based devmode snap from a non-core based strict snap as
+  For Xenial, the test covers running core and non-core based devmode snaps from core based strict snaps. 
+  For Bionic, the test covers running a non-core based devmode snap from a non-core based strict snap as
   well as running core and non-core based devmode snaps from a core based strict snap.
 
 systems: [ubuntu-1*]

tests/nested/manual/gadget-connections/task.yaml

@@ -1,5 +1,10 @@
 summary: Check that connections from gadget are automatically connected
 
+details: |
+    Snapd gadget snaps have influence over the default connections of seeded
+    snaps. The test verifies that a gadget snap can effectively connect
+    interfaces on a system that is booted for the first time.
+
 systems: [ubuntu-20.04-64]
 
 environment:

tests/nightly/sbuild/task.yaml

@@ -1,9 +1,9 @@
 summary: Ensure snapd builds correctly in sbuild
 
 details: |
-    Debian package of snapd is sensitive to new dependencies that are not
-    provided as other Debian packages. We prefer to discover build issues during
-    the development process, and not during the final stages of release
+    The Debian package of snapd is sensitive to new dependencies that are not
+    provided as other Debian packages. We prefer to discover build issues
+    during the development process, and not during the final stages of release
     preparation.
 
     This nightly test builds the Debian package, using the packaging/debian-sid