* Installing checkbox tutorial

* Minor: typo

* Review feedback

* Reference required by install snap

Minor: fixed links in install

* Forgotten snap reference file

* Fixed spelling

Minor: sorted wordlist.txt in alphabetical order, to do the same
  you can use the following in vim: ":%!sort"

* Change default/encouraged version to classic

* Implemented docs feedback

Uppercase C in checkbox
Link to installation in reference
Ubuntu is not code
blob -> blog
Uppercase S in snapcraft
CI expanded to Continuous Integration
Typos and grammar

* Add section in the release process about sending release e-mails

* Add comments on release e-mail

* Fix Markdown typo

* Change sample release note

* Use semver example in the tag release section

* Run black on lxd_proviuder and machine

* Install metabox provider in overlay

This installs the metabox provider in an overlay that is
automounted to the checkbox snap at boot. This is done to
avoid sideloading but also having a persistent install
of the metabox provider to be persistent across reboots.
Also, this uses the default loading mechanism of the snap
and it is closer to what it will do "in the wild"

* Document how to test snaps in README.md

* Minor: reformatted wonky indentation

* Fixed extra command in cmd

* Force snapcraft.yaml for s16 to install deps

There does not seem to be another way to install the dependencies
using the pyproject key. This is due to the fact that there is no
pip/setuptools/other backend that seem to support it on such an old
version of python

* Force also metabox source runs to install deps

Minor: python may not be python3, call python3

* New test plan check installation

* Dependency test script for metabox-provider

* Add metabox scenario to validate installation

* Renamed all jobs to the same name

* Removed test plan

* Tag test

* Minor: removed empty line

* Collect coverage of checkbox-ng and update source .coveragerc

Minor: Sort and update exclusions

* Collect coverage of checkbox-support and update source .coveragerc

Minor: Sort and update exclusions

* New coveragerc for providers

* Collect coverage of providers bins

Minor: Always call python via the correct env name in tox

* Minor: Fixed rotated dependencies

* Show missing lines in cli report

Minor: Fixed spacing

* Generate xml report for codecov.io

* Add coverage upload to tox action

* Add code coverage requirements

* Document coverage policy and howto

* Implement review feedback

* Init unittests for cmd_session.py

* Init unittests for deprecated modules

* Init unittests for inv_session.py

* Init unittests for remote_assistant.py

* Added missing dep to tox.ini

* Downgrade from assert_called from assertTrue(called)

Minor: python3.5 does not support assert_called

* Added new test for master invoke

Minor: Fixed typo in banner

* Added test for slave invoke

* Add unit test to mergers

* Add unit test to check config

* Add unit test to provider tools

* Add unit test to checkbox_cli

* Make tests compatible with ancient python

- Mock os.path.join for py<3.8
- Mock ArgumentParser for py<3.8
- Remove all assert_called_once as it is 3.6+

* Flagged unittests

Flagging is basically a way to give a category to unittests (grouping
them together)

* Enable carryover of all test runs

Given that we have a monorepo, we do not run all tests every time.
carryover makes codecov use any previous run as an oracle if a
submission is missing coverage data for an existing flag

* Renamed all providers with provider- prefix

#674)

* add a canary test plan

* add graphics job

* remove the repeated reboot jobs

* Make the remote api message clearer

* Tweaked wording

* Extract api version checking to function

* Test the new remote_api checking function

* Review feedback

* Fix: Assign bitrate and loopback on for Socketcan

Two changes are made in this commit.
1. For most of devices with CAN hardware component, the CAN interface
   cannot be brought up if the bitrate is not given. Therefore,
   assign the bitrate for eff and sff local socketCAN tests like what
   the remote socketCAN tests do.
2. For local socketCAN test, loopback is the must need parameter be
   assigned before bringing the CAN interface up.

* Update the bitrate to be 1,000,000

This should never have run via remote because:
- the configs on the controller are not taken into account
- the agent runs as root, so configs for the `ubuntu` user are pointless

There is more that we can improve in this test, but this patch at least
fixes the metabox runs.

* Upgrade with effective coverage tips

* Minor: Fixed typos

* Add reminder for coverage in the pr template

* add return code metabox scenario

* fix: checkbox remote now returns 1 on any test failed

* fixup! add return code metabox scenario

* forward return val from the main remote function

* remove useless break

* Minor: run black, formatting, import style

* Fixed failing checkbox run should return 1

* Canary documentation for starting release and how it works

* Explain versioning and how to tag

* Minor: better tag for beta promotion

* Add guideline for PR titles to PR template

* Review feedback

* Removed acronims as per feedback

* Add "Running first test plan" tutorial section

* Modify resource job unit documentation page

* Modify documentation glossary (test plan, job and resrouce sections)

* Add Wrapping up section in the "Running" tutorial

* Adding headers to the Running test plan tutorial

* Fixes based on feedback

* Fix typo

Co-authored-by: tang-mm <tang.mmeng@gmail.com>

---------

Co-authored-by: tang-mm <tang.mmeng@gmail.com>

* Add section about `check-config` in the config explanation

* Add section about config files in the Launcher tutorial

Instead of writing a whole separate tutorial section about config files,
a section is created within the tutorial about launchers. This section
briefly touches on config files and point to the existing explanation
for more information.

Fix CHECKBOX-752

* Add prompt to distinguish commands and output

* Replace username with more generic example

* Address feedback

* add empty tutorial provider

* add tutorial jobs that end in 3 different outcomes

* add additional units for the tutorial provider

* Add tutorial job that uses an environment variable

* Add readme for the tutorial provider

* Split tutorial provider name and namespace

* Amend existing tutorial pages to point to the newly created tutorial test plan

* Add tutorial provider to the checkbox22 snap

The base tutorial uses jobs and test plan from this provider as part of
the introduction to Checkbox, so it needs to be present in the snap.

* Add a Tutorial category

* Tutorial provider: update existing jobs and add manual and semi-auto jobs

* Tutorial provider: add interactive jobs to the test plan

---------

Co-authored-by: Pierre Equoy <pierre.equoy@canonical.com>

* Rename base tutorial rst files

* Change rst markup of the test report tutorial

Switch to the same markup used for other rst files, otherwise the
generated HTML page does not behave the same (no links at the bottom to
the previous/next section).

* Update Tutorial table of content

* Add support page pointing to our mailing list

* Update "Project and community" section of the index

* Update index of the tutorial section

Fix CHECKBOX-791

* Black Changes

* Changed version naming for all lp builds

* fix `pip install -e .` for focal (older pip)

PIP, when run in editable mode requires the projects to have `setup.py`,
it cannot do it just with setup.cfg and pyproject.toml.

This patch makes it so checkbox can be deployed and used from source.

* remove dynamic setup.py generation

* Refine Canary test plan

Remove a few jobs from the Canary test plan.

Some jobs are meant to be run on desktop images and do not run well (or
at all) on Ubuntu Core:

- the graphics-related ones
- i2c/i2c-device-detect (although it could technically run on a desktop
image, it requires a connected I²C device connected to run properly)

On the other hand, some jobs are meant to run on UC and not on desktop
images:

- ubuntu_core_features
- kernel-snap/booted-kernel-matches-current-grub
- parts_meta_info_attachment

A lot of the snap-related tests make requests to the snap store, and
it's preferable to avoid this to avoid false negative in the case where
the snap store is not responding.

The watchdog tests are meant to be run only on devices where the
watchdog device is enabled and properly setup. Watchdog is not part of
the coverage for laptops and desktops.

This commit also renames the pxu file to be aligned with the convention
used for other units in this provider.

Fix CHECKBOX-837

* Remove location/status job from canary test plan

Remove location job (and its after-suspend sibling) as it relies on a
snap (locationd) that we do not support anymore and that is therefore
not installed. As a result, the jobs will always be skipped.

* Remove disk/storage_device_.* test from canary test plan

Stress test that takes a long time to run and does not bring any benefit
in terms of testing Checkbox proper.

* Upgrade pip and setuptools before snapcraft_install

Minor: remove standalone pip install (use the snapcraft_install)
Minor: removed hack to create setup.py
Minor: Only stage provider as the provider install part

* Minor: Document why we remove pip after checkbox-support build

Minor: Removed commented out dependencies (moved in build)

* Moved all python dependencies in the same part

Minor: fixed all python paths to the correct version
Minor: remove outdated hack

* Updated comments and recovered dependency

* Recovered (miss)removed build-packages

Update rules to translate from dpkg to PEP440 version

This removes the (automatically included) -ubuntu or ~ubuntu
version postfix (optional from dpkg but violating PEP440) and
translates the dev annotation (~dev for dpkg) to a valid
PEP440 annotation (.dev)

Limit tox run to the env of the current matrix

Fix wrong name added to to_check