add the canary test plan #674
Conversation
Codecov Report
@@ Coverage Diff @@
## main #674 +/- ##
======================================
Coverage ? 1.95%
======================================
Files ? 126
Lines ? 14950
Branches ? 2583
======================================
Hits ? 292
Misses ? 14595
Partials ? 63 |
There was a problem hiding this comment.
This test plan looks good. It covers:
- normal jobs
- template jobs
- template jobs using the jinja2 engine
- different kinds of reboot
- suspend/resume
- after-suspend jobs
It doesn't have any graphics-related job though; how about adding graphics/{index}_driver_version_{product_slug} (it's a before and after suspend job that spits out some driver info)? It would requires graphics_card resource job.
I also have a question inline.
| com.canonical.certification::power-management/warm-reboot | ||
| com.canonical.certification::power-management/warm-reboot | ||
| com.canonical.certification::power-management/post-warm-reboot | ||
| com.canonical.certification::power-management/cold-reboot | ||
| com.canonical.certification::power-management/cold-reboot | ||
| com.canonical.certification::power-management/post-cold-reboot |
There was a problem hiding this comment.
why are power-management/warm-reboot and power-management/cold-reboot run twice?
There was a problem hiding this comment.
That's my mistake stemming from the approach I took to generate the list. Removed in 604108b
Great idea. So here goes your first addition compared to what we had previously for edge testing (as it used IoT TP, so no GPU stuff). Added in d30661e. |
Description
This test plan is meant to run on Checkbox versions that are candidates for a release. The jobs included here are explicitly listed so the same jobs are run in each session. Moreover, the jobs picked here are ones that on a stable device should all pass if the Checkbox stack is ok. In other words, failure to successfully run all the tests contained here should signify that there is a bug in the Checkbox code.
Resolved issues
Resolves CHECKBOX-795