Skip to content
This repository was archived by the owner on Feb 25, 2025. It is now read-only.

T23358 unknown status fixup#125

Merged
gctucker merged 19 commits intokernelci:masterfrom
gctucker:T23358-unknown-status-fixup
Oct 2, 2020
Merged

T23358 unknown status fixup#125
gctucker merged 19 commits intokernelci:masterfrom
gctucker:T23358-unknown-status-fixup

Conversation

@gctucker
Copy link
Copy Markdown
Collaborator

@gctucker gctucker commented Sep 10, 2020

Fix the status of test cases that have always failed, to show them as failures rather than "unknown". Also update a bunch of related things as a side effect, including:

  • show number of builds with warnings rather than "unknown" in badge counters
  • show number of test failures that aren't regressions rather than "unknown" in badge counters
  • drop total number of builds and tests to simplify UI with 3 badges rather than add one for "unknown" and have 5 badges
  • show failures for test cases that have always failed in orange, and unknown (skipped...) in grey
  • update pie charts with total number and builds with warnings / failed tests that aren't regressions + actual unknowns
  • update test plan status to show "fail" in orange when there are failures but no regressions

@gctucker gctucker requested a review from a team September 10, 2020 12:46
@gctucker gctucker force-pushed the T23358-unknown-status-fixup branch 3 times, most recently from 9cac3a0 to 55ac7c6 Compare September 14, 2020 23:44
@gctucker gctucker force-pushed the T23358-unknown-status-fixup branch 11 times, most recently from 4d85b63 to 425f28a Compare September 25, 2020 21:19
Following the refactoring of test views, a number of functions in
table/test.js aren't used any more so they can now be removed.

Signed-off-by: Guillaume Tucker <guillaume.tucker@collabora.com>
Align all badge counters to use "pass", "fail", "warning" and "total"
to map to green, red, yellow and grey.  That way, each table can now
define which counters to show.  Also simplify the logic in
tables/common.js to iterate over an array of nodes.

Signed-off-by: Guillaume Tucker <guillaume.tucker@collabora.com>
The "unknown" badge counter is not useful for builds, in fact it's
virtually never used in practice.  Instead, show the number of builds
that passed with at least one warning in the orange counter.  The
green counters now only show builds that passed with no warnings to
avoid counting some builds twice.

Also drop the total number of builds as it would not be equal to the
pass/warning/error totals if there were any unknown ones.  This makes
the view simpler with 3 counters rather than 5.

Signed-off-by: Guillaume Tucker <guillaume.tucker@collabora.com>
Rather than showing the number of "unknown" test cases, in the orange
count badges, show the number that failed but didn't cause a
regression.

Also drop the total number of test cases as it's not that useful and
would now be confusing with the number of unknown results not being
shown (like with build counts, to have 3 badges rather than 5).

Signed-off-by: Guillaume Tucker <guillaume.tucker@collabora.com>
Drop the "offline" status as it's not used anywhere.  It was once used
to show that some test platforms are offline, but this is not
something the KernelCI database keeps track of any more.

Signed-off-by: Guillaume Tucker <guillaume.tucker@collabora.com>
Reduce code duplication by making several html functions reuse
html._status().  This makes it easier to maintain and adjust the
different types of status.

Signed-off-by: Guillaume Tucker <guillaume.tucker@collabora.com>
Add the "warning" status and add html.warning() to create it.  Show
"warning" in amber and convert "unknown" to grey.

Signed-off-by: Guillaume Tucker <guillaume.tucker@collabora.com>
Use 4 different status instead of 3 for the pie charts: success,
warning, failure, unkonwn (green, amber, red, grey).  Update the
default function to count the results accordingly using the "WARNING"
status, and simplify syntax in some places.

Signed-off-by: Guillaume Tucker <guillaume.tucker@collabora.com>
Treat builds with warnings as a separate category, with a new button
and the html.warning() status icon.

Signed-off-by: Guillaume Tucker <guillaume.tucker@collabora.com>
Use the new "warning" status when showing a single build details, if
the build passed but had warnings.

Signed-off-by: Guillaume Tucker <guillaume.tucker@collabora.com>
Update the table showing test counts on the detailed view for a single
build, to align it with the top-level jobs pages (pass / failures /
regressions).

Signed-off-by: Guillaume Tucker <guillaume.tucker@collabora.com>
Update the status for each top-level test plan for a given kernel
revision, showing a warning when there were failures but no
regressions.  Update the pie chart accordingly, with total and
pass/fail/regression/unknown test counts.

Signed-off-by: Guillaume Tucker <guillaume.tucker@collabora.com>
Update the view showing the test plan runs to show failures that are
not regressions as warnings, with an extra button to be able to filter
them.  Also update the pie chart with the new 4 status
(pass/fail/regression/unkonwn).

Signed-off-by: Guillaume Tucker <guillaume.tucker@collabora.com>
@gctucker gctucker force-pushed the T23358-unknown-status-fixup branch 2 times, most recently from 69880bc to 1a588f2 Compare September 30, 2020 21:05
Update the per-lab test result summaries to show pass/fail/regression
numbers as per the rest of the UI.

Signed-off-by: Guillaume Tucker <guillaume.tucker@collabora.com>
Update the view to show the summary of test case results for a given
test plan run, using "warning" for test cases that have always failed
and "unknown" for others such as skipped.  Also update the pie chart
accordingly.

Signed-off-by: Guillaume Tucker <guillaume.tucker@collabora.com>
When a test case has always failed, show the status as a "warning" to
distinguish it from regressions and unknown (skipped).

Signed-off-by: Guillaume Tucker <guillaume.tucker@collabora.com>
Update the socs-soc / job / kernel views with failures as warnings to
align them with the jobs views.

Also add missing semicolons to pass eslint.

Signed-off-by: Guillaume Tucker <guillaume.tucker@collabora.com>
Update the socs-soc-job-kernel-plan view to show a warning sign when a
test plan run has some failures but no regressions, to align it with
the corresponding main test view.

Signed-off-by: Guillaume Tucker <guillaume.tucker@collabora.com>
Show the number of pass / fail / regression tests in each lab on the
socs-soc-job-kernel-plan view to align it with the corresponding main
test view.

Signed-off-by: Guillaume Tucker <guillaume.tucker@collabora.com>
@gctucker gctucker force-pushed the T23358-unknown-status-fixup branch from 1a588f2 to 52b4434 Compare September 30, 2020 21:52
@gctucker gctucker marked this pull request as ready for review September 30, 2020 22:04
@gctucker gctucker merged commit 4a92ef6 into kernelci:master Oct 2, 2020
@gctucker gctucker deleted the T23358-unknown-status-fixup branch October 2, 2020 08:59
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant