Skip to content

Commit

Permalink
Risks update
Browse files Browse the repository at this point in the history
  • Loading branch information
DanielRyanSmith committed Jun 24, 2022
1 parent 2cf2d8b commit af7872e
Showing 1 changed file with 8 additions and 9 deletions.
17 changes: 8 additions & 9 deletions rfcs/test_scoring_change.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,19 +53,18 @@ Passing percentage for directories containing multiple tests =
`sum of passing percentages of tests in directory / total number of tests in directory`

This will be implemented by adding a new query parameter, `view`. This can have
two valid values: `subtest` and `interop`. `subtest` will represent the default
view that exists on wpt.fyi. `interop` will display a view that more closely
mirrors the Interop-20** scores.
two valid values: `subtest` and `interop`. `subtest` will represent the current
view that exists on wpt.fyi and will be enabled by default. `interop` will
display a view that more closely mirrors the Interop-20** scores.

**Note**: In the interest of avoiding scored comparisons being visible outside
the tests that have been agreed upon for Interop-20**, this `interop` view will
only be available when viewing tests with an `Interop-20**` test label. The
`interop` view will not function outside of results without this label.

## Risks
* This change will cause a net loss in the percentage of passing
tests/subtests, as the harness status passing has been artificially inflating
some subtest numbers.
* This change will require a rework of the current process for aggregating and
summarizing test results, so older test runs will still be aggregated with the
harness status and display inflated passing percentages.
* This change will cause a discrepancy in the number of subtests when comparing
runs with old summary files and new summary files together, as the harness
status passing has been artificially inflating some subtest numbers. This is a
rework of the current process for aggregating and summarizing test results,
so older test runs will still be aggregated with the harness status.

0 comments on commit af7872e

Please sign in to comment.