[docs] Remove recommendations on test types #18261
Conversation
Preferences for test types differ among test maintainers, so the documentation should not make specific recommendations about this decision. Present the test types with more neutral language. This phrasing more closely aligns with following paragraph which describes preferences between test types.
This comment has been minimized.
This comment has been minimized.
| consist of two or more pages with assertions as to whether they render | ||
| identically or not. | ||
|
|
||
| * [Visual tests](visual) should be used for checking rendering where there is | ||
| a large number of conforming renderings such that reftests are impractical. | ||
| * [Visual tests](visual) are great for checking rendering where there is a |
There was a problem hiding this comment.
(They're never great. They're practically manual tests. Maybe we should just list them down by manual?)
| testing everything else. They are built with the testharness.js unit testing | ||
| framework, and consist of assertions written in JavaScript. | ||
| * [testharness.js](testharness) tests can be used to test most everything else | ||
| (a few exceptions follow). They are built with the testharness.js unit |
There was a problem hiding this comment.
I don't think we really give exceptions following, let's just cut that clause out.
There was a problem hiding this comment.
wdspec tests and manual tests are the exceptions I had in mind.
We could move this item to the end, but we're already stating that, "Manual tests are used as a last resort for anything that can't be tested using any of the above." We can only get away with using an unqualified "anything else" in the final item. We'll have to be more specific about one of these two.
If we change the structure like that, we should also update To be honest, I can't advise on the documentation of visual tests because I don't understand how they differ from manual tests. That's a problem with the documentation which I hoped to address separately, but maybe we should take it on first. How are visual tests authored? Are the expectations expressed in terms of reference documents or screen shots? Where are those expectations stored? Essentially, if a first-time contributor to WPT were to write a visual test, what would they need to do? This might be a hassle to explain in text. I'd be happy to have a call with you if that would be easier! |
Almost identically to manual tests; the only extra requirement is they render to a fixed state.
The tests need to be self-describing, so that from a screenshot you can tell if it passes or fails. The screenshots have to be per-browser and per-platform (these are comparable to
Wherever the runner choses.
|
|
Closing due to no action in the past year. If there's a desire to still land this from someone, please reopen and rebase it. |
Preferences for test types differ among test maintainers, so the
documentation should not make specific recommendations about this
decision.
Present the test types with more neutral language. This phrasing more
closely aligns with following paragraph which describes preferences
between test types.
This should resolve gh-16602.