Support V2 test format when displaying reports

[mcking65]

Assuming that #733 and #737 are complete, The V2 test format drives only a few more changes to reports.

Change command heading text to include settings

In the implementation of #733, the command result tables are preceded by a heading that has text:

COMMAND_NAME Results: X passed, Y failed

Example:

Down Arrow then Down Arrow Results: 4 passed, 2 failed

As specified for the headings that precede assertion tables in w3c/aria-at#977, if settings were specified for a command, include the settings in parentheses. If there are multiple settings, separate them with the string " and ".

Example:

Down Arrow then Down Arrow (browse mode on) Results: 4 passed, 2 failed

Command results table content changes

Update the table caption to match heading content.

In the assertion column, render the phrase form of the assertion specified in the assertionPhrase column of assertions.csv.

Change row sorting to:

1. priority descending: MUST then SHOULD then MAY
2. Within each priority group, sort with failures first then passes.
3. Within fail/pass groups, sort in assertion order specified for the test in the assertions column of tests.csv.

Changes to heading text on report pages

In the implementation of #733 on report pages where the set of command results tables for a test is preceded by a level 2 heading with text:

TEST_NAME (X passed, y failed)

Example:

Trigger an alert (2 passed, 2 failed)

Add the string "Test TEST_NUMBER: " to the beginning of the heading text. For example:

Test 1: Trigger an alert (2 passed, 2 failed)

Mockup

Here is a zip of an HTML file containing a mockup for VoiceOver results of 1 test:
TestResultsTableMockupForIssue744.zip

[mcking65]

I will be adding implementing option B from w3c/aria-at#999 into this issue as discussed in CG meeting.

[lolaodelola]

@ccanash (& relevant internal engineer) for internal review, this has been updated to the following format:

Option B
Add a new level 3 heading:

Results for each command

Then Change the level of the heading for each command to 4, keeping the same text as today:

B Results: 2 passed, 0 failed

Note, there is another level 3 heading for each test, "Run History", so there would be two level 3 headings for each test, one for results by command and one for run history.

See the issue that Matt linked to above and item 1 in the CG minutes

[ccanash]

Hi @mcking65 this was included in dec 5th release. Can you please confirm if it's ok to close?

[mcking65]

@ccanash @howard-e

There is one requirement in the specs for this issue that has not been completed. Under the heading "Command results table content changes", it says:

In the assertion column, render the phrase form of the assertion specified in the assertionPhrase column of assertions.csv.

The table still shows the statement form of the assertion instead of the phrase form of the assertion. The phrase form is intended to make the table content easier to understand when reading across a row. For example, instead of:

Priority	Assertion	Result
Must	Name 'W3C website' is conveyed	Passed

The table should read:

Priority	Assertion	Result
Must	Convey name 'W3C website'	Passed

[mcking65]

@ccanash

Please do one of the following:

1. Move this issue back to a status of in progress.
2. Open a new bug issue using my comment, give it a status of to-do in the same project, and close this issue.

[ccanash]

Hi @mcking65 , have created #949