Investigate test case expectation discrepancies

[joanmarie]

Test case expectation discrepancies

1. Name test case 543. Becase "Reset" is the displayed text on the push button, and there is no other name source, "Reset" should be exposed as the accessible name. WhatSock's tool shows the empty string as the calculated accessible name. Please verify that the spec includes this scenario in its algorithm and that the error is in the tool.

2. Name test cases 548, 733, 734, 735, 736, and 737. (These are essentially the same test, but with a different input type.) Our expectation is "crazy clown"; WhatSock's tool shows "crazy beard". Please verify the tool is correct according to the algorithm in the spec.

3. Name test case 600. Our expectation is the empty string; WhatSock's tool shows "Div with text". In this particular case, I think our expectation is correct. We do not want divs (or paragraphs or any other non interactive, textual element) to have accessible name calculated from content. Many divs, etc. have a bunch of text. And at least some screen readers will present the accessible name if there is one. Imagine arrowing down into a huge div/paragraph and having the screen reader spew out its full contents.

4. Name test cases 661, 662, 663, and 663a. (These are essentially the same test, but with a different input type.) Our expectation is a name of "foo bar baz"; WhatSock's tool is a name of "foobaz" and a description of "bar". I'll add a description test for the same markup. In the meantime, please verify the tool is correct according to the algorithm in the spec.

5. Name test cases checkbox-label-embedded-combobox, file-label-embedded-combobox, password-label-embedded-combobox, radio-label-embedded-combobox, text-label-embedded-combobox. (These are essentially the same test, but with a different input type.) Our expectation is a name of "Flash the screen 1 times." WhatSock's tool is a name of "Flash the screen 1 2 3 times." Here the numbers come from the list item text, and only the item with "1" has aria-selected="true". Is this scenario covered by the algorithm? If not, it should be. If it is, please verify that the tool has the correct expectations.

6. Name test cases checkbox-label-embedded-menu, file-label-embedded-menu, password-label-embedded-menu, radio-label-embedded-menu, text-label-embedded-menu. (These are essentially the same test, but with a different input type.) Our expectation is a name of "Flash the screen 1 times." WhatSock's tool is a name of "Flash the screen times." This is a variant of the previous test case. Determining whether or not our algorithm covers inclusion of aria-selected="true" in the name calculation is needed.

[accdc]

Hi, The fixes have been uploaded, and all of your reported issues should now be fixed! Except for one that is, which hopefully we can talk about at the next call, which is the use of embedded menus plus aria-selected. This is presenting me with a problem because aria-selected is not supported on role=menuitem. From: joanmarie [mailto:notifications@github.com] Sent: Thursday, February 22, 2018 6:17 AM To: WhatSock/w3c-alternative-text-computation <w3c-alternative-text-computation@noreply.github.com> Cc: Subscribed <subscribed@noreply.github.com> Subject: [WhatSock/w3c-alternative-text-computation] Investigate test case expectation discrepancies (#6) Test case expectation discrepancies 1. Name test case 543. Becase "Reset" is the displayed text on the push button, and there is no other name source, "Reset" should be exposed as the accessible name. WhatSock's tool shows the empty string as the calculated accessible name. Please verify that the spec includes this scenario in its algorithm and that the error is in the tool. 2. Name test cases 548, 733, 734, 735, 736, and 737. (These are essentially the same test, but with a different input type.) Our expectation is "crazy clown"; WhatSock's tool shows "crazy beard". Please verify the tool is correct according to the algorithm in the spec. 3. Name test case 600. Our expectation is the empty string; WhatSock's tool shows "Div with text". In this particular case, I think our expectation is correct. We do not want divs (or paragraphs or any other non interactive, textual element) to have accessible name calculated from content. Many divs, etc. have a bunch of text. And at least some screen readers will present the accessible name if there is one. Imagine arrowing down into a huge div/paragraph and having the screen reader spew out its full contents. 4. Name test cases 661, 662, 663, and 663a. (These are essentially the same test, but with a different input type.) Our expectation is a name of "foo bar baz"; WhatSock's tool is a name of "foobaz" and a description of "bar". I'll add a description test for the same markup. In the meantime, please verify the tool is correct according to the algorithm in the spec. 5. Name test cases checkbox-label-embedded-combobox, file-label-embedded-combobox, password-label-embedded-combobox, radio-label-embedded-combobox, text-label-embedded-combobox. (These are essentially the same test, but with a different input type.) Our expectation is a name of "Flash the screen 1 times." WhatSock's tool is a name of "Flash the screen 1 2 3 times." Here the numbers come from the list item text, and only the item with "1" has aria-selected="true". Is this scenario covered by the algorithm? If not, it should be. If it is, please verify that the tool has the correct expectations. 6. Name test cases checkbox-label-embedded-menu, file-label-embedded-menu, password-label-embedded-menu, radio-label-embedded-menu, text-label-embedded-menu. (These are essentially the same test, but with a different input type.) Our expectation is a name of "Flash the screen 1 times." WhatSock's tool is a name of "Flash the screen times." This is a variant of the previous test case. Determining whether or not our algorithm covers inclusion of aria-selected="true" in the name calculation is needed. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#6> , or mute the thread <https://github.com/notifications/unsubscribe-auth/ABx1aFGdkoIug2YTyjLsyk6zrgEcP7oFks5tXXbygaJpZM4SPYhc> . <https://github.com/notifications/beacon/ABx1aFZch2Pp-dzAdKABqI1-CXkQMVGDks5tXXbygaJpZM4SPYhc.gif>

[joanmarie]

I think that everything in this issue has been addressed. I've opened new issues for things where the discrepancy is between the test expectations and the implementations and my initial finding is that the implementation is correct and that the expectations (WG and WhatSock) are wrong. As a result, I am going to close this as fixed. Thanks!