Feedback: Toggle Button Test plan, V23.12.14: Shouldn't role be 'Toggle Button' instead of 'button'

[mcking65]

Description of Behavior

The assertion for 'role' is for role 'button', but the role in the accessibility API that should be conveyed is 'toggle button'.

Test Setup

• Test File: test-05-navForwardsToNotPressedToggleButton-nvda.collected.html
• AT: NVDA (Version 2023.3)
• Browser: Chrome (Version 120.0.6099.217)

[css-meeting-bot]

The ARIA-AT Community Group just discussed Issue 1031: Toggle Button Test plan, V23.12.14: Shouldn't role be 'Toggle Button' instead of 'button'.

The full IRC log of that discussion

<jugglinmike> Subtopic: Issue 1031: Toggle Button Test plan, V23.12.14: Shouldn't role be 'Toggle Button' instead of 'button'
<jugglinmike> github: https://github.com//issues/1031
<jugglinmike> Matt_King: This is a question about the "role" assertion
<jugglinmike> Matt_King: Right now the toggle button plan says that "role 'button'" is conveyed
<jugglinmike> Matt_King: Joe marked that as "passing," but when I looked at both JAWS and NVDA, the role that's coming across is "toggle button"
<jugglinmike> Matt_King: I would consider it a failure if I came to a toggle button and the screen reader just said "button"
<jugglinmike> Joe_Humbert: Apple doesn't have a non-selected state. I passed it because it said "button" and when it was selected, it had both a role and a state
<jugglinmike> Matt_King: I think this is a problem with a test. I think the assertion should be "toggle button"
<jugglinmike> Alyssa: but if it said "button pressed" or "button not pressed", then you could assume it was a toggle button
<jugglinmike> Alyssa: is the average consumer going to care if if it says "toggle button" as long as it also says the state?
<jugglinmike> Matt_King: The problem is that VoiceOver doesn't speak the "not pressed" state, so for a toggle button which is not pressed, it only says "button"
<jugglinmike> Matt_King: When I look at the ARIA specification, all of them say that the role that gets sent to the accessibility API is "toggle button" which is distinct from "button"
<jugglinmike> Joe_Humbert: I just verified: VoiceOver says "toggle button"
<jugglinmike> Matt_King: But what should we expect the screen reader to do?
<jugglinmike> Alyssa: It should say "toggle button"
<jugglinmike> Hadi: my interaction would be very different if I found a "toggle button" rather than a "button"
<jugglinmike> Joe_Humbert: most buttons conventionally don't have state
<jugglinmike> Matt_King: When I hear "button not pressed", I think "well, of course, all buttons are not pressed"
<jugglinmike> Joe_Humbert: There's another conversation about the difference between a button and a key
<jugglinmike> Joe_Humbert: A lot of buttons don't "bounce back." Keys, on the other hand
<jugglinmike> s/hand/hand.../
<jugglinmike> Joe_Humbert: Also, keep in mind that iOS doesn't have radio buttons
<jugglinmike> Joe_Humbert: It sounds like we're all in agreement that the test needs to change. The expected role should be "toggle button", not "button"
<jugglinmike> Matt_King: I think the test should be about what the screen reader is supposed to do. It should communicate the role in the Core-AAM, which in this case is "toggle button"
<jugglinmike> Matt_King: I think this just hasn't been documented properly because there are only two roles in all of ARIA which are like this: "menu button" and "toggle button"
<jugglinmike> Matt_King: I'll assign this issue to myself, and I'll update the test (that means we'll have to re-run the tests)

[css-meeting-bot]

The ARIA-AT Community Group just discussed Toggle button plan: set date to resolve.

The full IRC log of that discussion

<jugglinmike> Subtopic: Toggle button plan: set date to resolve
<jugglinmike> github: https://github.com//issues/1031
<jugglinmike> Matt_King: Given the consenses we reached last time, this is a one-word change to one test plan
<jugglinmike> Matt_King: Because the assertion changes, verdicts for any tests which include role will become invalid
<jugglinmike> IsaDC: All assertions that say "toggle button", I believe
<jugglinmike> Matt_King: Let's set a schedule for figuring this out
<jugglinmike> Matt_King: I will assign this re-assign this from me to IsaDC
<jugglinmike> s/assign this //
<jugglinmike> James_Scholes: We're doing "toggle button" as two words, correct?
<jugglinmike> IsaDC: Yes
<jugglinmike> Matt_King: Yes
<jugglinmike> Matt_King: So that new version of the "toggle button" test plan won't be available until you make the change, merge it, and advance the test from "R&D" to "review"
<jugglinmike> Matt_King: The only time we have the option to preserve data is if there is an incomplete run in the test queue
<jugglinmike> Matt_King: It seems like we'd want a way to copy all data for tests which haven't changed whenever we add a new test plan to the test queue. Then again, that seems like something that ought to be done automatically
<jugglinmike> James_Scholes: I guess it depends on the complexity of the changes between the two test plan versions
<jugglinmike> Matt_King: I don't know where that fits in with our roadmap
<jugglinmike> Matt_King: For now, though, for anybody assigned to NVDA with Chrome, we do have the bot available
<jugglinmike> Matt_King: And as far as I can tell, it's 100% reliable. I have not found a case where the NVDA bot gives me different output, and I've done this a number of times for three different test plans
<jugglinmike> jugglinmike: I can't say when Firefox support will be available, but I can see about getting an estimate for next week
<jugglinmike> IsaDC: Would anyone like to re-run all of "toggle button" from scratch?
<jugglinmike> Joe_Humbert: I can re-run the ones that I originally submitted--VoiceOver and Safari
<jugglinmike> Matt_King: A two-week timeframe would be great. That's because my next meeting with Vispero is three weeks from yesterday
<jugglinmike> Hadi: I volunteer to run in Chrome and JAWS again
<jugglinmike> [Matt_King reiterates the rationale for re-running the "toggle button" tests]
<jugglinmike> howard-e: Copying data in that way is currently possible, actually, at least for the Test Plan Reports which have not been marked "final"
<jugglinmike> Michael_Fairchild: in order to save time, could we ask the database administrators to manually copy the test results?
<jugglinmike> howard-e: That's feasible, but I'm also saying that there's a high chance that the data for the unchanged tests will be copied automatically
<jugglinmike> Matt_King: It sounds like the plan--first we get the new version. IsaDC does that.
<jugglinmike> Matt_King: That gets moved to Draft Review
<jugglinmike> Matt_King: Actually, howard-e, before she does that, she would deploy the latest changes in staging (following my completing my functional testing)?
<jugglinmike> Matt_King: If we deployed that, and then IsaDC does her part, then we would see some results carry forward automatically
<jugglinmike> Matt_King: After that, should we have howard-e try to manually copy some of Joe_Humbert's test results forward?
<jugglinmike> James_Scholes: I think we should. It would take multiple hours to re-run this test
<jugglinmike> James_Scholes: Should we add the new version to the queue now? Or should we do that later?
<jugglinmike> [Matt_King investigates the current behavior of the application]
<jugglinmike> Matt_King: IsaDC can make the change, and she can merge it, but do not advance it to draft review until howard-e has updated the app!
<jugglinmike> Matt_King: And howard-e will not update the app until Monday
<jugglinmike> IsaDC: Got it
<jugglinmike> James_Scholes: Should I un-assign you from JAWS and NVDA, Matt_King?
<jugglinmike> Matt_King: Sure
<jugglinmike> James_Scholes: Got it (because we already have results from IsaDC and Hadi)
<jugglinmike> s/because we already/I asked to un-assign you because we already/
<jugglinmike> Matt_King: There are some problems with this feature, and I've raised an issue about them
<jugglinmike> Matt_King: It's related to the "trend reporting" project that Bocoup is going to be working on next quarter
<jugglinmike> James_Scholes: the status says "draft", and that means there are no conflicts (because if there were conflicts, the status would describe them)
<jugglinmike> Matt_King: The order of events:
<jugglinmike> Matt_King: 1 - IsaDC is going to merge a new version of the test plan
<jugglinmike> Matt_King: 2 - I am going to test what's in staging and give the green light to howard-e
<jugglinmike> Matt_King: 3 - howard-e will release the changes to production
<jugglinmike> Matt_King: 4 - IsaDC can advance the new version from R&D to Draft Review
<jugglinmike> James_Scholes: after the new version is advanced, it doesn't automatically appear in the test queue, does it?
<jugglinmike> Matt_King: Yes, it does
<jugglinmike> Matt_King: The status of the new (February 23rd) version, will show "5 of 8 complete" (or something similar)
<jugglinmike> James_Scholes: that all sounds good. It does leave a feature result outstanding, though: to copy results from a test plan with a different status
<jugglinmike> Matt_King: Yes, that's right
<jugglinmike> Matt_King: We have an issue open for that, and it's related to the "trend reporting" work