Accessibility module in WebDriver BiDi?

[OrKoN]

Puppeteer exposes the information about the browser's a11y tree in a couple of ways:

1. via Accessibility.getFullAXTree to generate a snapshot of the a11y tree
2. via Accessibility.queryAXTree to query elements by the accessible name and role. This allows Puppeteer to implement element selectors based on the a11y tree data.

Additionally, there were requests recently to include A11y tree information into WebDriver https://goo.gle/chromedriver-a11ytree. I think we should discuss if we want to add these capabilities to WebDriver BiDi.

[christian-bromann]

One could argue that a very popular use case is to fetch an element via a11y name. WebdriverIO currently computes some complex xPath to make a good guess about it according to the spec but it doesn't work all time due to xPath limitations. It the would help a lot if this could be a primitive, e.g. a selector strategy.

[clshortfuse]

I would love for this to be supported. Had a request posted in the interop issue with reasonings for it:

Currently, I would like to ensure a Web Component presents a role at a certain point in the layout tree. For example:

<x-button role=none>
  #shadowroot
  <input type=button aria-labelledby=slot>
  <slot id=slot></slot>
</x-button>

And

<x-button role=button>
  #shadowroot
  <slot id=slot></slot>
</x-button>

<x-button> <!-- role set via ElementInternals -->
  #shadowroot
  <slot id=slot></slot>
</x-button>

It would make sense that the next "meaningful" ARIA role starting from <x-button> is button. But there are multiple ways to achieve this, including hiding the attribute via ElementInternals. I think I might be able to iterate the DOM and test each element manually, but I'm worried about closed shadowRoots. I have concerns with just iterating the DOM because I'm not really sure what's being exposed by the browsers to screen-readers. There might be groups or hidden nodes. The DOM and AXTree isn't guaranteed to be 1:1.

There are many web testing suites that just read the DOM, which isn't accurate. Manually testing in the browser is somewhat tedious, but at this point has helped me recognize some browser bugs with computed labels:

• https://bugzilla.mozilla.org/show_bug.cgi?id=1826194
• https://bugs.webkit.org/show_bug.cgi?id=254934

I am also aware of other AXLabel bugs, but haven't had time to file them. But most of them deal in one way or another with complex trees.

2. Use of a tree allows expanding testing beyond just label and role.

I recently found a mistake on my part where I had a custom element like above, but using <input type=checkbox>. I used aria-checked=mixed manually, assuming browsers did not apply .indeterminate as aria-checked=mixed. I probably had to do this with IE and it's just something I stuck with. When I used a tester (Playwright) that exposes their own version of an AXTree, I learned of this Chrome bug:

• https://bugs.chromium.org/p/chromium/issues/detail?id=1260371

So, after having checked the aria tree with its properties, I was able to rewrite some code around make sure all the 3 majors browsers properly show aria-checked=mixed

Related note (Option 4). I did write something a while ago to emulate a screen reader over a virtual COM port to test JAWS/NVDA, but getting it to work on anything but Windows was too hard of a task for me at the time so I abandoned it. The Webdriver solution seems good enough for me now.

Hi all. Are there any interest in an outputted tree in addition to the role/label systems?

1. The use of a tree will allow testing a role/label is present at certain point, not a certain node.

Currently, I would like to ensure a Web Component presents a role at a certain point in the layout tree. For example:

<x-button role=none>
  #shadowroot
  <input type=button aria-labelledby=slot>
  <slot id=slot></slot>
</x-button>

And

<x-button role=button>
  #shadowroot
  <slot id=slot></slot>
</x-button>

<x-button> <!-- role set via ElementInternals -->
  #shadowroot
  <slot id=slot></slot>
</x-button>

It would make sense that the next "meaningful" ARIA role starting from <x-button> is button. But there are multiple ways to achieve this, including hiding the attribute via ElementInternals. I think I might be able to iterate the DOM and test each element manually, but I'm worried about closed shadowRoots. I have concerns with just iterating the DOM because I'm not really sure what's being exposed by the browsers to screen-readers. There might be groups or hidden nodes. The DOM and AXTree isn't guaranteed to be 1:1.

There are many web testing suites that just read the DOM, which isn't accurate. Manually testing in the browser is somewhat tedious, but at this point has helped me recognize some browser bugs with computed labels:

• https://bugzilla.mozilla.org/show_bug.cgi?id=1826194
• https://bugs.webkit.org/show_bug.cgi?id=254934

I am also aware of other AXLabel bugs, but haven't had time to file them. But most of them deal in one way or another with complex trees.

2. Use of a tree allows expanding testing beyond just label and role.

I recently found a mistake on my part where I had a custom element like above, but using <input type=checkbox>. I used aria-checked=mixed manually, assuming browsers did not apply .indeterminate as aria-checked=mixed. I probably had to do this with IE and it's just something I stuck with. When I used a tester (Playwright) that exposes their own version of an AXTree, I learned of this Chrome bug:

• https://bugs.chromium.org/p/chromium/issues/detail?id=1260371

So, after having checked the aria tree with its properties, I was able to rewrite some code around make sure all the 3 majors browsers properly show aria-checked=mixed

Related note (Option 4). I did write something a while ago to emulate a screen reader over a virtual COM port to test JAWS/NVDA, but getting it to work on anything but Windows was too hard of a task for me at the time so I abandoned it. The Webdriver solution seems good enough for me now.

Originally posted by @clshortfuse in web-platform-tests/interop-accessibility#2 (comment)