AX: Add a fast path for assistive technologies that search for all live regions and all non-root web areas on macOS #41898

twilco · 2025-03-05T03:24:37Z

`0420ca0`

AX: Add a fast path for assistive technologies that search for all live regions and all non-root web areas on macOS
https://bugs.webkit.org/show_bug.cgi?id=289137
rdar://146185640

Reviewed by Chris Fleizach.

VoiceOver commonly sends remote search requests for all the live regions, and all the non-woot web areas on the page.
Prior to this commit, this always required a full tree traversal. With this commit, we eagerly maintain a pre-sorted
list of these objects, allowing us to answer these queries immediately. There are generally very few of these objects
on a page, so it's cheap to keep this list up-to-date.

To avoid confusion, the existing AXObjectCache::m_liveRegionObjects list is renamed to AXObjectCache::m_changedLiveRegions,
as that is what it actually holds (live regions that have changed and are waiting for notifications to be sent for them).

* LayoutTests/accessibility/mac/live-region-search-expected.txt: Added.
* LayoutTests/accessibility/mac/live-region-search.html: Added.
* Source/WebCore/accessibility/AXCoreObject.h:
* Source/WebCore/accessibility/AXObjectCache.cpp:
(WebCore::AXObjectCache::remove):
(WebCore::AXObjectCache::handleLiveRegionCreated):
(WebCore::AXObjectCache::postLiveRegionChangeNotification):
(WebCore::AXObjectCache::liveRegionChangedNotificationPostTimerFired):
(WebCore::AXObjectCache::handleRoleChanged):
(WebCore::AXObjectCache::handleAttributeChange):
(WebCore::AXObjectCache::performDeferredCacheUpdate):
* Source/WebCore/accessibility/AXObjectCache.h:
* Source/WebCore/accessibility/AXSearchManager.cpp:
(WebCore::AXSearchManager::findMatchingObjectsInternal):
(WebCore::AXSearchManager::findMatchingRange):
(WebCore::AXSearchManager::findNextStartingFrom):
(WebCore::AXSearchManager::findAllMatchingObjectsIgnoringCache):
* Source/WebCore/accessibility/AXSearchManager.h:
* Source/WebCore/accessibility/AccessibilityObject.cpp:
(WebCore::AccessibilityObject::updateRole):
* Source/WebCore/accessibility/AccessibilityObject.h:
* Source/WebCore/accessibility/AccessibilityRenderObject.cpp:
(WebCore::AccessibilityRenderObject::updateRoleAfterChildrenCreation):
* Source/WebCore/accessibility/cocoa/AXCoreObjectCocoa.mm:
(WebCore::AXCoreObject::sortedDescendants const):
* Source/WebCore/accessibility/isolatedtree/AXIsolatedObject.h:
* Source/WebCore/accessibility/isolatedtree/AXIsolatedTree.cpp:
(WebCore::AXIsolatedTree::sortedLiveRegions):
(WebCore::AXIsolatedTree::sortedNonRootWebAreas):
(WebCore::AXIsolatedTree::applyPendingChanges):
(WebCore::AXIsolatedTree::sortedLiveRegionsDidChange):
(WebCore::AXIsolatedTree::sortedNonRootWebAreasDidChange):
* Source/WebCore/accessibility/isolatedtree/AXIsolatedTree.h:
* Source/WebCore/accessibility/isolatedtree/mac/AXIsolatedObjectMac.mm:
(WebCore::AXIsolatedObject::allSortedLiveRegions const):
(WebCore::AXIsolatedObject::allSortedNonRootWebAreas const):
* Source/WebCore/accessibility/mac/AXObjectCacheMac.mm:
(WebCore::AXObjectCache::onDocumentRenderTreeCreation):
(WebCore::AXObjectCache::platformPerformDeferredCacheUpdate):
(WebCore::AXObjectCache::sortedLiveRegions const):
(WebCore::AXObjectCache::sortedNonRootWebAreas const):
(WebCore::AXObjectCache::addSortedObject):
(WebCore::AXObjectCache::removeLiveRegion):
(WebCore::AXObjectCache::initializeSortedIDLists):
* Source/WebCore/accessibility/mac/AccessibilityObjectMac.mm:
(WebCore::AccessibilityObject::allSortedLiveRegions const):
(WebCore::AccessibilityObject::allSortedNonRootWebAreas const):
* Source/WebCore/dom/Document.cpp:
(WebCore::Document::createRenderTree):

Canonical link: https://commits.webkit.org/291652@main

b192947

Misc	iOS, visionOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 win
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	✅ 🧪 wpe-wk2	✅ 🧪 win-tests
✅ 🧪 webkitperl	✅ 🧪 ios-wk2	✅ 🧪 api-mac	✅ 🧪 api-wpe
	✅ 🧪 ios-wk2-wpt	✅ 🧪 mac-wk1	✅ 🛠 wpe-cairo
	✅ 🧪 api-ios	✅ 🧪 mac-wk2	✅ 🛠 gtk
	✅ 🛠 vision	✅ 🧪 mac-AS-debug-wk2	✅ 🧪 gtk-wk2
	✅ 🛠 vision-sim	✅ 🧪 mac-wk2-stress	✅ 🧪 api-gtk
✅ 🛠 🧪 merge	✅ 🧪 vision-wk2	✅ 🧪 mac-intel-wk2	✅ 🛠 playstation
	✅ 🛠 tv	✅ 🛠 mac-safer-cpp
	✅ 🛠 tv-sim
	✅ 🛠 watch
	✅ 🛠 watch-sim

webkit-early-warning-system · 2025-03-05T03:24:45Z

EWS run on previous version of this PR (hash ee7be21)

Misc	iOS, visionOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	~~🛠 ios~~	❌ 🛠 mac	✅ 🛠 wpe	✅ 🛠 win
✅ 🧪 bindings	✅ 🛠 ios-sim	❌ 🛠 mac-AS-debug	~~🧪 wpe-wk2~~	✅ 🧪 win-tests
✅ 🧪 webkitperl	⏳ 🧪 ios-wk2	❌ 🧪 api-mac	✅ 🧪 api-wpe
	✅ 🧪 ios-wk2-wpt	❌ 🧪 mac-wk1	✅ 🛠 wpe-cairo
	⏳ 🧪 api-ios	❌ 🧪 mac-wk2	✅ 🛠 gtk
	~~🛠 vision~~	❌ 🧪 mac-AS-debug-wk2	⏳ 🧪 gtk-wk2
	~~🛠 vision-sim~~	❌ 🧪 mac-wk2-stress	✅ 🧪 api-gtk
	⏳ 🧪 vision-wk2	❌ 🧪 mac-intel-wk2	~~🛠 playstation~~
	✅ 🛠 tv	~~🛠 mac-safer-cpp~~
	~~🛠 tv-sim~~
	~~🛠 watch~~
	~~🛠 watch-sim~~

webkit-early-warning-system · 2025-03-05T03:49:47Z

EWS run on previous version of this PR (hash de7ae33)

Misc	iOS, visionOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 win
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	✅ 🧪 wpe-wk2	✅ 🧪 win-tests
✅ 🧪 webkitperl	✅ 🧪 ios-wk2	✅ 🧪 api-mac	✅ 🧪 api-wpe
	✅ 🧪 ios-wk2-wpt	❌ 🧪 mac-wk1	✅ 🛠 wpe-cairo
	✅ 🧪 api-ios	❌ 🧪 mac-wk2	✅ 🛠 gtk
	✅ 🛠 vision	❌ 🧪 mac-AS-debug-wk2	✅ 🧪 gtk-wk2
	✅ 🛠 vision-sim	❌ 🧪 mac-wk2-stress	✅ 🧪 api-gtk
	✅ 🧪 vision-wk2	❌ 🧪 mac-intel-wk2	✅ 🛠 playstation
	✅ 🛠 tv	✅ 🛠 mac-safer-cpp
	✅ 🛠 tv-sim
	✅ 🛠 watch
	✅ 🛠 watch-sim

webkit-early-warning-system · 2025-03-05T15:57:28Z

EWS run on previous version of this PR (hash a682fee)

Misc	iOS, visionOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 win
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	❌ 🧪 wpe-wk2	✅ 🧪 win-tests
✅ 🧪 webkitperl	✅ 🧪 ios-wk2	✅ 🧪 api-mac	✅ 🧪 api-wpe
	✅ 🧪 ios-wk2-wpt	✅ 🧪 mac-wk1	✅ 🛠 wpe-cairo
	~~🧪 api-ios~~	~~🧪 mac-wk2~~	✅ 🛠 gtk
	✅ 🛠 vision	✅ 🧪 mac-AS-debug-wk2	✅ 🧪 gtk-wk2
	✅ 🛠 vision-sim	✅ 🧪 mac-wk2-stress	✅ 🧪 api-gtk
	✅ 🧪 vision-wk2	✅ 🧪 mac-intel-wk2	✅ 🛠 playstation
	✅ 🛠 tv	✅ 🛠 mac-safer-cpp
	✅ 🛠 tv-sim
	✅ 🛠 watch
	✅ 🛠 watch-sim

hoffmanjoshua · 2025-03-05T17:42:10Z

Source/WebCore/accessibility/AXObjectCache.cpp

Out of curiosity, what is the motivation for moving children changed up here?

We need to update the accessibility tree before any calls to addSortedObject, as otherwise we'll be trying to find the right place for a sorted object based on outdated accessibility tree state. This moves handling children-changed up above where we would potentially call addSortedObject.

hoffmanjoshua · 2025-03-05T17:53:33Z

Source/WebCore/accessibility/mac/AXObjectCacheMac.mm

This won't only apply to live regions, right?

Oops, right!

hoffmanjoshua · 2025-03-05T17:53:52Z

Source/WebCore/accessibility/mac/AXObjectCacheMac.mm

Same question as above, this isn't just for live regions, right?

webkit-early-warning-system · 2025-03-05T18:20:48Z

EWS run on current version of this PR (hash b192947)

Misc	iOS, visionOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 win
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	✅ 🧪 wpe-wk2	✅ 🧪 win-tests
✅ 🧪 webkitperl	✅ 🧪 ios-wk2	✅ 🧪 api-mac	✅ 🧪 api-wpe
	✅ 🧪 ios-wk2-wpt	✅ 🧪 mac-wk1	✅ 🛠 wpe-cairo
	✅ 🧪 api-ios	✅ 🧪 mac-wk2	✅ 🛠 gtk
	✅ 🛠 vision	✅ 🧪 mac-AS-debug-wk2	✅ 🧪 gtk-wk2
	✅ 🛠 vision-sim	✅ 🧪 mac-wk2-stress	✅ 🧪 api-gtk
✅ 🛠 🧪 merge	✅ 🧪 vision-wk2	✅ 🧪 mac-intel-wk2	✅ 🛠 playstation
	✅ 🛠 tv	✅ 🛠 mac-safer-cpp
	✅ 🛠 tv-sim
	✅ 🛠 watch
	✅ 🛠 watch-sim

…ve regions and all non-root web areas on macOS https://bugs.webkit.org/show_bug.cgi?id=289137 rdar://146185640 Reviewed by Chris Fleizach. VoiceOver commonly sends remote search requests for all the live regions, and all the non-woot web areas on the page. Prior to this commit, this always required a full tree traversal. With this commit, we eagerly maintain a pre-sorted list of these objects, allowing us to answer these queries immediately. There are generally very few of these objects on a page, so it's cheap to keep this list up-to-date. To avoid confusion, the existing AXObjectCache::m_liveRegionObjects list is renamed to AXObjectCache::m_changedLiveRegions, as that is what it actually holds (live regions that have changed and are waiting for notifications to be sent for them). * LayoutTests/accessibility/mac/live-region-search-expected.txt: Added. * LayoutTests/accessibility/mac/live-region-search.html: Added. * Source/WebCore/accessibility/AXCoreObject.h: * Source/WebCore/accessibility/AXObjectCache.cpp: (WebCore::AXObjectCache::remove): (WebCore::AXObjectCache::handleLiveRegionCreated): (WebCore::AXObjectCache::postLiveRegionChangeNotification): (WebCore::AXObjectCache::liveRegionChangedNotificationPostTimerFired): (WebCore::AXObjectCache::handleRoleChanged): (WebCore::AXObjectCache::handleAttributeChange): (WebCore::AXObjectCache::performDeferredCacheUpdate): * Source/WebCore/accessibility/AXObjectCache.h: * Source/WebCore/accessibility/AXSearchManager.cpp: (WebCore::AXSearchManager::findMatchingObjectsInternal): (WebCore::AXSearchManager::findMatchingRange): (WebCore::AXSearchManager::findNextStartingFrom): (WebCore::AXSearchManager::findAllMatchingObjectsIgnoringCache): * Source/WebCore/accessibility/AXSearchManager.h: * Source/WebCore/accessibility/AccessibilityObject.cpp: (WebCore::AccessibilityObject::updateRole): * Source/WebCore/accessibility/AccessibilityObject.h: * Source/WebCore/accessibility/AccessibilityRenderObject.cpp: (WebCore::AccessibilityRenderObject::updateRoleAfterChildrenCreation): * Source/WebCore/accessibility/cocoa/AXCoreObjectCocoa.mm: (WebCore::AXCoreObject::sortedDescendants const): * Source/WebCore/accessibility/isolatedtree/AXIsolatedObject.h: * Source/WebCore/accessibility/isolatedtree/AXIsolatedTree.cpp: (WebCore::AXIsolatedTree::sortedLiveRegions): (WebCore::AXIsolatedTree::sortedNonRootWebAreas): (WebCore::AXIsolatedTree::applyPendingChanges): (WebCore::AXIsolatedTree::sortedLiveRegionsDidChange): (WebCore::AXIsolatedTree::sortedNonRootWebAreasDidChange): * Source/WebCore/accessibility/isolatedtree/AXIsolatedTree.h: * Source/WebCore/accessibility/isolatedtree/mac/AXIsolatedObjectMac.mm: (WebCore::AXIsolatedObject::allSortedLiveRegions const): (WebCore::AXIsolatedObject::allSortedNonRootWebAreas const): * Source/WebCore/accessibility/mac/AXObjectCacheMac.mm: (WebCore::AXObjectCache::onDocumentRenderTreeCreation): (WebCore::AXObjectCache::platformPerformDeferredCacheUpdate): (WebCore::AXObjectCache::sortedLiveRegions const): (WebCore::AXObjectCache::sortedNonRootWebAreas const): (WebCore::AXObjectCache::addSortedObject): (WebCore::AXObjectCache::removeLiveRegion): (WebCore::AXObjectCache::initializeSortedIDLists): * Source/WebCore/accessibility/mac/AccessibilityObjectMac.mm: (WebCore::AccessibilityObject::allSortedLiveRegions const): (WebCore::AccessibilityObject::allSortedNonRootWebAreas const): * Source/WebCore/dom/Document.cpp: (WebCore::Document::createRenderTree): Canonical link: https://commits.webkit.org/291652@main

webkit-commit-queue · 2025-03-05T22:28:20Z

Committed 291652@main (0420ca0): https://commits.webkit.org/291652@main

Reviewed commits have been landed. Closing PR #41898 and removing active labels.

twilco requested review from cdumez and rniwa as code owners March 5, 2025 03:24

twilco self-assigned this Mar 5, 2025

twilco added the Accessibility For bugs related to accessibility. label Mar 5, 2025

webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Mar 5, 2025

twilco force-pushed the eng/AX-Add-a-fast-path-for-assistive-technologies-that-search-for-all-live-regions-and-all-non-root-web-areas-on-macOS branch from ee7be21 to de7ae33 Compare March 5, 2025 03:49

twilco force-pushed the eng/AX-Add-a-fast-path-for-assistive-technologies-that-search-for-all-live-regions-and-all-non-root-web-areas-on-macOS branch from de7ae33 to a682fee Compare March 5, 2025 15:57

hoffmanjoshua reviewed Mar 5, 2025

View reviewed changes

twilco force-pushed the eng/AX-Add-a-fast-path-for-assistive-technologies-that-search-for-all-live-regions-and-all-non-root-web-areas-on-macOS branch from a682fee to b192947 Compare March 5, 2025 18:20

fleizach approved these changes Mar 5, 2025

View reviewed changes

twilco added merge-queue Applied to send a pull request to merge-queue and removed merging-blocked Applied to prevent a change from being merged labels Mar 5, 2025

webkit-commit-queue force-pushed the eng/AX-Add-a-fast-path-for-assistive-technologies-that-search-for-all-live-regions-and-all-non-root-web-areas-on-macOS branch from b192947 to 0420ca0 Compare March 5, 2025 22:28

webkit-commit-queue merged commit 0420ca0 into WebKit:main Mar 5, 2025

webkit-commit-queue removed the merge-queue Applied to send a pull request to merge-queue label Mar 5, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

AX: Add a fast path for assistive technologies that search for all live regions and all non-root web areas on macOS #41898

AX: Add a fast path for assistive technologies that search for all live regions and all non-root web areas on macOS #41898

Uh oh!

twilco commented Mar 5, 2025 •

edited by webkit-early-warning-system

Loading

Uh oh!

webkit-early-warning-system commented Mar 5, 2025 •

edited

Loading

Uh oh!

webkit-early-warning-system commented Mar 5, 2025 •

edited

Loading

Uh oh!

webkit-early-warning-system commented Mar 5, 2025 •

edited

Loading

Uh oh!

hoffmanjoshua Mar 5, 2025

Uh oh!

twilco Mar 5, 2025

Uh oh!

hoffmanjoshua Mar 5, 2025

Uh oh!

twilco Mar 5, 2025

Uh oh!

hoffmanjoshua Mar 5, 2025

Uh oh!

webkit-early-warning-system commented Mar 5, 2025 •

edited

Loading

Uh oh!

webkit-commit-queue commented Mar 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

AX: Add a fast path for assistive technologies that search for all live regions and all non-root web areas on macOS #41898

AX: Add a fast path for assistive technologies that search for all live regions and all non-root web areas on macOS #41898

Uh oh!

Conversation

twilco commented Mar 5, 2025 • edited by webkit-early-warning-system Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

0420ca0

Uh oh!

webkit-early-warning-system commented Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

webkit-early-warning-system commented Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

webkit-early-warning-system commented Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hoffmanjoshua Mar 5, 2025

Choose a reason for hiding this comment

Uh oh!

twilco Mar 5, 2025

Choose a reason for hiding this comment

Uh oh!

hoffmanjoshua Mar 5, 2025

Choose a reason for hiding this comment

Uh oh!

twilco Mar 5, 2025

Choose a reason for hiding this comment

Uh oh!

hoffmanjoshua Mar 5, 2025

Choose a reason for hiding this comment

Uh oh!

webkit-early-warning-system commented Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

webkit-commit-queue commented Mar 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

twilco commented Mar 5, 2025 •

edited by webkit-early-warning-system

Loading

`0420ca0`

webkit-early-warning-system commented Mar 5, 2025 •

edited

Loading

webkit-early-warning-system commented Mar 5, 2025 •

edited

Loading

webkit-early-warning-system commented Mar 5, 2025 •

edited

Loading

webkit-early-warning-system commented Mar 5, 2025 •

edited

Loading