Issue reading menu items with VoiceOver on IOS and OSX

[bartbunting]

Checklist

• I have updated to the latest available Home Assistant version.
• I have cleared the cache of my browser.
• I have tried a different browser to see if it is related to my browser.

Describe the issue you are experiencing

When navigating to a menu for example Settings with VoiceOver enabled the menu items are not read correctly. On IOS either in the native home assistant app or in Safari they are read as "menu item" on OSX Safari they are read as "Group".
This used to work correctly. Unfortunately I am uncertain of the exact version change that caused the regression.
It may have been the change from IOS 16 to 17 or a change in home assistant. Either way the entire settings menu is now unusable.

Describe the behavior you expected

I expect each menu item to be read by VoiceOver with its name e.g. Integrations etc.

Steps to reproduce the issue

...Turn on VoiceOver on an iPhone running IOS 17 or later. This can be done from settings -> accessibility -> VoiceOver. (It may be convenient to enable the triple tap option to toggle VoiceOver) alternatively you can ask Siri to turn it on and off.
Go to the settings menu in home assistant. Notice that when you touch each menu item it is not read correctly.

What version of Home Assistant Core has the issue?

2023-11-3

What was the last working version of Home Assistant Core?

No response

In which browser are you experiencing the issue with?

safari in IOS 17.1 and Safari in OSX 14.1.1

Which operating system are you using to run this browser?

No response

State of relevant entities

No response

Problem-relevant frontend configuration

No response

Javascript errors shown in your browser console/inspector

No response

Additional information

No response

[bartbunting]

I have also tried using Google Chrome on iOS. Same problem. Although my understanding is it uses the same rendering engine. So not surprising I guess.
Firefox on OS X does not have the issue. Have not been able to test with Google Chrome on OS X yet but will do so shortly. Suspect it will not have the issue as chrome on windows does not.

[steverep]

I just noticed this as well, and I think something in WebKit 17 broke it. I'll look into it.

[steverep]

Still researching, but I can confirm nothing has changed in HA to cause this. It's pretty clearly a regression in Webkit and/or VoiceOver. It seems to affect the accessible name of anything inside a shadow DOM slot, so it also affects accordion buttons (e.g. trigger or action headings in the automation editor).

Here are some recent related Webkit bugs I've found so far:

• https://bugs.webkit.org/show_bug.cgi?id=260772
• https://bugs.webkit.org/show_bug.cgi?id=254934

Haven't found one that pinpoints this just yet.

In the meantime, the (albeit very annoying) workaround would be to use VoiceOver's screen recognition feature temporarily to read those items.

[steverep]

Okay I believe I found an existing WebKit bug for this at https://bugs.webkit.org/show_bug.cgi?id=264410. It's already been fixed but only merged a few days ago so it's unclear when and what Safari versions will get the fix.

That bug definitely applies to the expansion/accordion panels, but I'm not 100% certain it applies to the menu and combo-box situation. I've inquired about it on the issue.

[steverep]

Unfortunately, the fix for that last bug I referenced landed in 17.2 and still did not fix the issues. I had to file several more bugs with Apple:

• Slotted accessible name affected by CSS flex display
• Accessible name in shadow slot fails in shadow DOM
• Default contents of child slot are not used for accessible name

[steverep]

Those 3 bugs I filed were fixed today by Apple so I'm closing this for now. I'll comment again when I know what Safari version has the fix.

[bartbunting]

Does not appear to be fixed in iOS 17.3

[steverep]

Does not appear to be fixed in iOS 17.3

Nope. Just got a reply from Apple that it should hopefully be in the next preview version:

https://bugs.webkit.org/show_bug.cgi?id=266916#c9

[steverep]

If you have a Mac, you should be able to test the fix in the current TP version:

https://webkit.org/blog/14931/release-notes-for-safari-technology-preview-187/

[steverep]

Fixes are in iOS 17.4 and everything appears to be working again.

[bartbunting]

I can confirm this.There seems to be some extra chatting us in some dialogues which I thought was a similar or the same issue but apparently not.Will try and figure out how to describe it more specificallyRegardsBartOn 7 Mar 2024, at 12:44 AM, Steve Repsher ***@***.***> wrote: Fixes are in iOS 17.4 and everything appears to be working again. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>