(mobile a11y) - I hear night sky, flashlight, spot light, sun

[terracoda]

I just listened to RaP for the first time on my iPhone7 (iOS 14.2) to test out device dependent help text from #86.
Unrelated to the help text, I heard a couple of funny things:

On the Home page at the start of the list of sim screen buttons, I heard:

• "one, list start night sky", then discover screen, button & description
• "two, night sky" , then create screen button & description

In Navbar on the Discover Screen, when I navigate to the sim screens, I also hear some strange content.

• 1, flashlight, spotlight, sun Home Screen button
• 2, flashlight, spotlight, sun Discover Screen button
• 3, flashlight, spotlight, sun Discover Screen button

@zepumph, when you get a chance, could you have a look to see where these words are coming from?

[zepumph]

On my samsung with TalkBack I don't hear this. Everything sounds normal. @KatieWoe can you reproduce this on an iPhone?

@terracoda it would also be helpful if you can answer a couple of questions:

1. What link did you use to test this? If a dev version please post.
2. Can you reproduce this on master right now? https://bayes.colorado.edu/dev/phettest/ratio-and-proportion/ratio-and-proportion_en.html?brand=phet
3. If reproduceable, are there any particular voice over settings that might change this behavior that you can think of?

[terracoda]

Ok, on the link above, I hear "night sky" on the Home Screen at the beginning of the list, but I no longer hear "flash light" and "spot light" on any screens.

[zepumph]

Is this is safari? Can you try in Chrome?

[zepumph]

Actually, @KatieWoe, I can reproduce this on my iPad, so I'll unassign you for now. Sorry for confusion.

[terracoda]

With VoiceOver and iOS VoiceOver, it is best to use Safari.

[terracoda]

I don't have Chrome installed on my iPhone7 :-) I am indeed using Safari, iOS 14.2.

[terracoda]

@zepumph, I can try the link on iOS 13.5.1 I haven't updated the iPhone8.

[zepumph]

I will try to create a dedicated HTML snippet to reproduce this on. "Night Sky" after the list number on the the home screen buttons happens for me consistently on iPad in the regular sim, but not in the a11y view.

I won't get to this today though.

[terracoda]

@zepumph, RaP is loading from PhETTest. When you have a moment, could you post a dev link for to try out.

[zepumph]

This has unvetted flipped description for the ratio hands from #245 et all, so take it with a grain of salt.

https://phet-dev.colorado.edu/html/ratio-and-proportion/1.0.0-dev.74/phet/

[terracoda]

I'll get to this today :-)

[terracoda]

In the latest version on master, using phettest on bayes, I now only hear "night sky" on the home page.

I am using iOS 14.2 and Safari on iphone 7.

Next, I will now test on iOS 13.5.1 and Safari on iphone 8.

[terracoda]

hmm, the sim is not loading on my iPhone 8. I'll have to check back again later.

[terracoda]

@zepumph, I do not hear "night sky" when using 13.5.1 on iPhone8. These strange strings might be additional issue related to iOS 14?

[KatieWoe]

For phetsims/qa#582
I've been hearing "night sky" seemingly when there are bullets or buttons being read. I noticed in Faraday's Law that it said "sky" so I'm wondering if background color has something to do with it?

[zepumph]

I cannot seem to reproduce this in a standalone way. When I copy the static HTML out of the sim, navigating to the first list item sounds like:

"Discover Screen, Button, List start, Explore this screen to discover set cahllenge ratios". This is all as a single line, and the discover screen is highlighted. This is swiping right from the top of my static HTML page. This is when I copied the ENTIRE HTML of the sim, not just the PDOM. I get the same results with just the PDOM.

With a subsection of static HTML, I set the background color to be "blue", and there was no difference in how it was described.

When I do this in the sim (swiping right from the top). I get two separate items:

"1, List Start, Night Sky" . . . Then swiping again
"Discover Screen, Button, Explore this screen to discover set challenge ratios.

Something about the actual running of the sim makes VO behave differently. I really thought I would have been able to find an isolated case here.

I am interested in two things from here.

1. How important is this bug? How much more time should I devote to it (perhaps for @emily-phet to decide)
2. Does @jessegreenberg have any other ideas about how I might be able to reproduce this outside of a sim context.

[terracoda]

@KatieWoe mentioned she heard "sky" in Faraday's Law. I am curious where.
I have only heard these strange utterances on the Home Page and in the Navbar, not from within the actual sim screen.

@zepumph, I'd like to test the latest version of the sim again on my two iOS devices. The sim is not loading on Bayes PhET Test, so I need a new dev version at your convenience.

[terracoda]

And regarding your questions in #256 (comment), I have a feeling we will have to mark this bug as a won't fix (after I test the latest version).

That said, I think we need to remain vigilant on this issue to try and find the source. I haven't found any related information by googling. It's got to be some strange iOS 14 bug.

[terracoda]

I found one comment on the applevis forum that indicates that these words might be related to Apple's new image detection/description feature.

Please see: https://www.applevis.com/blog/apple-releases-ios-141-and-ipados-141-voiceover-fixes-and-support-new-iphone-models

It's a strange and erratic bug, I was just testing dev version 78 (latest that I have). The bug does not exist on iOS 13.5.1 (I think I have already noted that).

And when I just tested again on iOS 14.2, the first few times I read through the screen buttons on the home screen, I heard "night sky", but after returning to the home screen from screen 1, I could not get the sim to say "night sky". I tired a page refresh and a reload of the dev link and still not "night sky".

Also, I do not hear any strange content on the buttons in the bottom navbar.

I think this bug is low priority.

The other bugs created by iOS 14 bugs are much more serious.

[zepumph]

The sim is not loading on Bayes PhET Test

Can you please describe your issue? On what devices? What does "not loading" mean as it pertains to the PhET Sim, its splash screen, and its loading bar. I just found that https://bayes.colorado.edu/dev/phettest/ratio-and-proportion/ratio-and-proportion_en.html?ea&brand=phet loaded just fine for me on my chrome/windows.

[terracoda]

It's not loading on my phones.
I get the PhET splash page forever after loading the sim.
The loading bar flashes, but never starts filling.
I don't know how to troubleshoot on my phone ;-)
I can add more detail tomorrow.

[terracoda]

I'm using Safari.

[KatieWoe]

I know bayes has been slower than our old phettest. Not sure if that's the problem, or it's something else, but sometimes just refreshing the page if it's being slow can help.

[terracoda]

I still hear these strange little phrases when using mobile VO. Recently hear "Day Time" when I loaded Faraday's Law.
I don't think the phrases get terribly in the way, ad they definitely seem to be an iOS VoiceOver bug.

[zepumph]

@terracoda and I worked on this again. @terracoda led the conversation towards feeling like they were image descriptions. Using that, and a guide here as inspiration, we were able to find a setting in VO that controls "image description". Though now on iOS 14.3 the setting is under "verbosity". When turning off that setting, I am unable to reproduce this bug. Thus it seems like these words are VoiceOver's poor attempts at interpreting our graphical content. Good investigative work @terracoda!

We confirmed that all non-PDOM, graphical elements in the sim had aria-hidden="true" on their HTML elements. Still this doesn't keep iOS from saying these descriptions. I'm not sure why iOS doesn't ignore them when they are buried behind aria-hidden. No matter, I will add this to our a11y bugs doc and move on.

Tagging @KatieWoe to note the conclusion, let any other QA know as needed (do we have an a11y-focused QA person these days?) and to close if all looks good.

[KatieWoe]

Thanks for the heads up. Great work!