Slider noises play continuously with iOS VO

[KatieWoe]

Device
iPad (2018)
iOS
12.2
Browser
Safari
Problem Description
For phetsims/qa#313
When opening the PhET menu it is possible for the slider sounds to play continuously
Steps to reproduce

1. Open sim fresh with VO
2. Move both sliders up and down a bit
3. Open PhET menu

Screenshots
https://drive.google.com/file/d/1YLSejrYu5GfiArUCjDYijp970P8r5T44/view?usp=sharing

[jessegreenberg]

I am testing on an iPad Air 2 iOS 12.2. After trying for ~12 minutes I haven't been able to see this happen. I will try on my iPad 2.

[jessegreenberg]

I was just able to reproduce this, but I am not sure how yet. I moved the voltage slider up to 5.5V then opened the PhET menu and heard continuous sound.

[jessegreenberg]

Now it is happening every single time!

[jessegreenberg]

OK, I figured out how to make this happen consistently. If the PhET menu is clicked before the CurrentSoundGenerator has a chance to set fadeCountdownTimer to Number.NEGATIVE_INFINITY, the problem doesn't happen. But if it is clicked after that, the problem occurs. That means as long as you wait ~5 seconds to click the PhET menu the problem will happen every time (on fresh reload).

[jessegreenberg]

I suspect this is happening due to the way scenery "enables" sound on the first pointer event and how CurrentSoundGenerator fades out sound, but I am not certain how it all works.

[jessegreenberg]

So this is not related to the PhET menu, but the activation of Sound. I can also reproduce by moving a slider, waiting 5 seconds, then double tapping the sim. The double tap sends a pointer event to the sim which enables sound.

[terracoda]

Nice detective work, @jessegreenberg.

[jessegreenberg]

I created the above two issues to investigate this further in tambo, a solution will be explored there, possibly with scenery changes as well.

[jbphet]

I'm able to duplicate this on the PhET iPad "Pauling" consistently. It's running iOS version 11.4.1. One thing that I noticed that doesn't seem to be noted above is that sound is not playing at all prior to the time that this occurs, so that would seem to lend credence to @jessegreenberg's observation in one of the comments above where he says that this problem is likely due to the activation of sound.

[jbphet]

The problem is that Voice Over is intercepting touch events, and the code in soundManager is not able to receive the touchstart event that it usually uses to resume the audio context. I looked at this with @jessegreenberg, since he knows the a11y input structure pretty well, and we determined that if we added an emit for scenery.Display.userGestureEmitter to changeAction in scener/input/Input.js, and then listened for emits as well as touchstart on iOS devices, this can be used to resume the audio context.

I don't know if this is a reasonable thing to do. I spoke with @jonathanolson over Slack on the topic, and he had some thoughts, so I'm going to assign this to him so that he can record them and help me figure out what to try next.

[jonathanolson]

That seems reasonable to me.

[jbphet]

Thanks @jonathanolson.

I've implemented the fix on master. It involves commits to both tambo and scenery, so branches with the needed changes will be needed to be pulled into the Ohm's Law release branch. Assigning to @jessegreenberg for that, since there may be other changes that he needs to pick up.

[jbphet]

For the record, and in case I ever need to look at this again, I did do some investigation of why this was happening when the volume should have been at zero. The summary is this: In this particular browser, setting output levels on gain nodes while the audio context is suspended doesn't seem to work. So, the CurrentSoundGenerator instance executes all the code to fade out the sound, but setting the gain value to zero before the audio context is running doesn't work, so as soon as the audio context is running, the sound is produced. It may be possible to prevent this by fully stopping the loop when the output level goes to zero, but since we have a fix for the bigger picture issue, I'm not going to take the time to do that right now.

[jessegreenberg]

Excellent, thanks @jbphet, branches created in scenery and tambo, dependencies.json updated.

[jessegreenberg]

I also tested to verify this was fixed with iOS 12 VoiceOver + Safari.

[KatieWoe]

This issue seems solved in 1.4.0-rc.3