Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Voicing does not work with Chromebook #328

Closed
Tracked by #899
Nancy-Salpepi opened this issue Jan 27, 2023 · 10 comments
Closed
Tracked by #899

Voicing does not work with Chromebook #328

Nancy-Salpepi opened this issue Jan 27, 2023 · 10 comments

Comments

@Nancy-Salpepi
Copy link

Test device
Chromebook

Operating System
109.0.5414.94

Browser
chrome

Problem description
For phetsims/qa#886, Voicing doesn't work for Friction using the Chromebook. When I click on things I see the little megaphone appear in the sim tab, but I don't hear anything.
Voicing only seems to be working in published GFLB and JT (not in published RaP). It didn't work for any of the sims I tried in master (Friction, RaP, GFLB).

Voicing works with mac + chrome and win10 + chrome.

Troubleshooting information: !!!!! DO NOT EDIT !!!!! Name: ‪Friction‬ URL: https://phet-dev.colorado.edu/html/friction/1.6.0-rc.1/phet/friction_all_phet.html Version: 1.6.0-rc.1 2023-01-17 16:32:22 UTC Features missing: applicationcache, applicationcache, touch Flags: pixelRatioScaling User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36 Language: en-US Window: 1536x781 Pixel Ratio: 1.7999999523162842/1 WebGL: WebGL 1.0 (OpenGL ES 2.0 Chromium) GLSL: WebGL GLSL ES 1.0 (OpenGL ES GLSL ES 1.0 Chromium) Vendor: WebKit (WebKit WebGL) Vertex: attribs: 16 varying: 31 uniform: 1024 Texture: size: 16384 imageUnits: 16 (vertex: 16, combined: 32) Max viewport: 16384x16384 OES_texture_float: true Dependencies JSON: {}
@jessegreenberg
Copy link
Contributor

I have a chromebook, I can take a look.

@jessegreenberg
Copy link
Contributor

jessegreenberg commented Jan 27, 2023

I just tried 1.6.0-rc.1 on a chromebook 104.0.5112.83 (Official Build) (64 bit) and heard Voicing come through.
Then I upgraded to 105.0.5195.134 (Official Build) (64 bit) and still heard Voicing come through.
Then I upgraded to 109.0.5414.94 (Official Build) (64 bit) and all Voicing was broken. Something about Chromebook changed.

EDIT: I am seeing that Voicing on latest ChromeOS works ~5% of the time. Usually I hear nothing but sometimes I do hear "Voicing On" or the content from a reading block.

EDIT: I confirmed it does seem better in published JT. Though there was once or twice where I didn't hear any content.

EDIT: When we do not hear anything, window.speechSynthesis.speaking is stuck true. I tried to call speechSynthesis.cancel and it did not change the value of speaking. Exact same behavior as phetsims/a11y-research#183.

EDIT: Here is the output from using the debug option of UtteranceQueue
image

Not ready to announce because synth is stuck "speaking".

I made this demo HTML:

<!DOCTYPE html>
<html>
<head>
  <meta charset="UTF-8">
  <title>TEST PAGE</title>
</head>
<body>
  <button id="button">Speak!</button>
</body>

<script>

  const button = document.getElementById( 'button' );
  button.addEventListener( 'click', event => {
    speechSynthesis.speak( new SpeechSynthesisUtterance( 'Hey there' ) );
  } );

</script>
</html>

At first I didn't hear anything. Then I closed all other sim tabs and I started to hear the "Hey there" every button press.

@jessegreenberg
Copy link
Contributor

I found that if I remove the workaround added in phetsims/gravity-force-lab-basics#303 the problem goes away. The problem from that issue does come back though.

Specifically:
https://github.com/phetsims/utterance-queue/blob/2f64e9580771087750bdc3f1c5ac778abb089f1f/js/SpeechSynthesisAnnouncer.ts#L448-L457

I believe that the times Voicing was working on Chrome 109 were the times I started to use it before the workaround was used.

I was curious why this issue is NOT in published GFL:B, even the workaround references an issue in that sim. But it is explained by this comment: phetsims/gravity-force-lab-basics#303 (comment)

I tried putting a string in the workaround like

synth.speak( new SpeechSynthesisUtterance( 'waking' ) );

and the problem was fixed. Then, I tried speaking an empty space and the problem still seems fixed. Yuck...

The issue is that speaking an empty string breaks SpeechSynthesis in ChromeOS. I verfied this by changing my HTML test to

<!DOCTYPE html>
<html>
<head>
  <meta charset="UTF-8">
  <title>TEST PAGE</title>
</head>
<body>
  <button id="button">Speak!</button>
</body>

<script>

  const button = document.getElementById( 'button' );
  button.addEventListener( 'click', event => {
    speechSynthesis.speak( new SpeechSynthesisUtterance( '' ) );
  } );

  window.setInterval( () => {
    console.log( speechSynthesis.speaking );
  } )

</script>
</html>

And when I press the button I see
Screenshot 2023-01-27 4 26 03 PM

So that is indeed the problem.

@zepumph can we meet next week to discuss whether we should continue with this workaround (adding a space), possible maintenance releases, or removing this workaround entirely to avoid workaround maintenance pain.

@jessegreenberg
Copy link
Contributor

Discussed. We prefer adding a space to the workaround. It seems like the least amount of effort, and lets us keep using the workaround for slow speech synthesis on Chromebooks.

Back to me to add the space. Ill look at which published sims have this problem and use that to inform whether we proceed with a maintenance release.

@zepumph
Copy link
Member

zepumph commented Feb 1, 2023

@jessegreenberg, I'm going to mark this as ready to cherry-pick since the original change for friction has been made, to be verified in RC and/or with you on master. Thanks!

@jessegreenberg
Copy link
Contributor

@Nancy-Salpepi can you please verify that this is fixed in master?

@Nancy-Salpepi
Copy link
Author

Great job! Working just fine in master with Friction (and RaP).

@Nancy-Salpepi Nancy-Salpepi removed their assignment Feb 2, 2023
@jessegreenberg
Copy link
Contributor

I think that is it for this issue, the only published sim with this problem is RaP (phetsims/ratio-and-proportion#550).

Commit to cherry pick: phetsims/utterance-queue@feff87d

@jessegreenberg
Copy link
Contributor

cherry-picked into the friction-1.6 branch of utterance-queue. Ready to verify.

@Nancy-Salpepi
Copy link
Author

This looks good in rc.3. Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants