Skip to content
This repository has been archived by the owner on Nov 22, 2022. It is now read-only.

Add text-to-speech feature #63

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

Add text-to-speech feature #63

wants to merge 7 commits into from

Conversation

fluks
Copy link
Contributor

@fluks fluks commented Feb 3, 2016

Hi, I added an option to listen to Google Translate's TTS voice. Works on the Firefox developer edition(46.0a2), I can't test it on the main release because of the extension signature enforcement, even I have 'xpinstall.signatures.required' in about:config, still won't install if I toggle it to false.

To listen to the voice you need to press the Listen label.

fluks added 4 commits February 3, 2016 06:33
Add a menuitem to the popup to speak out the selection. Press the
'Listen' to play the speech.

If the from language is set to auto, translate() sets a variable to
the detected language.

TODO
  - Can play speeches on top of each other, allow only one instance to
    play at once.
  - Leaks memory probably now. Every time a speech is played, a new
    background Page is created and it's never destroyed. Create only one
    global page at the beginning and send messages to play?
I fixed the problem described in a previous commit about creating
page-worker.Page objects every time user played the audio. Now
Firefox throws unsafe CPOW usage errors, the audio plays but this can't
be good.

TODO
  - Use Web Audio API, just steal someone else's work.
Use Web Audio API instead of contentscript, which I tried previously, to
play the speech.

TODO
  - Volume
  - Speech speed
  - Both above need settings
@PerfectSlayer
Copy link
Collaborator

Thanks for your contribution!
I'll try to test it this weekend if no one validate it before ;)

@fluks
Copy link
Contributor Author

fluks commented Feb 4, 2016

I was able to install it to the normal version of Firefox now.

The first time you listen the speech it works, but after that it doesn't. Strange that in the developer version I didn't notice this behavior. It looks like a new AudioContext must be created every time playing the speech. I wonder if the audio thing might leak resources if you don't clean up after it every time. Is there any good way to test for memory leaks in FF?

fluks added 2 commits February 5, 2016 03:02
Create a new AudioContext in the click handler every time the speech is
played and free resources after the playing has ended.

Is it necessary to free resources at all? If so, can you do it in the
AudioBufferSourceNode.ended callback?
@fluks
Copy link
Contributor Author

fluks commented Feb 5, 2016

Now, a new AudioContext is created every time when you want to listen to the speech. I'll try to research about freeing up resources afterwards, is it needed at all. And about the Web Audio API in general.

I have been running this code all day on my browser and it seems to work fine, but I'd like to know for sure it's correct.

@PerfectSlayer
Copy link
Collaborator

Sad to see AudioContext behavior is not the same accros the Firefox versions..
Feel free to share a link to your signed dev version to allow us to test on current the Firefox 😉

@bpierre
Copy link
Owner

bpierre commented Feb 5, 2016

Thanks @fluks, looks great!

The AudioContext should be freed automatically by the garbage collector if there is no references to it? You can use this tool to debug memory in Firefox: https://developer.mozilla.org/en-US/docs/Tools/Memory

@fluks
Copy link
Contributor Author

fluks commented Feb 5, 2016

PerfectSlayer:
Sad to see AudioContext behavior is not the same accros the Firefox versions..
Feel free to share a link to your signed dev version to allow us to test on current the Firefox 😉

It should work now on the current version of Firefox and on the developer version also. You can test the feature by just pulling my branch? I haven't signed anything yet, do I really need to?

I was able to install an unsigned extension by:

  • setting xpinstall.signatures.required to false
  • restarting the browser (I'm not sure if this is required, someone said so on the #extdev channel)
  • installing from Tools > Add-on: clicking on the settings icon and selecting 'Install Add-on From File'

Installing an unsigned extension via url bar by pointing it to the location of the extension on the file system failed.

bpierre:
The AudioContext should be freed automatically by the garbage collector if there is no references to it?

I'll remove the AudioBufferSourceNode.onended callback altogether and see what happens

@fluks
Copy link
Contributor Author

fluks commented Feb 7, 2016

Here's a signed xpi for you to test, @PerfectSlayer, if you still need it.

https://github.com/fluks/gtranslate/raw/xpi/gtranslate-0.13.0-fx%2Ban.xpi

@PerfectSlayer
Copy link
Collaborator

fluks:
It should work now on the current version of Firefox and on the developer version also. You can test the feature by just pulling my branch? I haven't signed anything yet, do I really need to?

Thanks. I thought they already remove the xpinstall.signatures.required setting (but no !).

By the way, it seems to work fine! 👏
If I could suggest an improvement, it's to stop the current AudioContext if a new one is started. Otherwise, you will have the voices that overlap.

@PerfectSlayer
Copy link
Collaborator

So, is-it stable enough? Does anyone test-it?
@bpierre Do you consider approving it?

@bpierre
Copy link
Owner

bpierre commented Feb 24, 2016

I haven’t tested it yet, but I definitely consider to approve it.

Some things I see:

  • I think we need to make it optional for the user, as some people might prefer to not have this feature. Should we enable it by default?
  • It should be made optional for the provider to support this feature. I plan to add Bing at some point, and I am not sure they will have this feature.
  • For the same portability reasons, what would you think of moving the XHR part, and maybe the playing part too, inside of the provider (providers/google-translate.js? That way, index.js would just ask the provider to “play the translation” if the feature is supported, all the details belong to the provider itself.

@fluks
Copy link
Contributor Author

fluks commented Feb 26, 2016

So, is-it stable enough? Does anyone test-it?

It has been in my tests. I have been using a version with a feature you suggested, such that the voices won't overlap. I haven't committed this version.

@fluks
Copy link
Contributor Author

fluks commented Feb 26, 2016

It should be made optional for the provider to support this feature. I plan to add Bing at some point, and I am not sure they will have this feature.

The translator page can speak and the official API also has this feature.
https://msdn.microsoft.com/en-us/library/ff512405.aspx

For the same portability reasons, what would you think of moving the XHR part, and maybe the playing part too, inside of the provider (providers/google-translate.js? That way, index.js would just ask the provider to “play the translation” if the feature is supported, all the details belong to the provider itself.

Makes sense, I'll do that.

I don't know is this feature wanted by many users, but the change in the ui is quite minor currently. Here's a pic.

@fluks
Copy link
Contributor Author

fluks commented Feb 26, 2016

This code doesn't address the overlapping problem. To stop the audio being played, you need to call AudioBufferSourceNode.stop(). So, that state should be saved somewhere (source variable). Maybe the providers should be classes?

I also removed some unnecessary code.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants