Firefox Voice is an experiment from Mozilla Emerging Technologies.
Firefox Voice is a browser extension that allows you to give voice commands to your browser, such as "what is the weather?" or "find the gmail tab". Ultimately the goal is to see if we can facilitate meaningful user interactions with the web using just voice-based interactions. Initially the goal is to provide any useful interactions.
If you are using Windows, please install WSL, as the installation won't work from a normal Windows command prompt.
The developer installation is:
npm install npm start
This will launch a new Firefox browser with the extension installed. You should probably have Nightly or Developer Edition installed.
By default this will use Firefox Nightly, but you can override this with the environmental variable
$FIREFOX (you can point it to a release version, but some things may not work; also you can use a localized Firefox or an unbranded Firefox). You can also set
$PROFILE to a directory where the profile information is kept (it defaults to
By default messaging-related logging messages aren't shown, you can turn logging up slightly with
$LOG_LEVEL=messaging (or like
LOG_LEVEL=messaging npm start).
Any changes you make should cause any .jsx files to be recompiled and the extension will be reloaded.
In Firefox Voice there are several separate processes where things run (see also Anatomy of an extension):
- The "background page". This is a persistent page that belongs to the extension, and is where most of the work is done. For debugging this specifically see this
- The popup. This is it's own page (in
extension/popup/) and handles some of the initial lifecycle of invoking an intent. In most ways it is a normal page, but it runs in the short-lived popup. See the next section for a technique to debug this.
- The recorder tab. This is its own pinned tab that holds the media stream (because we have to keep this open to avoid permission issues). It is its own page. You can use the normal debugging tools on it.
- The search tab. This is also its own pinned tab that holds Google searches. It is not long-lived (each search causes it to reload), but it is specifically managed by the extension. The extension-specific code is run in content scripts, and normal debugging tools mostly work but can be finicky.
- Other content scripts. Any page that the extension manages directly (e.g., clicking controls, reading information) has content scripts injected.
The most reliable way to debug these is with the Browser Console, which should open automatically, or you can open with Tools > Web Developer > Browser Console. You should change the settings on the console using the gear icon in the upper-right, and turn on Show Content Messages (otherwise logging from the popup and some of these other sources will not be displayed). This setting should persist.
Debugging the popup
The popup can be hard to debug, since it disappears and there's no debugging tools. But the popup can also run in a tab. The easiest way to do this is to run:
OPEN_POPUP_ON_START=1 npm start
This will open the popup in a tab and reopen it whenever the extension restarts. Reloading the tab is equivalent to reopening the popup.
Writing a new command / intent
Please see Writing An Intent.
Using in-development versions
It's possible to install and use in-development versions of the extension. Every commit to
master is built into the dev build, and when we prepare for a release and merge to
stage is used to create the stage build.
NOTE THAT THESE VERSIONS INCLUDE EXTRA DATA COLLECTION
We are using these builds for internal testing with more-than-normal data collection. We have not yet implemented data collection controls.
- Install dev version
- Install stage version (Note: stage isn't always updated!)
- Install production version
- Logs of updates
The version numbers are increased for each release and each commit, but are not sequential.
Viewing Intent Information
There is an index of intents (commands) that is viewable if you open the panel, click on the gear/settings, and follow the "Intent Viewer" link.
Developing in Android
This is very experimental, but to develop for Firefox for Android (not Fenix), install Firefox (release) on your Android device.
To try, run:
npm run start-android
You may see an error message
Android device ... was not found in list: ["99EAP164UC"]: if so, then 99EAP164UC (for example) is your Android device name. Try again:
export ANDROID_DEVICE=99EAP164UC npm run start-android
You might have to install
adb and enable some permissions as well, look in the console for more instructions.
For some more information:
- This tutorial on Developing extensions for Firefox for Android
- See the web-ext docs and the section "Testing in Firefox for Android"
- How to get developer options on Android (to turn on USB access)
Demo inter-process communication for Android
On Android we're experimenting with collecting voice outside Firefox and then sending the text of the command into Firefox.
For demonstration purposes only there is an option to see a URL being opened and use that as the source of an intent. To enable this set the environmental variable
$EXECUTE_INTENT_URL to the base URL, and use
?text=... to pass in the text. For instance:
export EXECUTE_INTENT_URL=https://mozilla.github.io/firefox-voice/assets/execute.html npm run start-android
While we may enable something similar on desktop, it will use a different mechanism.
See the guidelines for contributing to this project.
This project is governed by a Code Of Conduct.
To disclose potential a security vulnerability please see our security documentation.
Made with contributors-img.
This module is licensed under the Mozilla Public License, version 2.0.