Use the Assistant Library for OK Google hotwording #64

drigz · 2017-05-25T11:18:39Z

This requires installation of the wheel for the Google Assistant
Library, which includes a closed-source hotword implementation.

This allows the user to configure responses on IFTTT, so that local actions have an IFTTT response with the Assistant's voice.

This requires installation of the wheel for the Google Assistant Library, which includes a closed-source hotword implementation.

proppy · 2017-05-25T23:13:29Z

src/main.py


+    # The ok-google trigger is handled with the Assistant Library, so we need
+    # to catch this case early.
+    if args.trigger == 'ok-google':


It'd be nice to also make the button work with the library (it supports start_conversation).

@proppy not sure if it helps; but, I pulled down this branch and tested it. is this the start-conversation you are talking about?

"Hey Google, how far away is Japan?"
< "Japan is 6xxx miles away as way of the crow flies."
"Hey Google, and from California?"
< "Japan is 5xxx miles away from California."

console output below. notice the "how far away is Japan" and the "and from California" follow up:

$ src/main.py --trigger="ok-google" [2017-05-26 22:25:17,165] INFO:root:ON_MUTED_CHANGED: {'is_muted': False} [2017-05-26 22:25:17,166] INFO:root:ON_START_FINISHED [2017-05-26 22:25:17,167] INFO:main:ready... Say "OK, Google" then speak, or press Ctrl+C to quit... [2017-05-26 22:25:24,515] INFO:root:ON_CONVERSATION_TURN_STARTED [2017-05-26 22:25:24,516] INFO:main:listening... [2017-05-26 22:25:28,365] INFO:root:ON_END_OF_UTTERANCE [2017-05-26 22:25:28,366] INFO:main:thinking... [2017-05-26 22:25:28,385] INFO:root:ON_RECOGNIZING_SPEECH_FINISHED: {'text': 'how far away is Japan'} [2017-05-26 22:25:28,907] INFO:root:ON_RESPONDING_STARTED: {'is_error_response': False} [2017-05-26 22:25:33,156] INFO:root:ON_RESPONDING_FINISHED [2017-05-26 22:25:33,157] INFO:root:ON_CONVERSATION_TURN_FINISHED: {'with_follow_on_turn': False} [2017-05-26 22:25:33,158] INFO:main:ready... [2017-05-26 22:25:34,972] INFO:root:ON_CONVERSATION_TURN_STARTED [2017-05-26 22:25:34,974] INFO:main:listening... [2017-05-26 22:25:37,748] INFO:root:ON_END_OF_UTTERANCE [2017-05-26 22:25:37,749] INFO:main:thinking... [2017-05-26 22:25:37,784] INFO:root:ON_RECOGNIZING_SPEECH_FINISHED: {'text': 'and from California'} [2017-05-26 22:25:38,856] INFO:root:ON_RESPONDING_STARTED: {'is_error_response': False} [2017-05-26 22:25:42,704] INFO:root:ON_RESPONDING_FINISHED [2017-05-26 22:25:42,705] INFO:root:ON_CONVERSATION_TURN_FINISHED: {'with_follow_on_turn': False} [2017-05-26 22:25:42,707] INFO:main:ready...

Also, I played the Trivia game and answered five questions in turn.

"Hey Google, let's play a game."

I decided against this as it seemed to hard to document having three different backends and three different triggers, but hotword trigger only works with one backend.

ensonic

Thanks for giving it a try!

ensonic · 2017-05-26T10:16:07Z

scripts/install-deps.sh

@@ -33,6 +33,7 @@ sudo pip3 install --upgrade pip virtualenv
 cd "${scripts_dir}/.."
 virtualenv --system-site-packages -p python3 env
 env/bin/pip install -r requirements.txt
+env/bin/pip install --upgrade https://github.com/googlesamples/assistant-sdk-python/releases/download/0.3.0/google_assistant_library-0.0.2-py2.py3-none-linux_armv7l.whl


Shouldn't this go to requirements.txt?

The wheels is available on pypi now:
https://pypi.python.org/pypi/google-assistant-library

ensonic · 2017-05-26T10:16:36Z

config/voice-recognizer.ini.default

+
+# Uncomment to play Assistant responses for local actions.  You should make
+#  sure that you have IFTTT applets for your actions to get the correct
+#  response, and also that your actions do not call say().


remove one of the leading spaces in the two lines above?

ensonic · 2017-05-26T10:17:12Z

src/actionbase.py

@@ -33,6 +33,16 @@ def get_phrases(self):
        """Get a list of all phrases that are expected by the handlers."""
        return [phrase for h in self.handlers for phrase in h.get_phrases()]

+    def would_handle(self, command):


can_handle()? Same below.

ensonic · 2017-05-26T10:20:53Z

src/main.py

@@ -121,7 +121,7 @@ def main():
    parser.add_argument('-O', '--output-device', default='default',
                        help='Name of the audio output device')
    parser.add_argument('-T', '--trigger', default='gpio',
-                        help='Trigger to use {\'clap\', \'gpio\'}')
+                        help='Trigger to use {"clap", "gpio", "ok-google"}')


We can do: choices=['clap', 'gpio', 'ok-google'],
https://docs.python.org/2.7/library/argparse.html#choices

ensonic · 2017-05-26T10:22:45Z

src/main.py

+        print('    https://developers.google.com/assistant/sdk/prototype/'
+              'getting-started-pi-python/run-sample'
+              '#get_the_library_and_sample_code')
+        raise


Shouldn't we rather point to our requirements.txt?

ensonic · 2017-05-26T10:25:15Z

src/main.py

@@ -204,6 +273,45 @@ def do_recognition(args, recorder, recognizer, player):
            time.sleep(1)


+class StatusUi(object):
+


remove blank line before doc string

The blank-line-before-docstring style is used consistently at the project. However, it seems it's based on an old version of PEP8: hhatto/autopep8#194

I'd prefer to fix in a another PR.

ensonic · 2017-05-26T10:26:05Z

src/main.py

+    """Gives the user status feedback.
+
+    The LED and optionally a trigger sound tell the user when the box is
+    ready, listening and thinking.


"... when the box is ready, listening or thinking."?

codecov-io · 2017-05-30T08:21:48Z

Codecov Report

Merging #64 into master will increase coverage by 0.19%.
The diff coverage is 66.66%.

@@            Coverage Diff            @@
##           master     #64      +/-   ##
=========================================
+ Coverage    9.76%   9.95%   +0.19%     
=========================================
  Files           3       3              
  Lines        1720    1727       +7     
  Branches      295     297       +2     
=========================================
+ Hits          168     172       +4     
- Misses       1542    1543       +1     
- Partials       10      12       +2

Impacted Files	Coverage Δ
src/actionbase.py	`90.32% <66.66%> (-9.68%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 250b8cb...7fd0903. Read the comment docs.

drigz · 2017-05-30T08:22:16Z

PTAL

sheridat · 2017-05-30T09:51:19Z

I am a newcomer - so this question might not make a lot of sense but the sentence above "This requires installation of the wheel for the Google Assistant Library, which includes a closed-source hotword implementation" has me worried.

I associate wheel with updating python libraries via pip.

People are aware that there is a new version of the google assistant SDK is available BUT that it is for armv7 capable processors. People are also aware that it supports "hotword"

Many of us are running the AIY software on Pi Zero's and Pi W's which are armv6 based and therefore we cannot use that new version of the the assistant SDK.

My worry is that the change you are making here means that once it is merged - if I was to git pull this repository and run the "install deps script" the google assistant software will be updated and my installation will stop working because the google assistant will want an armv7 processor.

But as we say in england - I may be (entirely) barking up the wrong tree.

eduncan911 · 2017-05-30T11:34:48Z

You have chased the cat up the correct tree.

The SDK update in this PR will restrict installation in ARMv7 going forward.

The binary is specific to detecting the hot words, OK Google and Hey Google. They have rigorously tested it for false positives. But why they kept it closed sourced we don't know. For ARMv7 though, they mention they are leveraging some of the newer CPU instructions to speed things up.

To address your concern though, just don't upgrade. Or, fork it and use your on branch. You can merge changes in, but kept the wheel out of your branch.

drigz · 2017-05-30T11:42:26Z

We could remove google-assistant-library from requirements.txt so that it's still usable on ARMv6, (maybe ok_google_requirements.txt?) and amend the error message to say "ok-google triggering is not available on your platform" for ARMv6 etc, and "Please pip install -r ok_google_requirements.txt" for ARMv7. What do you think?

sheridat · 2017-05-30T12:57:37Z

ummmmmhhhhhhhhhhh. Is it realistic for the newcomers to start using Git to maintain their own fork/branch and maintain it. We are struggling with how to update it. A compromise such as that outlined by Drigz seems a better solution and only a little inconvenient for people using the armv7 processor. I would hate to think that some kids using this on a Zero or W suddenly need to become Git gurus.

eduncan911 · 2017-05-30T13:23:14Z

I know the struggles quite well, as I am not a Python guru myself and stumble through most of this.

I think this discussion really belongs over at the Assistant SDK repo though, not in this PR:

https://github.com/googlesamples/assistant-sdk-python

You can read a marketing overview of what the Assistant does (or will do soon):

https://developers.google.com/assistant/sdk/

Basically, the Assistant SDK does a lot more than just hot wording. It is what does the interrupting of the vocal commands, for example. Removing it would pretty much render this "Voice Recognizer" project dead and useless. The "hot wording" is just one feature in a set of a much larger feature set that this repo is leveraging.

This repo is basically just a wrapper framework with some triggers and actions to leverage the Assistant SDK core library.

sheridat · 2017-05-30T13:31:00Z

I see - and I can see the clamour for new features.

drigz · 2017-05-31T13:26:36Z

@ensonic, the library is now optional. Users who git pull and try to use -T ok-google will get a message saying the dep is only for Pi 2/3, and how to install it.

WDYT?

sheridat · 2017-06-01T08:19:55Z

Hi, I've closed my issue above

drigz added 2 commits May 25, 2017 13:02

Add a new option: --assistant-always-responds

9ddd9ec

This allows the user to configure responses on IFTTT, so that local actions have an IFTTT response with the Assistant's voice.

Use the Assistant Library for OK Google hotwording

1a3ece1

This requires installation of the wheel for the Google Assistant Library, which includes a closed-source hotword implementation.

drigz requested review from proppy and ensonic May 25, 2017 11:18

googlebot added the cla: yes label May 25, 2017

proppy reviewed May 25, 2017

View reviewed changes

ensonic suggested changes May 26, 2017

View reviewed changes

proppy mentioned this pull request May 26, 2017

pip install --upgrade google-assistant-sdk broke main.py #56

Closed

eduncan911 mentioned this pull request May 27, 2017

Voice Trigger #59

Closed

Noiprox6 mentioned this pull request May 29, 2017

AlexaPi seems to kill googlevoicehat until reboot alexa-pi/AlexaPi#230

Closed

Update after review

7fd0903

sheridat mentioned this pull request May 31, 2017

Potentially Disenfranchise Raspberry PI Zero and W AIY users googlesamples/assistant-sdk-python#68

Closed

Make library optional for Pi 0/1 users

4aab3b9

ensonic approved these changes May 31, 2017

View reviewed changes

drigz merged commit 5b2cd9c into google:master May 31, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use the Assistant Library for OK Google hotwording #64

Use the Assistant Library for OK Google hotwording #64

drigz commented May 25, 2017

proppy May 25, 2017

eduncan911 May 27, 2017 •

edited

Loading

drigz May 30, 2017

ensonic left a comment

ensonic May 26, 2017

proppy May 26, 2017

ensonic May 26, 2017

ensonic May 26, 2017

ensonic May 26, 2017

ensonic May 26, 2017

ensonic May 26, 2017

drigz May 30, 2017

ensonic May 26, 2017

codecov-io commented May 30, 2017 •

edited

Loading

drigz commented May 30, 2017

sheridat commented May 30, 2017 •

edited

Loading

eduncan911 commented May 30, 2017

drigz commented May 30, 2017 •

edited

Loading

sheridat commented May 30, 2017

eduncan911 commented May 30, 2017 •

edited

Loading

sheridat commented May 30, 2017

drigz commented May 31, 2017

sheridat commented Jun 1, 2017

		@@ -204,6 +273,45 @@ def do_recognition(args, recorder, recognizer, player):
		time.sleep(1)


		class StatusUi(object):

Use the Assistant Library for OK Google hotwording #64

Use the Assistant Library for OK Google hotwording #64

Conversation

drigz commented May 25, 2017

Choose a reason for hiding this comment

eduncan911 May 27, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ensonic left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-io commented May 30, 2017 • edited Loading

Codecov Report

drigz commented May 30, 2017

sheridat commented May 30, 2017 • edited Loading

eduncan911 commented May 30, 2017

drigz commented May 30, 2017 • edited Loading

sheridat commented May 30, 2017

eduncan911 commented May 30, 2017 • edited Loading

sheridat commented May 30, 2017

drigz commented May 31, 2017

sheridat commented Jun 1, 2017

eduncan911 May 27, 2017 •

edited

Loading

codecov-io commented May 30, 2017 •

edited

Loading

sheridat commented May 30, 2017 •

edited

Loading

drigz commented May 30, 2017 •

edited

Loading

eduncan911 commented May 30, 2017 •

edited

Loading