Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simple Snowboy support #177

Merged
merged 2 commits into from Nov 4, 2017

Conversation

@renekliment
Copy link
Member

commented Feb 25, 2017

Issue this PR fixes (if applicable): #53

Description:

What everybody has been waiting for. This works, there might be improvements later, but is mergeable.

The installation has to be done manually for now:

# Install snowboy dependencies as described in https://github.com/Kitt-AI/snowboy#ubunturaspberry-pipine64nvidia-jetson-tx1nvidia-jetson-tx2
# If you're on Raspbian < Stretch or you simply have older swig3.0, you need to install a newer than from the repo.
sudo apt-get install swig3.0 libatlas-base-dev

# install snowboy
sudo pip install git+https://github.com/Kitt-AI/snowboy.git

I think that's fine since it's experimental for now. Auto install should come in a later PR.

Checklist:

  • Update Wiki (new platform, installation, configuration changes etc)
  • PR Location (usually master for hotfixes, dev for general development)
@renekliment renekliment added this to the 1.5 milestone Feb 25, 2017
@renekliment renekliment referenced this pull request Feb 25, 2017
0 of 3 tasks complete
@renekliment renekliment force-pushed the renekliment:snowboy-support branch 8 times, most recently from 424cdb9 to eee1649 Feb 25, 2017
@renekliment renekliment force-pushed the renekliment:snowboy-support branch 3 times, most recently from 01eb37e to 9c0dda6 Mar 5, 2017
@pfiol

This comment has been minimized.

Copy link

commented Mar 7, 2017

Hi @renekliment I finally got time to test this one. It works perfectly on my RapsberryPi B+. Great work! :)

However, I've noticed that it is quite slow when collecting the answers from amazon. Does it records your voice for a fixed amount of time or it detects the silence to proceed to launch the query to amazon? Any clue where this delay may come from?

@renekliment

This comment has been minimized.

Copy link
Member Author

commented Mar 7, 2017

@pfiol Thanks for the testing! There is a VAD. Also, yesterday I've rebased it on dev, where there is now a huge change to stream audio to AVS, instead of recording it all at once. Post your log in debug mode (w/o pocketsphinx stuff).

@pfiol

This comment has been minimized.

Copy link

commented Mar 7, 2017

@renekliment Yeah! I was about to comment again, just found the #44 issue. I think that's exactly what I was looking for. I will try it this afternoon. That's the only missing piece of the puzzle for me.

I've named my alexapi as "R2D2", replaced the "yes" and "hello" mp3 files for R2d2 sounds, and put everything inside a R2D2-like kitchen jar! hahaha it's awesome! :)

I will let you know my findings on the new code.

@renekliment renekliment force-pushed the renekliment:snowboy-support branch from 9c0dda6 to 912b702 Mar 7, 2017
renekliment referenced this pull request in renekliment/AlexaPi Mar 7, 2017
@pfiol

This comment has been minimized.

Copy link

commented Mar 17, 2017

Hi @renekliment
I have locally merged this PR with the Master branch, to take benefit of the new voice "streaming" to amazon, and also use the snowboy trigger, and it works.

The only problem I found is that I have to wait around 2-3 seconds after it says "yes" to start making the question, Do you have any idea why there's such a delay and how to fix it?

@pfiol

This comment has been minimized.

Copy link

commented Mar 23, 2017

Just in case it someone tried to merge this PR with the master branch. I'm facing some delays after wake word detection. See the logs below:

2017-03-23 21:55:17 DEBUG: Keyword 1 detected
2017-03-23 21:55:17 DEBUG: Stopping audio play
2017-03-23 21:55:17 DEBUG: Stopped play. [Errno 3] No such process
2017-03-23 21:55:17 DEBUG: Playing audio: /opt/AlexaPi/src/resources/alexayes.mp3
2017-03-23 21:55:17 DEBUG: Started play. sox -q /opt/AlexaPi/src/resources/alexayes.mp3 -t alsa vol -2 dB pad 0 0
2017-03-23 21:55:19 DEBUG: Finished play. 
2017-03-23 21:55:19 DEBUG: Setting up recording
2017-03-23 21:55:19 DEBUG: Start recording
2017-03-23 21:55:19 DEBUG: Starting new HTTPS connection (1): access-alexa-na.amazon.com
2017-03-23 21:55:20 DEBUG: Start sending speech to Alexa Voice Service
2017-03-23 21:55:24 DEBUG: End recording
2017-03-23 21:55:24 DEBUG: Finished sending speech to Alexa Voice Service
2017-03-23 21:55:25 DEBUG: Processing Request Response...

It looks like it takes a lot of time to finish the "alexayes.mp3" and later again it takes a second or two to actually start sending/streaming the request to amazon. Afterwards everything goes fine, the reply from amazon is very fast.

Any idea?

@pfiol

This comment has been minimized.

Copy link

commented Apr 13, 2017

@renekliment I finally got all the pieces together for my personal project. I modified a little bit the code on this branch and used a customised wakeup word. This is the result: https://www.youtube.com/watch?v=E2xg2_iwZv4

I hope you like it!

@probststefan

This comment has been minimized.

Copy link

commented May 22, 2017

Will this pull request be merged in the near future?

@renekliment

This comment has been minimized.

Copy link
Member Author

commented May 25, 2017

@probststefan Well, I might have some time for AlexaPi again, so, it might. Snowboy can now be installed with pip (Kitt-AI/snowboy#73), which should solve some of the tasks in the description.

To avoid doing something that very few people want, I ask: Do you guys still want this?

@dapetcu21

This comment has been minimized.

Copy link

commented May 25, 2017

I support this and think would be useful. Without snowboy, AlexaPi is unusable for me.

@joe248

This comment has been minimized.

Copy link

commented May 25, 2017

I've been waiting and hoping for this as well.

@shreyasubale

This comment has been minimized.

Copy link

commented May 25, 2017

Same here, for me pocketsphinx almost never works properly ( accent ) and snowboy does well.

@revere521

This comment has been minimized.

Copy link

commented May 26, 2017

@renekliment I'd also like to have it...so yes please :)

@ezar

This comment has been minimized.

Copy link

commented Jun 11, 2017

+1

@renekliment renekliment force-pushed the renekliment:snowboy-support branch 2 times, most recently from b536e58 to c34f61c Oct 14, 2017
@revere521

This comment has been minimized.

Copy link

commented Oct 14, 2017

!!!! awesome - Thanks @renekliment!

i am running the pip install right now, do i need to git pull the current alexapi to activate it, or make any config changes?

@renekliment renekliment force-pushed the renekliment:snowboy-support branch from c34f61c to 1dfb95a Oct 14, 2017
@renekliment

This comment has been minimized.

Copy link
Member Author

commented Oct 14, 2017

@revere521 You need to checkout this PR branch. For example like this: https://github.com/alexa-pi/AlexaPi/blob/master/CONTRIBUTING.md#if-you-want-to-check-out-someone-elses-branch-for-example-a-pull-request

There is a new section in the config. You can simply add it to your config and disable pocketsphinx and enable snowboy.

@renekliment renekliment force-pushed the renekliment:snowboy-support branch 2 times, most recently from 0575647 to 399ff7b Oct 14, 2017
@revere521

This comment has been minimized.

Copy link

commented Oct 14, 2017

i had to manually grab the deb for the armhf version of swig3.0 - http://mirrordirector-mb.raspbian.org/raspbian/pool/main/s/swig/swig3.0_3.0.10-1.1_armhf.deb

the pip install kept failing... i think travis.yml mentions the amd64.deb

not sure if that matters or not - mine is a raspbian based pi 3

@renekliment

This comment has been minimized.

Copy link
Member Author

commented Oct 14, 2017

@revere521 Did you run the apt commands from the snowboy's README? I've put it in the PR description to be clear.

@revere521

This comment has been minimized.

Copy link

commented Oct 14, 2017

I did, but the raspbian version that installs from apt is 3.0.2, so i just read through your chnages in the .yml and did that stuff manually, just to be sure. that seemed to be the only thing i had to do was fix the .deb

@renekliment

This comment has been minimized.

Copy link
Member Author

commented Oct 14, 2017

@revere521 I see. What Raspbian version are you using?

@revere521

This comment has been minimized.

Copy link

commented Oct 14, 2017

sorry, i got tied up - im on Raspbian Jessie, should be whatever is the current vanilla - not stretch

@revere521

This comment has been minimized.

Copy link

commented Oct 14, 2017

thats the only difference i have, and i use the platform - magicmirror, because it is a magic mirror too - and it seems to be working nicely

@renekliment renekliment force-pushed the renekliment:snowboy-support branch from 919e031 to 3969a3c Oct 14, 2017
@renekliment

This comment has been minimized.

Copy link
Member Author

commented Oct 14, 2017

@revere521 Can you compare it with pocketsphinx in regard of reliability (note there is also a value you can tweak) and CPU usage?

@renekliment renekliment force-pushed the renekliment:snowboy-support branch from 3969a3c to 1210773 Oct 14, 2017
@revere521

This comment has been minimized.

Copy link

commented Oct 14, 2017

sorry just got home -- CPU usage at idle is fine - i haven't had time to really put it through the paces yet. The audio response was a bit choppy, but my internet throughput may be to blame. One question, it does not seem to be respecting my custom wakeword - is that set someplace else? I notice that the selection is specifically only under pocketsphinx.

top

edit, fixed my screencap

@renekliment

This comment has been minimized.

Copy link
Member Author

commented Oct 15, 2017

@revere521 Dope.

I've added the info about that to the wiki, see https://github.com/alexa-pi/AlexaPi/wiki/Triggers#snowboy

In the future, there might be an intergration in AlexaPi that would allow to train a personal model right from AlexaPi via snowboy's API, but that's a matter of another PR. As the title suggests, this is a basic support.

@revere521

This comment has been minimized.

Copy link

commented Oct 15, 2017

cool, thanks! it does seem better overall - and i don't want to seem unappreciative that you made this available, because i am.

Its going to be some new learning and dialing in - but thats why we play with software like this :)

i'll see if there is a way to create a custom wake word/hotword .pmdl on the Pi and let you know what happens.

edit - laziness prevails - there were already lots of hotword profiles and you can just download one you like from the dashboard: https://snowboy.kitt.ai/dashboard

then put it in /usr/local/lib/python2.7/dist-packages/snowboy/resources/ and select it in the config

edit 2, i used this one - https://snowboy.kitt.ai/hotword/47 "smart mirror"

@renekliment

This comment has been minimized.

Copy link
Member Author

commented Oct 16, 2017

@revere521 Preferably, store it in /etc/opt/AlexaPi/. It might get deleted on snowboy upgrade in your location.

@revere521

This comment has been minimized.

Copy link

commented Oct 16, 2017

thats not a bad point, but for some reason when i do that i get ERROR: Urlerror: [Errno 111] connection refused. errors in the log. I may try to troubleshoot more later; but weirdness.....

@renekliment

This comment has been minimized.

Copy link
Member Author

commented Oct 16, 2017

@revere521 That particular error is probably related to MagicMirror. I'll make the log message more clear soon.

@revere521

This comment has been minimized.

Copy link

commented Oct 16, 2017

It does seem like its an inability to communicate with the listener for the plugin for the magic mirror; but it goes away when i put the umdl in the distribution folder - or at least thats what i observed

@revere521

This comment has been minimized.

Copy link

commented Oct 19, 2017

Been using it for a few days, works very well and after setting the sensitivity to 0.4 - no false responses have occurred that i am aware of

absolutely fantastic

@renekliment

This comment has been minimized.

Copy link
Member Author

commented Nov 4, 2017

So it's been 20 days since I've posted this. It's not ideal, but it's mergeable. I'm probably keeping the label experimental, since it requires manual installation and the code could use improvements.

Anyways, merging to dev.

@renekliment renekliment merged commit 8cc9f02 into alexa-pi:dev Nov 4, 2017
1 check passed
1 check passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@renekliment renekliment deleted the renekliment:snowboy-support branch Nov 4, 2017
@michibuddha

This comment has been minimized.

Copy link

commented Nov 5, 2017

I am a quite noob to all this but I just built my RPi echo will it resolve the 5 to 10 sec delay I have after asking question??

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.