Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issues 238 - Philips Hue Skill #240

Conversation

ChristopherRogers1991
Copy link
Collaborator

#238 - Phillips Hue Skill

@@ -93,3 +93,10 @@ max_time = 600 # in seconds
notify_delay = 5 # in seconds
rate = 16000
channels = 1

[PhillipsHueSkill]
verbose = False
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A verbose option doesn't really seem necessary for this. I would recommend just having one bit of dialog feedback.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to keep this in, if you're not strongly against it. I live in a small studio apartment, so for me, the lights changing is all the feedback I need. Having Mycroft tell me what I can see him doing is just noise - it's how the Amazon Echo works, and I hate it (trying to turn off the lights before bed results in a jarring 'OK!' any time the volume is left too high).

Someone with a larger home, however, may appreciate the feedback, if, for example, they are trying to turn off the living room lights from the bedroom.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not super strongly against it. I'll do some testing with other team members to see what we think.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, cool. If I do lose via the democratic process here though, can I ask if there's a practical reason to remove it? The overhead for keeping it seems pretty negligible (it's one extra line per intent type that makes use of it, and an essentially non-existent perf hit from the extra ifs) so I'm curious what your concerns are.

Just as a note, this functionality is important enough to me as a user, I would remove this if I had to to get it into core, but fork and add it back for my own Mycroft. I really, really hate the confirmations from the Echo, and wish I could disable them there.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with keeping it, I think the only thing would be whether it would default to false or true.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sweet. I'm fine with either as the default. Let me know if you, or anyone else, has a preference.

@ChristopherRogers1991
Copy link
Collaborator Author

The unit tests are going to fail with this latest commit. I didn't want to take the time to update them yet, because I'm having some trouble getting Mycroft to take the correct actions.

@ethanaward, in the latest commit, I tried restructuring how I handle all of my keywords, as you'd mentioned in a few of your comments (using an IncreaseKeyword and a LightsKeyword to turn the brightness up, for example).

Since doing this, all of the actions still work for certain phrasings, however, some phrases that I would expect would work do not. For example, if I say "turn up the lights,' the brightness increases as expected, however, if I say "increase the brightness of the lights," he fails to find a matching intent, and searches Wolfram Alpha. Similarly, if I say "turn up the color temperature of the lights," everything works, however, if I say "increase the color temperature of the lights," he gives me the weather report. "Turn down" and "decrease" for both of these intent types works out the same way, with "turn down" working correctly, and "decrease" giving me wolfram alpha or weather.

With the way I have things structured, I would expect to have decent matches, so I'm confused why things are working.

Do you have any ideas? Should I just go back to the longer, more well defined keywords?

@ethanaward
Copy link
Contributor

My first guess is some sort of vocab conflict. The weather intent requires only temperature as a keyword, so if decrease isn't working for some reason, it would make sense that it would trigger. I'm not entirely sure why that it, though. @clusterfudge , any ideas?

And could you post some logs of what the intent parser is interpreting?

@clusterfudge
Copy link
Contributor

It's going to be a bit before I can go deep on this change. My recommendation would be to test the skill in isolation (via the skills container) and see if it works as you intend. It is possible to walk through the intent tests in a debugger, but it's not well documented and I haven't done it in a while.

@ChristopherRogers1991
Copy link
Collaborator Author

The links on this docs page for setting up the skills sdk, and installing dependencies both currently just say they are works in progress. Rather than screw around trying to set them up on my own, I just made a dummy branch, and stripped out all the other skills. Even without the other skills, it does not seem to be creating the intents properly. Logs for the run can be found here

Let me know if there is anything else you need, or anything else I should try.

@ChristopherRogers1991
Copy link
Collaborator Author

Does anyone (@clusterfudge in particular) have any idea why this intent test would pass, but using the phrase would fail in practice:

2016-09-17 13:47:48,349 - Skills - DEBUG - {"message_type": "recognizer_loop:wakeword", "context": null, "metadata": {"session": "802db0e1-66a7-4be1-a85a-2a94051d2378", "utterance": "hey mycroft"}}
2016-09-17 13:47:49,443 - Skills - DEBUG - {"message_type": "recognizer_loop:utterance", "context": null, "metadata": {"session": "802db0e1-66a7-4be1-a85a-2a94051d2378", "utterances": ["lights to 85%"]}}
2016-09-17 13:47:49,456 - intent__init__ - ERROR - 
Traceback (most recent call last):
  File "/home/chris/Code/mycroft-core/mycroft/skills/intent/__init__.py", line 47, in handle_utterance
    utterance, num_results=100))
StopIteration
2016-09-17 13:47:49,461 - Skills - DEBUG - {"message_type": "intent_failure", "context": null, "metadata": {"utterance": "lights to 85%"}}
2016-09-17 13:47:49,463 - wolfram_alpha__init__ - DEBUG - Could not determine intent, falling back to WolframAlpha Skill!
2016-09-17 13:47:49,467 - Skills - DEBUG - {"message_type": "enclosure.mouth.think", "context": null, "metadata": {}}
2016-09-17 13:47:49,478 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): cerberus.mycroft.ai
2016-09-17 13:47:52,511 - requests.packages.urllib3.connectionpool - DEBUG - "GET /wolframalpha/v2/query?input=lights+to+85%25 HTTP/1.1" 200 475
2016-09-17 13:47:52,518 - Skills - DEBUG - {"message_type": "speak", "context": null, "metadata": {"utterance": "Sorry, I couldn't find an answer for lights to 85%. Perhaps you meant lights instead?"}}

The regex is (?P<Value>100|[1-9][0-9]?)(\D|$), which seems to parse out what I want just fine:

>>> x = re.compile(u'(?P<Value>100|[1-9][0-9]?)(\D|$)')
>>> r = x.search('set my lights to 50')
>>> r.group('Value')
'50'
>>> r = x.search('set my lights to 50%')
>>> r.group('Value')
'50'
>>> r = x.search('set the brightness to 50 on my lamp')
>>> r.group('Value')
'50'
>>> r = x.search('set the brightness to 80% on my lamp')
>>> r.group('Value')
'80'

But everything with a '%' after the value fails to recognize the intent:

2016-09-17 13:56:08,437 - Skills - DEBUG - {"message_type": "recognizer_loop:record_begin", "context": null, "metadata": {}}
2016-09-17 13:56:12,335 - Skills - DEBUG - {"message_type": "recognizer_loop:record_end", "context": null, "metadata": {}}
2016-09-17 13:56:12,341 - Skills - DEBUG - {"message_type": "recognizer_loop:wakeword", "context": null, "metadata": {"session": "802db0e1-66a7-4be1-a85a-2a94051d2378", "utterance": "hey mycroft"}}
2016-09-17 13:56:14,263 - Skills - DEBUG - {"message_type": "recognizer_loop:utterance", "context": null, "metadata": {"session": "802db0e1-66a7-4be1-a85a-2a94051d2378", "utterances": ["set the brightness to 100 for my lamp"]}}
2016-09-17 13:56:14,280 - Skills - DEBUG - {"message_type": "SetBrightnessIntent", "context": {"target": null}, "metadata": {"confidence": 0.3344594594594595, "Group": "lamp", "target": null, "intent_type": "SetBrightnessIntent", "Value": "100", "BrightnessKeyword": "brightness", "utterance": "set the brightness to 100 for my lamp"}}
2016-09-17 13:56:14,281 - phue - DEBUG - {'bri': 254}
2016-09-17 13:56:14,281 - phue - DEBUG - PUT /api/F9uWIDcslLgVb54fG47wxyR7VjT6ZfDVq6LtVeCk/groups/2/action {"bri": 254}
2016-09-17 13:56:14,299 - phue - DEBUG - [{"success":{"/groups/2/action/bri":254}}]
2016-09-17 13:56:14,299 - phue - DEBUG - [[{u'success': {u'/groups/2/action/bri': 254}}]]
2016-09-17 13:56:14,299 - phue - DEBUG - {'on': True}
2016-09-17 13:56:14,300 - phue - DEBUG - PUT /api/F9uWIDcslLgVb54fG47wxyR7VjT6ZfDVq6LtVeCk/groups/2/action {"on": true}
2016-09-17 13:56:14,327 - phue - DEBUG - [{"success":{"/groups/2/action/on":true}}]
2016-09-17 13:56:14,327 - phue - DEBUG - [[{u'success': {u'/groups/2/action/on': True}}]]
2016-09-17 13:56:20,352 - Skills - DEBUG - {"message_type": "recognizer_loop:record_begin", "context": null, "metadata": {}}
2016-09-17 13:56:23,139 - Skills - DEBUG - {"message_type": "recognizer_loop:record_end", "context": null, "metadata": {}}
2016-09-17 13:56:23,147 - Skills - DEBUG - {"message_type": "recognizer_loop:wakeword", "context": null, "metadata": {"session": "802db0e1-66a7-4be1-a85a-2a94051d2378", "utterance": "hey mycroft"}}
2016-09-17 13:56:24,645 - Skills - DEBUG - {"message_type": "recognizer_loop:utterance", "context": null, "metadata": {"session": "802db0e1-66a7-4be1-a85a-2a94051d2378", "utterances": ["set my lights to 10"]}}
2016-09-17 13:56:24,651 - Skills - DEBUG - {"message_type": "SetBrightnessIntent", "context": {"target": null}, "metadata": {"LightsKeyword": "lights", "confidence": 0.3684210526315789, "target": null, "intent_type": "SetBrightnessIntent", "Value": "10", "utterance": "set my lights to 10"}}
2016-09-17 13:56:24,651 - phue - DEBUG - {'bri': 25}
2016-09-17 13:56:24,652 - phue - DEBUG - PUT /api/F9uWIDcslLgVb54fG47wxyR7VjT6ZfDVq6LtVeCk/groups/0/action {"bri": 25}
2016-09-17 13:56:24,672 - phue - DEBUG - [{"success":{"/groups/0/action/bri":25}}]
2016-09-17 13:56:24,673 - phue - DEBUG - [[{u'success': {u'/groups/0/action/bri': 25}}]]
2016-09-17 13:56:24,673 - phue - DEBUG - {'on': True}
2016-09-17 13:56:24,675 - phue - DEBUG - PUT /api/F9uWIDcslLgVb54fG47wxyR7VjT6ZfDVq6LtVeCk/groups/0/action {"on": true}
2016-09-17 13:56:24,702 - phue - DEBUG - [{"success":{"/groups/0/action/on":true}}]
2016-09-17 13:56:24,703 - phue - DEBUG - [[{u'success': {u'/groups/0/action/on': True}}]]
2016-09-17 13:56:28,158 - Skills - DEBUG - {"message_type": "recognizer_loop:record_begin", "context": null, "metadata": {}}
2016-09-17 13:56:32,949 - Skills - DEBUG - {"message_type": "recognizer_loop:record_end", "context": null, "metadata": {}}
2016-09-17 13:56:32,955 - Skills - DEBUG - {"message_type": "recognizer_loop:wakeword", "context": null, "metadata": {"session": "802db0e1-66a7-4be1-a85a-2a94051d2378", "utterance": "hey mycroft"}}
2016-09-17 13:56:34,334 - Skills - DEBUG - {"message_type": "recognizer_loop:utterance", "context": null, "metadata": {"session": "802db0e1-66a7-4be1-a85a-2a94051d2378", "utterances": ["set the brightness of my light - 50%"]}}
2016-09-17 13:56:34,343 - intent__init__ - ERROR - 
Traceback (most recent call last):
  File "/home/chris/Code/mycroft-core/mycroft/skills/intent/__init__.py", line 47, in handle_utterance
    utterance, num_results=100))
StopIteration
2016-09-17 13:56:34,344 - Skills - DEBUG - {"message_type": "intent_failure", "context": null, "metadata": {"utterance": "set the brightness of my light - 50%"}}
2016-09-17 13:56:34,344 - wolfram_alpha__init__ - DEBUG - Could not determine intent, falling back to WolframAlpha Skill!
2016-09-17 13:56:34,346 - Skills - DEBUG - {"message_type": "enclosure.mouth.think", "context": null, "metadata": {}}
2016-09-17 13:56:34,348 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): cerberus.mycroft.ai
2016-09-17 13:56:40,615 - requests.packages.urllib3.connectionpool - DEBUG - "GET /wolframalpha/v2/query?input=set+the+brightness+of+my+light+-+50%25 HTTP/1.1" 200 534
2016-09-17 13:56:40,630 - Skills - DEBUG - {"message_type": "speak", "context": null, "metadata": {"utterance": "Sorry, I couldn't find an answer for set the brightness of my light - 50%. Perhaps you meant light - 50% instead?"}}
2016-09-17 13:56:40,634 - Skills - DEBUG - {"message_type": "recognizer_loop:audio_output_start", "context": null, "metadata": {}}
2016-09-17 13:56:40,635 - Skills - DEBUG - {"message_type": "enclosure.eyes.blink", "context": null, "metadata": {"side": "b"}}
2016-09-17 13:56:40,957 - Skills - DEBUG - {"message_type": "enclosure.mouth.viseme", "context": null, "metadata": {"code": "4:0.062,3:0.184,0:0.285,2:0.408,0:0.514,4:0.642,0:0.748,3:0.880,2:0.968,3:1.013,3:1.107,3:1.140,5:1.249,0:1.398,3:1.463,3:1.494,0:1.541,3:1.627,0:1.769,3:1.830,3:1.953,2:2.001,5:2.140,6:2.260,2:2.275,3:2.421,0:2.516,3:2.571,3:2.647,0:2.758,4:2.850,2:2.940,0:3.046,3:3.118,3:3.163,0:3.227,3:3.356,0:3.435,5:3.481,4:3.545,0:3.660,3:3.747,0:3.901,3:3.990,5:4.097,0:4.195,5:4.272,3:4.371,0:4.468,4:4.579,2:4.729,3:4.840,0:4.947,3:5.047,3:5.180,4:5.288,"}}
2016-09-17 13:56:46,350 - Skills - DEBUG - {"message_type": "recognizer_loop:audio_output_end", "context": null, "metadata": {}}
2016-09-17 13:56:46,354 - Skills - DEBUG - {"message_type": "recognizer_loop:audio_output_start", "context": null, "metadata": {}}
2016-09-17 13:56:46,357 - Skills - DEBUG - {"message_type": "enclosure.eyes.blink", "context": null, "metadata": {"side": "b"}}
2016-09-17 13:56:46,552 - Skills - DEBUG - {"message_type": "enclosure.mouth.viseme", "context": null, "metadata": {"code": "4:0.150,4:0.208,2:0.244,0:0.321,0:0.470,4:0.532,3:0.615,0:0.701,2:0.728,4:0.818,0:0.915,3:1.004,3:1.078,3:1.140,0:1.318,3:1.407,5:1.514,0:1.641,5:1.720,3:1.819,0:1.918,4:2.029,2:2.179,3:2.322,0:2.404,3:2.479,3:2.554,0:2.613,3:2.685,3:2.807,3:2.840,0:2.972,3:3.101,4:3.272,"}}
2016-09-17 13:56:49,950 - Skills - DEBUG - {"message_type": "recognizer_loop:audio_output_end", "context": null, "metadata": {}}
2016-09-17 13:56:55,300 - Skills - DEBUG - {"message_type": "recognizer_loop:record_begin", "context": null, "metadata": {}}
2016-09-17 13:56:58,806 - Skills - DEBUG - {"message_type": "recognizer_loop:record_end", "context": null, "metadata": {}}
2016-09-17 13:56:58,808 - Skills - DEBUG - {"message_type": "recognizer_loop:wakeword", "context": null, "metadata": {"session": "802db0e1-66a7-4be1-a85a-2a94051d2378", "utterance": "hey mycroft"}}
2016-09-17 13:57:00,358 - Skills - DEBUG - {"message_type": "recognizer_loop:utterance", "context": null, "metadata": {"session": "802db0e1-66a7-4be1-a85a-2a94051d2378", "utterances": ["set my lights to 10%"]}}
2016-09-17 13:57:00,362 - intent__init__ - ERROR - 
Traceback (most recent call last):
  File "/home/chris/Code/mycroft-core/mycroft/skills/intent/__init__.py", line 47, in handle_utterance
    utterance, num_results=100))
StopIteration
2016-09-17 13:57:00,363 - Skills - DEBUG - {"message_type": "intent_failure", "context": null, "metadata": {"utterance": "set my lights to 10%"}}
2016-09-17 13:57:00,364 - wolfram_alpha__init__ - DEBUG - Could not determine intent, falling back to WolframAlpha Skill!
2016-09-17 13:57:00,365 - Skills - DEBUG - {"message_type": "enclosure.mouth.think", "context": null, "metadata": {}}
2016-09-17 13:57:00,367 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): cerberus.mycroft.ai
2016-09-17 13:57:05,691 - requests.packages.urllib3.connectionpool - DEBUG - "GET /wolframalpha/v2/query?input=set+my+lights+to+10%25 HTTP/1.1" 200 460
2016-09-17 13:57:05,709 - Skills - DEBUG - {"message_type": "speak", "context": null, "metadata": {"utterance": "I couldn't get a response for set my lights to 10%. Maybe you meant lights?"}}
2016-09-17 13:57:05,712 - Skills - DEBUG - {"message_type": "recognizer_loop:audio_output_start", "context": null, "metadata": {}}
2016-09-17 13:57:05,713 - Skills - DEBUG - {"message_type": "enclosure.eyes.blink", "context": null, "metadata": {"side": "b"}}
2016-09-17 13:57:05,942 - Skills - DEBUG - {"message_type": "enclosure.mouth.viseme", "context": null, "metadata": {"code": "4:0.128,0:0.188,3:0.320,2:0.408,3:0.453,3:0.556,3:0.606,3:0.685,0:0.753,3:0.833,0:0.893,2:1.035,0:1.074,3:1.204,4:1.249,0:1.377,3:1.476,3:1.585,5:1.652,6:1.737,2:1.753,3:1.906,0:2.014,3:2.078,4:2.135,0:2.264,3:2.344,0:2.490,3:2.560,3:2.647,3:2.700,0:2.736,3:2.861,0:2.986,3:3.085,4:3.173,2:3.307,3:3.433,0:3.540,3:3.624,3:3.700,4:3.808,"}}
2016-09-17 13:57:09,840 - Skills - DEBUG - {"message_type": "enclosure.eyes.blink", "context": null, "metadata": {"side": "b"}}
2016-09-17 13:57:09,846 - Skills - DEBUG - {"message_type": "recognizer_loop:audio_output_end", "context": null, "metadata": {}}
2016-09-17 13:57:09,850 - Skills - DEBUG - {"message_type": "recognizer_loop:audio_output_start", "context": null, "metadata": {}}
2016-09-17 13:57:09,856 - Skills - DEBUG - {"message_type": "enclosure.eyes.blink", "context": null, "metadata": {"side": "b"}}
2016-09-17 13:57:09,973 - Skills - DEBUG - {"message_type": "enclosure.mouth.viseme", "context": null, "metadata": {"code": "4:0.212,4:0.273,0:0.429,4:0.484,0:0.591,0:0.676,2:0.718,4:0.811,0:0.904,3:0.977,3:1.054,3:1.115,0:1.297,3:1.380,3:1.453,4:1.587,"}}
2016-09-17 13:57:11,623 - Skills - DEBUG - {"message_type": "recognizer_loop:audio_output_end", "context": null, "metadata": {}}

Explicitly adding the '%' to the pattern made no difference.

@clusterfudge
Copy link
Contributor

clusterfudge commented Sep 17, 2016

This is a case where Adapt is a little bit brittle. I hadn't expected regex entities to be used to parse out part of a token (i.e. the '50' from '50%'), and as such, there's no test cases for it, and totally expectedly it doesn't work :).

I don't have an easy fix for adapt at this time, however I can recommend a change to your code. I was able to get the intent to fire by changing your regex slightly.

from

(?P<Value>100|[1-9][0-9]?)(\D|$)

to

(?P<Value>100|[1-9][0-9]?%?)(\D|$)

The optional '%' allowed both 50 and 50% to parse correctly, and triggered the intent for me. It would then be on you to strip out the percent symbol in your intent handler. I don't have any good ideas on how to fix this in Adapt right now, but will reference this pull/comment in the event I come up with something.

I also don't have a good explanation as to why your intent test is succeeding when it didn't work as part of mycroft-core; I've been out of the project for a while, and running unit tests in a debugger is currently broken for me. I don't have the time to invest into getting it working again, so I'm just going to leave that as a mystery for the time being.

@ChristopherRogers1991
Copy link
Collaborator Author

Thanks! I updated the regex to allow for the optional '%.'

@ChristopherRogers1991
Copy link
Collaborator Author

ChristopherRogers1991 commented Dec 10, 2016

I've opened this pull request to the skills repo. Assuming that's where skills are going to be tracked going forward, perhaps we should close this out in favor of that one?

@aatchison
Copy link
Contributor

@ChristopherRogers1991
I was thinking the same. Close if we are in agreement.

@ethanaward
Copy link
Contributor

Closing since this has been merged into the skills repo with MycroftAI/mycroft-skills/#3

@ethanaward ethanaward closed this Dec 20, 2016
emphasize pushed a commit to emphasize/mycroft-core that referenced this pull request Jan 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants