Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Google Actions for Assistant #9632

Merged
merged 6 commits into from Oct 18, 2017
Merged

Conversation

philk
Copy link
Contributor

@philk philk commented Sep 30, 2017

Description:

I'm looking for some feedback on this both as a component and as a specific implementation.

The current emulated_hue implementation can only work with either the Echo or Google Home, but not both at the same time so I took some of the ideas from the Alexa component.

Google provides a Smart Home API for interacting with Google Assistant. It's a relatively straightforward API that just requires some mapping of Home-Assistant objects to a couple of schemas. I think that part all makes sense and is a reasonable component for hass. I will acknowledge there was a lot of copy paste from the hue and Alexa components to figure out how aiohttp works. Happy to take feedback on the /right/ way to do things for sure 😅

Where I'm most looking for feedback though is really UX, basically the Google Actions deploy and setup process. You have to set up OAuth for your action to authenticate against and the way I did it is...hacky to say the least. I'm really curious what the path forward for the Cloud Auth stuff is but right now it looks pretty AWS specific and more for handling login to hass rather than being an OAuth provider.

I'm also not super happy with having to ask people to go through the setup steps for creating an app in GCP. We could probably break it down to:

  1. Download Google's gactions tool
  2. Copy a json file and fill in some fields
  3. Run the gactions test command with the json file

Which isn't /too/ bad but doesn't feel as user friendly as hass has gotten lately (OMG, not having to deal with Zwave compilation and install, life saving).

Pull request in home-assistant.github.io with documentation (if applicable): home-assistant/home-assistant.io#3657

Example entry for configuration.yaml (if applicable):

google_assistant:
  project_id: someproject-2d0b8
  client_id: supersecretid
  access_token: extradoublesecrettoken
  exposed_domains:
    - switch
    - cover
    - light
    - group

Checklist:

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • Local tests with tox run successfully. Your PR cannot be merged unless tests pass
  • New files were added to .coveragerc.

@philk philk requested a review from a team as a code owner September 30, 2017 21:29
@homeassistant
Copy link
Contributor

Hello @philk,

When attempting to inspect the commits of your pull request for CLA signature status among all authors we encountered commit(s) which were not linked to a GitHub account, thus not allowing us to determine their status(es).

The commits that are missing a linked GitHub account are the following:

Unfortunately, we are unable to accept this pull request until this situation is corrected.

Here are your options:

  1. If you had an email address set for the commit that simply wasn't linked to your GitHub account you can link that email now and it will retroactively apply to your commits. The simplest way to do this is to click the link to one of the above commits and look for a blue question mark in a blue circle in the top left. Hovering over that bubble will show you what email address you used. Clicking on that button will take you to your email address settings on GitHub. Just add the email address on that page and you're all set. GitHub has more information about this option in their help center.

  2. If you didn't use an email address at all, it was an invalid email, or it's one you can't link to your GitHub, you will need to change the authorship information of the commit and your global Git settings so this doesn't happen again going forward. GitHub provides some great instructions on how to change your authorship information in their help center.

    • If you only made a single commit you should be able to run
      git commit --amend --author="Author Name <email@address.com>"
      
      (substituting Author Name and email@address.com for your actual information) to set the authorship information.
    • If you made more than one commit and the commit with the missing authorship information is not the most recent one you have two options:
      1. You can re-create all commits missing authorship information. This is going to be the easiest solution for developers that aren't extremely confident in their Git and command line skills.
      2. You can use this script that GitHub provides to rewrite history. Please note: this should be used only if you are very confident in your abilities and understand its impacts.
    • Whichever method you choose, I will come by to re-check the pull request once you push the fixes to this branch.

We apologize for this inconvenience, especially since it usually bites new contributors to Home Assistant. We hope you understand the need for us to protect ourselves and the great community we all have built legally. The best thing to come out of this is that you only need to fix this once and it benefits the entire Home Assistant and GitHub community.

Thanks, I look forward to checking this PR again soon! ❤️

Copy link
Member

@balloob balloob left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is awesome and off to a great start! Yes, OAuth2 is currently pretty cumbersome but that's something we can improve later. I think for now the current flow is fine.

Added some comments, mainly around let's follow the pattern in alexa/smart_home.py and decouple message handling from I/O.

.coveragerc Outdated
@@ -74,6 +74,9 @@ omit =
homeassistant/components/google.py
homeassistant/components/*/google.py

homeassistant/components/googleactions.py
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not needed, you wrote tests 👍

'{}/r/{}#access_token={}&token_type=bearer&state={}'


def setup(hass: HomeAssistant, yaml_config: Dict[str, Any]):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please make this async

@asyncio.coroutine
def async_setup(hass, config):

def __init__(self, hass: HomeAssistant, cfg: Dict[str, Any]) -> None:
"""Initialize instance of the view."""
super().__init__()
self.hass = hass
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can get the hass instance from the request, you don't have to store it in the instance:

# inside get(…)
hass = request.app['hass']

"""Initialize Google Actions view."""
super().__init__()

self.hass = hass
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same.


return is_default_exposed or explicit_expose

def handle_sync(self, hass: HomeAssistant, request_id: str):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please decouple the I/O from the handling of the messages. Have a look at the Alexa smart home skill as an example: https://github.com/home-assistant/home-assistant/blob/dev/homeassistant/components/alexa/smart_home.py#L32-L44

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would even consider breaking it up in multiple files: google_assistant/smart_home.py and google_assistant/http.py

def post(self, request: Request):
"""Handle Google Action requests."""
auth = request.headers.get('Authorization', '')
if self.access_token not in auth:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's do actual comparison. not in only checks if string A exists inside string B.

@philk
Copy link
Contributor Author

philk commented Oct 2, 2017

Thanks for the feedback, I hadn't looked at the rewritten Alexa component but I like that layout and I'm stealing some ideas from there that should tidy up my code quite a lot 😁
Should just take a few days to re-org things.

@Gagnon06
Copy link

Gagnon06 commented Oct 9, 2017

Hi! Great idea! I found a project that could be use as a reference : Phlex. The setup is somewhat easy. It is actually a server that connects to a Plex server instance to enable voice commands. It connects to Google Assistant and Alexa using a simple login interface. The Google Action is officially accessible from Android phones and Google Home and is named Flex TV. This way, users don't have to create their own private Google Action but still can use a private server.

I tried this project and it seems to work properly, but I haven't look at the source since it is written in php, but I think it could be a good help for you.

Good work!

@balloob
Copy link
Member

balloob commented Oct 9, 2017

We should not depend on Phlex as we should not promote the use of external gateway services.

@Gagnon06
Copy link

Gagnon06 commented Oct 9, 2017

I am not talking about dependency, but more as a design reference.

@philk
Copy link
Contributor Author

philk commented Oct 9, 2017

Yeah, I'm not sure exactly how they do it but I don't want people's credentials passing through a system I run which is the only way I can think of to host one "Home-Assistant" action that shows up for everyone in their Google Home app.

@arsaboo
Copy link
Contributor

arsaboo commented Oct 9, 2017

Can't wait to get this merged!!

@kylerw
Copy link

kylerw commented Oct 13, 2017

Is there setup documentation for this, yet? I'd love to give it a go....

@philk
Copy link
Contributor Author

philk commented Oct 13, 2017

Not yet, should get some time to work on it this weekend.

@isteaks
Copy link

isteaks commented Oct 14, 2017 via email

@philk
Copy link
Contributor Author

philk commented Oct 14, 2017

The way the Google Actions works is either through a service like API.ai (now DialogFlow) or through the Google Compute Platform (via the Actions API). The short version (since I'm currently working on the step by step docs) is:

  1. Create a project in the Google Dev Console
  2. Fill out all the information for the project like URLs, name, etc and get your OAuth credentials.
  3. Put your project in "Test" (exposes it to your account)
  4. Add config keys (project_id, client_id, access_token) to Hass configuration.yaml
  5. Add your app/project in the Google Home app

Like I said in my initial comment it's a lot more complicated than I'd like it to be but at least somewhat less so than the current downgrade the Google Home app process to get emulated_hue working.

@cnrd
Copy link
Contributor

cnrd commented Oct 14, 2017

than the current downgrade the Google Home app process to get emulated_hue working.

I know it probably isn't of much use, but it looks like Google (or Philips) blocked this from working, as even the downgraded app now redirect to the Hue auth page. (I haven't tried downgrading the Google app or anything like that yet, to see if that could fix it).

@roofuskit
Copy link

Emulated Hue will not work at all now.

@isteaks
Copy link

isteaks commented Oct 15, 2017 via email

@kylerw
Copy link

kylerw commented Oct 16, 2017

@philk Sorry to be impatient.... But, any update on the setup guide so some of us can test it?

@kylerw
Copy link

kylerw commented Oct 16, 2017

attempted to set this up and got a good way there. During the setup process within the Home app i'm getting:

Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/aiohttp/web_protocol.py", line 422, in start resp = yield from self._request_handler(request) File "/usr/local/lib/python3.6/site-packages/aiohttp/web.py", line 306, in _handle resp = yield from handler(request) File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 213, in coro res = yield from res File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 213, in coro res = yield from res File "/config/custom_components/http/ban.py", line 58, in ban_middleware_handler return (yield from handler(request)) File "/usr/src/app/homeassistant/components/http/__init__.py", line 428, in handle result = yield from result File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 210, in coro res = func(*args, **kw) File "/config/custom_components/googleactions.py", line 139, in get headers={'Location': generated_url}) TypeError: json_message() got an unexpected keyword argument 'headers'�[0m

@kylerw
Copy link

kylerw commented Oct 16, 2017

Looks like it didn't like the init.py in the custom_components/http folder so I overwrote it. Now I'm getting:

Couldn't update setting. Check your connection.

Within the Home App after it redirects.

@cnrd
Copy link
Contributor

cnrd commented Oct 16, 2017

I get something close to that, it redirects in the Google Home app, goes to a spinner, then says: "Something went wrong, please try again".

- The way I originally wrote the MAPPING_COMPONENT and the way it's actual
  used changed so the comment was updated to match that.
- Use const's in more places
- Handle the ActivateScene command correctly
Was failing on 3.4.2 and 3.5, this is more correct anyway.
@arsaboo
Copy link
Contributor

arsaboo commented Oct 17, 2017

@ALL Guys, let us use this thread in the forums to discuss support/configuration issues and not pollute this PR.

Others will face similar issues and would benefit from this discussion.

@arsaboo
Copy link
Contributor

arsaboo commented Oct 18, 2017

@kylerw Please don't use the PR for support/feature requests.

@home-assistant home-assistant deleted a comment from kylerw Oct 18, 2017
@home-assistant home-assistant deleted a comment from kylerw Oct 18, 2017
@home-assistant home-assistant deleted a comment from cytech Oct 18, 2017
@home-assistant home-assistant deleted a comment from cnrd Oct 18, 2017
@balloob
Copy link
Member

balloob commented Oct 18, 2017

Deleting all comments not related to the code.

@home-assistant home-assistant deleted a comment from cytech Oct 18, 2017
@home-assistant home-assistant deleted a comment from Z-pi Oct 18, 2017
@home-assistant home-assistant deleted a comment from cytech Oct 18, 2017
@home-assistant home-assistant deleted a comment from cnrd Oct 18, 2017
@home-assistant home-assistant deleted a comment from cytech Oct 18, 2017
@home-assistant home-assistant deleted a comment from 333ryan18 Oct 18, 2017
@home-assistant home-assistant deleted a comment from kylerw Oct 18, 2017
@home-assistant home-assistant deleted a comment from cnrd Oct 18, 2017
@home-assistant home-assistant deleted a comment from kylerw Oct 18, 2017
@home-assistant home-assistant deleted a comment from kylerw Oct 18, 2017
@home-assistant home-assistant locked and limited conversation to collaborators Oct 18, 2017
@balloob balloob merged commit 9d20a53 into home-assistant:dev Oct 18, 2017
@balloob
Copy link
Member

balloob commented Oct 18, 2017

🎉 🐬 Thanks.

@philk philk deleted the googleactions branch October 18, 2017 05:54
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet