openHAB Google Assistant

openHAB Google Assistant is based on Google Cloud Function powered by Firebase and realized by Node.js. This serverless application connects the Google Assistant platform with the users openHAB instance through the openHAB Cloud service and lets the user control IoT devices through the Google Assistant. The openHAB Smart Home app lets you connect, query, and control devices through openHAB cloud infrastructure.

openHAB-cloud is the Smart Home IoT cloud engine in this setup and provides both the main openHAB business logic for the web services and proxying, as well as the web portal used to administrate the granted application in the frontend. It handles authentication, and ultimately handles requests from the Google Assistant. openHAB-cloud is also the access point and backend for the Node.js based openhHAB Google Cloud function app that acts as mediator and adapter code. This Adapter will receive commands from the Google Assistant and has listeners for POST requests for receiving SYNC, QUERY or EXEC smart home device control messages towards the openHAB-cloud. The path for requests to this adapter is /openhabGoogleAssistant.

Google Home Graph: The Google related parts of any Smart Home action rely on Google Home Graph, a database that stores and provides contextual data about the home and its devices. For example, Home Graph can store the concept of a living room that contains multiple types of devices (a light, television, and speaker) from different manufacturers. This information is passed to the Google Assistant in order to execute user requests based on the appropriate context.

General Instructions

Requirements

• Google account with "Actions on Google" and "Google Cloud Functions" access
• oAuth2 Server/Provider (like Google Cloud or Amazon Login)
• openHAB server that a Google Cloud service endpoint can access

Google Cloud Functions

• Enable the Cloud Functions API and install the Google Cloud SDK by following this quickstart
• Modify functions\config.js.
  1. Change host to point to your openhab cloud instance, for example: openhab.myserver.com. Do not include https, if you do you'll get DNS errors.
  2. Change path to the rest API. Defaults to rest\item. The trailing slash is important, you'll get a 404 if you leave it off.
• Deploy the openhabGoogleAssistant (openhab home automation) function with the following command

cd openhab-google-assistant/functions
gcloud beta functions deploy openhabGoogleAssistant --stage-bucket staging.<PROJECT ID>.appspot.com --trigger-http

• This commands will deploy the function to Google Cloud and give you the endpoint address. Keep the address somewhere, you'll need it (something like https://us-central1-<PROJECT ID>.cloudfunctions.net/openhabGoogleAssistant).

Create OAuth Credentials on Google Cloud

You'll need to create OAuth credentials to enable API access.

• Visit the Credentials Page
  1. Select "Create Credentials" -> "OAuth client id"
  2. Select Web Application and give it a name. I left the restrictions open.
• Copy the client id and the client secret, you'll need these in the next step.

Actions on Google

Actions on Google is Google's platform for developers to extend Google Assistant. Here you need to develop your actions to engage users on Google Home, Pixel, and other surfaces where the Google Assistant is available.

• Create and setup an "Actions on Google" project on the Actions Console using the Actions SDK.
  1. Select your existing project
  2. Select "Smart Home Actions". The fulfilment URL is the one saves from the glcoud beta functions you saved earlier.
  3. Fill out all the App information. Feel free to use fake data and images, you're not actually going to submit this.
  4. Move on to Account linking.
  • Select Authorization Code
  • Enter the client ID and client secret from the OAuth Credentials you created earlier
  • Authorization URL should be something like: https://openhab.myserver.com/oauth2/authorize
  • Token URL should be something like https://openhab.myserver.com/oauth2/token
  • The scope thing is completely unknown and appears to be broken. I set it to google-assistant but it doesn't seem to do what I think it should do.
  • Testing instructions: "None"
  1. Hit save. You're not actually going to submit this for testing, we just need to set it up so we can deploy it later.

Deploy your action

When you ask your assistant to “Turn on the light”, it will use the auth bearer Token and call the specified endpoint. To specify which endpoint the Google Assistant should call, you need to create an action.json similar to the one below, with your endpoint URL.

• Update the openhab-google-assistant/action.json file and specify the Google Cloud Functions endpoint. This is not your server, this is the endpoint given to you from the call to gcloud beta functions

{
  "actions": [{
    "name": "actions.devices",
    "deviceControl": {
    },
    "fulfillment": {
      "conversationName": "automation"
    }
  }],
  "conversations": {
    "automation" :
    {
     "name": "automation",
     "url": "https://YOUR-OPENHAB-CLOUD-URL/openhabGoogleAssistant"
    }
  }
}

• Afterwards deploy this action file using the following command:

gactions update --action_package action.json --project <YOUR-APPLICATION>

Google Assistant will call the service endpoint: https://YOUR-OPENHAB-CLOUD-URL/openhabGoogleAssistant. This web service will receive parameters (intents) from Google and will query/modify openHAB items through openHAB-cloud depending on those parameters.

• You need to Add "App information”, including name and account linking details to the Actions Console
• Afterwards please run the following command in the gaction CLI:

gactions test --action_package action.json --project <YOUR-APPLICATION>

Setup your Database

• SSH into to your openhab cloud instance
• Open the mongodb client mongo and enter these commands

use openhab
db.oauth2clients.insert({ clientId: "<CLIENT-ID>", clientSecret: "<CLIENT SECRET>"})
db.oauth2scopes.insert({ name: "any"})
db.oauth2scopes.insert( { name : "google-assistant", description: "Access to myopenHAB specific API for Actions on Google Assistant", } )

Testing & Usage on Google App

• Make sure Google Play Services is up to date
• Visit "Google" app entry in Google Play Store on Android
• Set up the voice-activated speaker, Pixel, or Android phone (version 6+) with the same test account
• Make sure you're the correct user
• Start the updated Google Home app on your phone
• Go to the devices Settings > Home control > Add device and select the [test] open hab
• Login at your Backend (e.g. myopenhab.org) with your username and password
• You will now be able to see your previously tagged items and devices
• You can now control those devices from the Google assistant

If you're lucky this works! You'll need to configure your items (below) and then sync again. If it didn't work, try the workaround below.

Workaround for scope issues

If you're getting error messages about an unknown scope, you can try this:

• SSH into to your openhab cloud instance
• Edit the file routes/oauth2.js:
  1. Comment out line 121: scope = req.oauth2.req.scope; and insert the following line above it: scope = 'any';

  #scope = req.oauth2.req.scope;
  scope = 'any'
  

• Restart your server and attempt to authorize again.

Item configuration

• In openHAB 2 Items are exposed via Homekit tags, the following is taken from the homekit binding in openHAB2:

  Switch KitchenLights "Kitchen Lights" <light> (gKitchen) [ "Lighting" ]
  Dimmer BedroomLights "Bedroom Lights" <light> (gBedroom) [ "Lighting" ]
  

Currently the follwoing Tags are supported (also depending on Googles API capabilities):

• ["Lighting"]
• ["Switchable"]
• ["CurrentTemperature"] (will be added soon)
• ["Thermostat"] (will be added soon)

The following screenshots show the setup and the service linkage (myopenhab.org) procedure within the Google App:

Example Voice Commands

Here are some example voice commands:

• Turn on Office Lights
• Turn off Pool Waterfall
• Turn on House Fan
• Turn on Home Theater Scene

Logging & Debugging

To check your deployed openhHAB Google Cloud function app logs and debugging use the follwoing command:

gcloud beta functions logs read openhabGoogleAssistant

Limitations & Known Issues

• Sometimes the Account Linkage needs to be done twice and repeated
• Currently there is support for any switchable device (light, plug etc.)
• Thermostats are not fully implemented
• More Unit Test & Integration Test will be added soon

References

• https://developers.google.com/actions/extending-the-assistant
• https://developers.google.com/actions/smarthome/
• https://cloud.google.com/functions/docs/how-to