Skip to content

Home Assistant

Jump to bottom Edit New page
ArtMG edited this page Jan 9, 2020 · 21 revisions

Home Assistant

This article looks specifically at the home automation software called Home Assistant (HA or Hass).

See also:

About the project

Home Assistant (HA) was started by Paulus Schoutsen in Sep 2013 when he posted his automation script, turning on the lights at sunset, onto github. In April 2018 Ubiquiti hired Paulus as a dedicated employee to work full time on HA, ostensibly to move it closer to being 'production-ready' as well as capable of integrating with its own products. Although community members might be suspicious of such a move, there is no sign of undue influence over the project by this vendor - I had been a HA user for nearly six months before I discovered the Ubiquiti link.

Paulus and the founder of Hass.IO, Pascal Vizeli, formed Nahu Casa Inc in Sep 2018 to run Home Assistant Cloud services. This is an alternative to self-hosting components on premises, especially to allow a secure ingress for webhooks and UIs. The service charge is used to host not only HAC, but also to cover the costs of hosting the community infrastructure.

Comparing deployment options

Home Assistant is software that can be installed on a variety of operating systems. There are three basic modes of deployment:

  • python venv - install Home Assistant on an existing device to try it out
  • Docker - fully flexible containerised services
  • Hass.io - dedicated device using custom OS and basic containers

venv

Installing Home Assistant into a Python Venv (virtualisation at the python layer) is an easy and efficient way to isolate a single python application on a machine.

The method is covered in detail in the article:

https://github.com/artmg/MuGammaPi/wiki/home-assistant-venv

docker

  • you need to set up your base OS and install docker
  • you have full control and full access to the OS, with a little more overhead perhaps
  • add the home-assistant image to get started
  • add an additional container for each additional third party features you want
  • of course you have to configure all this yourself
  • but you can drive this through version-controlled config files for easy restore and rollback

Despite it seeming like a lot to take in, it can actually be very quick and straightforward to get going

https://github.com/artmg/MuGammaPi/wiki/Home-Assistant-Docker

Hass.IO

  • Hass.io images are available for a quick, turnkey solution to set up a dedicated device
  • they are based on HassOS
    • using buildroot, this has some hardware integration built-in
    • it has a basic environment for running Docker containers
    • containers: home-assistant, supervisor, plus one for each add-on
    • although there is an 'ssh add on' it's limited to application executables
  • The Hass.io Supervisor container has it's own specific GUI
    • this is how you manage it's own specific Add-ons ecosystem
      • Home Assistant cannot use Hass.io add-ons
      • however you can sually install the software features yourself 'the hard way'
  • there is no command line access to the HassOS - the supervisor GUI is the only control
  • If you have a regular OS and really want to install Hass.io, you can simply follow the Generic Linux Installer instructions

Hass.io image

This is intended as the simplest method, which also has the widest range of built-in features.

Getting more into Hass.io

  • upon boot, the ethernet lights should come on within 20 seconds or so (otherwise it might be jammed at startup)

  • using the physical console (directly connected screen and keyboard) you can log on as root with no password

    • this gives the Hass.io CLI
    • commands such as hw info
    • login gives access to HassOS host
      • this might pause as it spins up a docker container
    • you can see the complex 8 partitions
    • with the boot partition /mnt/boot/config.txt
    • the host has limited features, enough to host the docker containers

  • Hass.io Add-ons:

  • Enable Host SSH

DNS Issues

Although the simplicity and container separation is good, it did give me problems with DNS resolution in my network.

Hass.io DNS issues

Inside containers cannot resolve!

  • check in hassos ( -p 22222 ) vs ssh container
    • cat /etc/resolv.conf

tried...

vi /etc/NetworkManager/system-connections/default
[ipv4]
method=manual
address=192.168.1.28/24,192.168.1.254
dns=192.168.1.254

According to Hass.io devs this is a 'docker issue', so perhaps I'll leave Hass.io alone for now, and come back and try again in a few months. In the meantime, plain old Home Assistant works well for me.

Workflow

Development process

Your overall steps in building out your configuration may involve:

  • Configuration tweaks:
    • Begin editing entities in web UI
    • Check the dev_state to see the settings
    • move important settings into configuration files

Environments:

  • set up a 'dev' instance in a virtual machine (e.g. Lubuntu on Virtualbox)
  • save your config changes to GitLab (private repos for free)
  • when tested, pull this config to your Live device

Configuration outside of yaml

Although your main means of controlling the HA config is via the configuration.yaml file and its subordinates, there is also some data from the GUI that goes into other places:

  • .storage subfolder
    • auth
      • user accounts for GUI
    • auth_provider.homeassistant
      • hashed passwords for GUI users
    • core.entity_registry
      • discovered devices that have had custom attributes entered in the GUI
      • e.g. entity_id renamed to something more friendly
  • home-assistant_v2.db
    • this is generally the historical data recored

Initial configuration in git

see https://github.com/artmg/lubuild/blob/master/help/use/git-source-control.md#workflow-examples

  • home assistant automatically creates a default 'starting' configuration file-set on first run
  • ensure you are in the folder containing these
  • follow process for creating a new repo
  • use the following for the initial load
tee -a .gitignore << EOF!
/secrets.yaml
/home-assistant_v2.db*
/home-assistant.log
/.uuid
/.storage
/.HA_VERSION
EOF!

Custom Components

Manual components

Custom Components are Python projects that can be called as components in your configuration.yaml file to extend core features. If you are trying to use custom components manually then you will need to put them into the custom_components subfolder of your config. You will find insight into the structure in https://community.home-assistant.io/t/how-to-package-custom-components/107475/10

To enable the component simply add a section into your configuration yaml

component_name:

but beware as they are not supported - you might want to keep them in a separate section of your config file so you can easily disable them if you want to diagnose an issue with the supported core of HA.

HACS

This is a Community Store allowing you to browse, install and update custom components from github

sudo -u homeassistant -H -s
HACS_DIR=/home/homeassistant/.homeassistant/custom_components
mkdir $HACS_DIR
cd $HACS_DIR

hacs:
  token:
  • pull the latest config version
  • restart the server
  • ensure you have added firewall allow rules for github.com

NB: if you wish you can still use custom components manually without HACS (see below)

Custom cards

Custom Cards are extensions to the Lovelace UI system which come as JavaScript files that need to be served by the webserver.

Manual example

The folder containing the HA configuration files is considered as an alias to /config so a ./www/ subfolder of it will be available at the front-end. Lovelace configuration has an internal alias to www called local where you must supply explicit filenames including versions. There is also the customcards alias, where no version is needed.

https://github.com/kalkih/mini-media-player

sudo -u homeassistant -H -s
mkdir -p www
cd www
wget https://github.com/kalkih/mini-media-player/releases/download/v1.2.2/mini-media-player-bundle.js
  • add this to your ui-lovelace.yaml
resources:
  - url: /local/mini-media-player-bundle.js?v=1.2.2
    type: module

custom components in workflow

Using the manual approach to custom components will keep all files within the configuration folder and its subfolders. This makes them easy to manage, even in a Docker deployment.

You may choose to leave all custom components to be stored and managed within your own git config repository. However you might consider that this constitutes unnecessary duplication and decide instead to exclude the content but reference the source as follows.

  • in your configuration folder...
    • add an exclusion into your .gitignore file
      • www
    • build a script to pull all external components upon demand
      • e.g. www-get-components.sh 
# pull external components into www subfolder of configuration directory
(
  cd "$(dirname "$0")"
  mkdir -p www
  cd www

  wget https://github.com/kalkih/mini-media-player/releases/download/v1.4.1/mini-media-player-bundle.js
)

Abstractions

Although this might be a little hard to get your head around at first, you can abstract entities in HA to make things easier to use. For example you can create groups of objects, so that several items can be controlled simultaneously. Or you can use automations to define behaviours, that can then be controlled via the interface or by other automations.

Light Groups

Here is a simple abstraction, but useful in a room with many identical lights.

https://www.home-assistant.io/components/light.group/

Note that the resulting entity ids will be based on the name but using the typical entity ID format, lowercase, underscores for spaces, dashes and non-alphnums e.g. Kitchen Lights becomes light.kitchen_lights

For some background on alternatives to Light Groups with pros and cons see

Entity renaming using groups

Entity IDs for many devices are automatically generated upon discovery. Although these could be modified through the front end, which makes changes in the Entity Registry, there is another way to rename entity IDs purely through configuration:

  • Using Groups that contain a single entity

Validate that nested groups are acceptable

Automations to give human spaces context

Thanks to @Tinkerer in HA Discord channels for these suggestions

Scenes

A scene is a desired state that you would like to reproduce

Scenes vs Scripts

Capture scene from current settings

Scenegen to capture current settings into yaml files for Scenes

https://github.com/home-assistant/scenegen

# credit https://www.home-assistant.io/docs/ecosystem/scenegen/
git clone https://github.com/home-assistant/scenegen.git
cd scenegen
sudo pip3 install configparser
API tokens
  • Create a Long Lived Access Token (LLAT):
    • log onto instance
    • show the Hass Menu (top left)
    • click on the (i) account info symbol to see your Profile
    • Long-Lived Access Tokens / Create Token
    • copy the token and save somewhere safe
scenegen example
LLAT=...
PORT=443
./scenegen.py https://server.domain.name:$PORT/api/states -k $LLAT

[certificate error]

tried:

sudo pip3 install certifi

sudo pip3 install requests[security]

requests.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",)

Template switch groups

If you want to group items from different domains, say lights and switches, but have them work as a single group, you can use a template switch - see some examples

Remote system operations

To permit this server to remotely operate other systems, for example rebooting or shutting them down, you first of all need to be in the context of the service that will make the remote calls, e.g.

# in a venv install
#
# HA will act in the following context, 
# so become that user before creating the keys
sudo -u homeassistant -H -s

# otherwise
pushd /home/$LOCAL_SERVICE_USER

# Exit the HA user context
exit

# OR the folder
popd

example of using this

reboot_test:
  alias: Reboot Test Script
  sequence:
    - service: shell_command.reboot_test

shell_command:
  myserver_reboot: 'ssh -i /home/homeassistant/.ssh/sysmain_reboot_key sysmain@my_server'

Hardware installation

Serial device

If you use Home Assistant and have access to a console or SSH, you can check out which device is being used

# discover serial device
ls -la /dev/serial* 
# or 
ls -la /sys/class/tty/ | grep serial
or 
ls -la /sys/class/tty/*/device/driver

If you get permissions errors in the logs, it may be because Raspbian defaults to making a console available via GPIOs.

# check whether root can use the serial device
sudo echo AT >/dev/ttyXXXX
# a Permission Denied error with sudo indicates the device is in use

You can disable this to allow your device integration full access to the port:

sudo cp /boot/cmdline.txt /boot/cmdline.txt.original
# remove option   console=serial0,115200
sudo editor /boot/cmdline.txt

Displays

Fully Kiosk Browser

Using a tablet as a Home Assistant display should have been a great idea. Unfortunately the default power management of a tablet expects you to either have it in your hand, fondling it, or being left ununsed. This made the Family User Experience quite poor.

Upon many a recommendation in the HA community I tried Fully Kiosk Browser and all of a sudden the tablet display was worked as I had always dreamed it should.

The steps to setting it up were pretty simple:

  • install Fully Kiosk
    • Play Store
      • search Fully Kiosk
      • ensure you got Fully Kiosk Browser & App Lockdown by Fully Factory Kiosk Solutions
      • Install
    • Allow all the permissions it asks for (virtual omnipotence)
    • Remote Admin - ON
    • Set the password

Note that I chose the Remote Admin which is a PLUS (paid) option. I would rather be sitting at my PC to set it up, rather than standing by the kitchen wall. And one of my key drivers for using Fully is the Device Wakeup Automation, which is another PLUS option. And at the end of the day, 6 euros is a small price for a happy family.

Note the flip-side of the display always being ready to spring into action is an increase in power usage. This means I'm having to enable the switch controlling it's power supply almost daily instead of weekly or fortnightly.

There is a custom component that allows control of the display https://github.com/daemondazz/homeassistant-displays but if you simply want to read the battery sensor see https://community.home-assistant.io/t/fully-kiosk-browser/41675

See some example Automations for battery sensors https://community.home-assistant.io/t/charge-my-android-tablet-automatically-when-battery-goes-less-than-10-stop-charging-when-100/61296 and how to use the sensor states https://community.home-assistant.io/t/automation-based-on-value-template/126670/2

Thing connections

Controlling devices without proprietary gateway

Many smart device systems are built around a proprietary gateway device. Examples include Ikea Trådfri and Philips Hue lighting systems. However as these are all simply Zigbee meshes where the gateway provides an API for application-level access, it should be feasible to avoid the gateway.

Candidates:

NB: Early versions of some hardware may not be fully compatible, and may require the proprietary gateway to upgrade their firmware. https://community.home-assistant.io/t/52108/158

Hue Bulb reset

As I could not reset Hue bulbs with an on/off light sequence, I set up a separate python venv for huethief

#### Set up the venv and software
# install system packages if not already installed
sudo apt install -y git python3-venv

# create the folder and permission it for interactive use
sudo mkdir /srv/huethief
sudo chown pi:pi /srv/huethief

# create the venv, add the software and the python prereqs
python3 -m venv /srv/huethief
cd /srv/huethief
git clone https://github.com/vanviegen/hue-thief
cd /srv/huethief/hue-thief
source /srv/huethief/bin/activate
pip3 install wheel
pip3 install -r requirements.txt


#### run the software
cd /srv/huethief/hue-thief
source /srv/huethief/bin/activate
sudo systemctl stop home-assistant@homeassistant

# power on the devices and immediately run...
python3 hue-thief.py /dev/ttyAMA0

If have installed docker, then there is a really easy to use Dockerfile at https://github.com/vanviegen/hue-thief#docker-image

Sonoff Switch

  • Tried with Tuya app
    • device would not connect to wifi
  • Tried with eWeLink app
    • device recognised and connected to wifi
    • despite plenty of allow rules, could not get it functioning
  • anyhow the solution with the stock firmware requires an unsupported custom component

so...

Flashing the Sonoff

There is extremely detailed documentation at https://github.com/arendst/Sonoff-Tasmota/wiki/Prerequisites but here is an overview

If you

See also firmware listed at https://www.home-assistant.io/docs/mqtt/discovery/#support-by-third-party-tools

Tasmota devices

Sonoff Tasmota without MQTT or custom components

Mechanical devices

Troubleshooting & Diagnostics

YAML help

Sometimes issues arise simply because of yaml file formatting. If you're still new-ish to yaml then this complete idoit's guide will help a lot, and even seasoned yaml-users might get something from this whirlwind explanation of yaml syntax.

Troubleshooting Automations

  • check the configuration.yaml has a valid automation section
    • it could use an automations.yaml file
    • or an automations folder with include dir merge list
  • ensure that the automation is registered at startup
  • If you are using templates for values try them out manually
    • Developer Tools / Template
    • Paste your template into the editor to see the resulting value

bellows devices

This example is for venv

sudo -u homeassistant -H -s
source /srv/homeassistant/bin/activate

# is this needed?
export EZSP_DEVICE=/dev/ttyAMA0

bellows devices -D /srv/Data/zigbee.db

# or
bellows -d /dev/null devices -D /srv/Data/zigbee.db

Removing ZHA devices

If you have issues with ZHA devices then remove them using the Configuration / ZHA pane. Dropping down the list and choosing the name is now simpler than the old method of using the service interface (dev-service) and selecting:

zha.remove {"ieee_addr": "...."}

How to find the ieee address

It is now easier as you can simply Look at the object in the Entity Registry, although the 'Cog' settings menu still tells you the last few bytes.

remove from database

If you still get errors you might try manual removal. Get the IEEE address from the GUI and then follow this.

WARNING: this risks messing up your database, especially if you have not already removed the item as well as you can via the interface. Be Warned! And make sure you hang onto the backup made below.

# install sqlite3 if you don't have it yet

# move to the folder containing your zigbee.db, 
# e.g. where your configuration.yaml is
cd path


ieee_addr=00:15:8d:00:02:9a:78:82
sql_db=zigbee.db

# back up the database just in case
sudo cp $sql_db{,.`date +%y%m%d.%H%M%S`}

sudo sqlite3 $sql_db "delete from attributes where ieee=\""$ieee_addr"\""
sudo sqlite3 $sql_db "delete from devices where ieee=\""$ieee_addr"\""
sudo sqlite3 $sql_db "delete from output_clusters where ieee=\""$ieee_addr"\""
sudo sqlite3 $sql_db "delete from clusters where ieee=\""$ieee_addr"\""
sudo sqlite3 $sql_db "delete from endpoints where ieee=\""$ieee_addr"\""

Maintenance

Config backup

Zigbee Device Backup

Once you have configured your Zigbee devices, you might consider backing up the database file specified in the zha section of configuration.yaml

General backup of databases and storage

# as main (admin) user:

# You should quiesce your services 
# to maintain db integrity during the copy
sudo service home-assistant@homeassistant stop

cd $HOME
STAMP=`date +%y%m%d-%H%M%S`
mkdir backup.$STAMP
cd backup.$STAMP
mkdir data
cp -p ../hadata/* data/
mkdir storage
sudo cp -p ../haconf/.storage/* storage/
cd ..
sudo service home-assistant@homeassistant start
Add a custom footer
Add a custom sidebar
Clone this wiki locally