Installation Docker OCI

It is possible to run RaspberryMatic also as a docker/OCI container on existing amd64, aarch64/arm64 or arm-based systems. While the docker version comes with almost all functionality that a "normal" RaspberryMatic system provides, some functionality like, e.g. the network setup or WebUI-based update mechanism has been explicitly disabled since these operations are usually performed on the system hosting the docker container itself rather than within the container.

Furthermore, while almost the same hardware requirements in terms of homematic rf-modules apply like with a standard RaspberryMatic system, the kernel drivers for, e.g. the RPI-RF-MOD, HM-MOD-RPI-PCB or HmIP-RFUSB as well as for the HB-RF-USB/HB-RF-USB-2 and the HB-RF-ETH have to be installed on the hosting machine rather than within the container. The container then just uses these preloaded kernel drivers. This driver installation can be either performed using a dedicated install-docker.sh installation script or manually. In either cases, kernel modules are installed that are kindly developed and distributed by the author of piVCCU (@alexreinert).

Important

Known limitations

To be able to communicate with a HmIP-HAP or HmIPW-DRAP device, the docker container implementation outlined here utilizes a macvlan docker network which makes it necessary to have two static ip addresses (<IP-ADDRESS> and <AUX-ADDRESS) available from your local network which are outside of any DHCP range (e.g. NOT supplied by your router) and are exclusively assigned to the docker container as well as to your docker host so that both systems (docker host and container) can communicate with eachother.

Prerequisites

1. Ensure you have docker installed on your host system (where the container will run on):

   sudo sh -c "wget -qO - https://get.docker.com | bash -"

2. Make sure to add the current user to the docker group so that it can execute docker commands:

   sudo usermod -aG docker $USER
   exec su -l $USER

3. Make sure the docker environment works correctly by executing the hello-world example docker:

   docker run hello-world

After having applied these steps your host system should be equipped to run all common docker containers.

Installation Procedure

For installing all necessary requirements to create and run the RaspberryMatic docker container one can either use an automatic script-based method (install-docker.sh) or follow manual installation steps which use either docker pull, docker load or even a docker compose based approach to run/manage the RaspberryMatic container.

Please note, that in the following documentation sections one has to specify/replace certain placeholders like <IP-ADDRESS>, <INTERFACE>, etc. with their respective substitutions in configuration files, scripts or executions of helper scripts. To chose the right substitution values for these placeholders they are explained here:

• <IP-ADDRESS>: a manually chosen static ip address (e.g. 192.168.178.4) which will be assigned to the docker container. Make sure to chose an ip address from your normal LAN subnet which is NOT part of your normal DHCP assignable range your router normally supplies to network clients.
• <AUX-ADDRESS>: a manually chosen static ip address (e.g. 192.168.178.3) which will be used as an additional so-called auxiliary ip address for your docker host system (where the container will run on) so that it can communicate with the docker container on the same subnet. Make sure to chose an ip address from your normal LAN subnet which is NOT part of your normal DHCP assignable range your router normally supplies to network clients. Also make sure to NOT use the same IP address like the host is already using.
• <INTERFACE>: the name of the network interface (e.g. eth0) on your docker host system which is used to communicate with your LAN (should be automatically identified).
• <SUBNET>: the subnet specifier (e.g. 192.168.178.0/24) of your normal LAN (should be automatically identified).
• <GATEWAY>: the ip address of the default gateway (e.g. 192.168.178.1) of your normal LAN (should be automatically identified).

Automatic Installation

For users with a lower amount of Linux or docker knowledge or for users who want to use an easy installation procedure, a dedicated all-in-one installation script (install-docker.sh) can be used to install all additional prerequisites for the RaspberryMatic container so that it can be correctly created and will run on the docker equipped host system:

1. Execute the following command to install all necessary dependencies and create the container:

   wget -qO - https://raspberrymatic.de/install-docker.sh | bash -

   The script will make sure all dependencies are installed and might request sudo rights to do so. If this is the case and the script notifies to do so please re-run it with sudo rights accordingly:

   sudo sh -c "wget -qO - https://raspberrymatic.de/install-docker.sh | bash -"

   In addition, this script might request some additional information from you, like specifying a reserved ip address (<IP-ADDRESS>) which will be assigned to the container as well as a so-called auxiliary ip address (<AUX-ADDRESS>) which will be assigned to the host system itself so that it can communicate with the started docker container. Furthermore, it will try to identify the network interface (<INTERFACE>), the subnet (<SUBNET>) and the default gateway ip (<GATEWAY>) automatically, but allows a user to change it during the installation procedure.

2. After the script has created the docker container you should be able to start the container using:

   docker start -i ccu

   NOTE: Omit -i if you want to start the container non-interactive and put it in background.

This script can also be used from time to time to ensure the docker installation is still correct or according to the lastest changes of RaspberryMatic. Furthermore, in case you run into some issues (e.g. because you updated the host system itself) this script could help to get the docker environment running again. Make sure, however, to update the script over time (in case you downloaded it) because we might introduce changes to it as we move forward.

Manual Installation

If you prefer to install all prerequisites, dependencies manually on your docker host system as well as prefer to create the docker container manually or if you are using docker solutions like docker compose, please make sure to following the following procedure in a step-by-step manner.

Container Host Dependencies

To install all necessary third-party dependencies on the docker host system for the RaspberryMatic container to work correctly the following step have to be performed one after another:

1. Install pivccu-modules-dkms kernel modules (required for RPI-RF-MOD, HM-MOD-RPI-PCB or HmIP-RFUSB use) by executing:

   sudo apt install wget ca-certificates build-essential bison flex libssl-dev gpg
   wget -qO - https://apt.pivccu.de/piVCCU/public.key | sudo gpg --dearmor -o /usr/share/keyrings/pivccu-archive-keyring.gpg
   sudo sh -c 'echo "deb [signed-by=/usr/share/keyrings/pivccu-archive-keyring.gpg] https://apt.pivccu.de/piVCCU stable main" >/etc/apt/sources.list.d/pivccu.list'
   sudo apt update
   sudo apt install pivccu-modules-dkms

2. If you are using a RaspberryPi and want to use a RPI-RF-MOD or HM-MOD-RPI-PCB connected to GPIO (NOT necessary if you want to use a HB-RF-USB, HB-RF-USB-2, HB-RF-ETH or HmIP-RFUSB):

   1. Install the necessary device tree patches:

      sudo apt install pivccu-modules-raspberrypi

   2. Disable bluetooth:

      sudo bash -c 'cat <<EOT >>/boot/config.txt
      dtoverlay=disable-bt
      EOT'
      systemctl is-active --quiet hciuart.service && sudo systemctl disable hciuart.service

      NOTE: If there is no /boot/config.txt, you may put these changes under /boot/firmware/usercfg.txt

   3. Disable serial console

      sudo sed -i /boot/cmdline.txt -e "s/console=\(serial0\|ttyAMA0\),[0-9]\+ //"

      And alternative method (if you run into problems) for disabling the serial console or ttyAMA0 would be to run the following script:

      sudo bash -c 'cat <<EOT >>/boot/config.txt
      dtoverlay=miniuart-bt
      core_freq=250
      EOT'

      NOTE: If there is no /boot/config.txt, you may put these changes under /boot/firmware/usercfg.txt

      Disable and mask any serial getty service under your OS so that ttyAMA0 will be free:

      sudo systemctl stop serial-getty@ttyAMA0.service && sudo systemctl disable serial-getty@ttyAMA0.service && sudo systemctl mask serial-getty@ttyAMA0.service 

3. If you are using a HmIP-RFUSB, make sure you DO NOT have a file /etc/udev/rules.d/10-hmiprfusb.rules installed, otherwise delete it:

   sudo rm -f /etc/udev/rules.d/10-hmiprfusb.rules

4. Make sure to execute the following so that a /etc/udev/rules.d/99-Homematic.rules udev rule file is created with the right entries:

   sudo bash -c 'cat <<EOF >/etc/udev/rules.d/99-Homematic.rules
   ATTRS{idVendor}=="1b1f" ATTRS{idProduct}=="c020", ENV{ID_MM_DEVICE_IGNORE}="1"
   ATTRS{idVendor}=="1b1f" ATTRS{idProduct}=="c00f", ENV{ID_MM_DEVICE_IGNORE}="1"
   ATTRS{idVendor}=="0403" ATTRS{idProduct}=="6f70", ENV{ID_MM_DEVICE_IGNORE}="1"
   ATTRS{idVendor}=="10c4" ATTRS{idProduct}=="8c07", ENV{ID_MM_DEVICE_IGNORE}="1"
   EOF'

5. Install eq3_char_loop kernel module to automatic module load section:

   sudo sh -c 'echo eq3_char_loop >/etc/modules-load.d/eq3_char_loop.conf'

6. Start+Load all necessary kernel modules on host:

   sudo service pivccu-dkms start
   sudo modprobe eq3_char_loop

   NOTE: If modprobe eq3_char_loop fails it could be necessary to update the kernel-headers (especially on Raspberry Pi OS 64-bit):

   sudo apt update
   sudo apt install raspberrypi-kernel raspberrypi-kernel-headers raspberrypi-bootloader

   Afterwards you may have to execute the pivccu-dkms installation again and also install the eq3_char_loop kernel module to the automatic module load section:

   sudo apt install pivccu-modules-dkms
   sudo sh -c 'echo eq3_char_loop >/etc/modules-load.d/eq3_char_loop.conf'

7. Create a dedicated macvlan docker network named ccu for the RaspberryMatic container to use (can be skipped if you are going to use the docker compose method to setup and run the container):

   docker network create -d macvlan \
                         --opt parent=<INTERFACE> \
                         --subnet <SUBNET> \
                         --gateway <GATEWAY> \
                         ccu

   NOTE: Make sure to adapt the values for all <XXX> placeholder accordingly (see above).

8. Setup a "shim" network link so that the docker host itself can access the docker container, which usually isn't the case because of network encapsulation:

   sudo ip link add ccu-shim link <INTERFACE> type macvlan mode bridge
   sudo ip addr add <AUX-ADDRESS> dev ccu-shim
   sudo ip link set ccu-shim up
   sudo ip route add <IP-ADDRESS> dev ccu-shim protocol static

   NOTE: Make sure to adapt the values for all <XXX> placeholder accordingly (see above).

   The above ip commands to create a shim network have to be usually executed manually after each reboot of the docker host system because they are not persistent. You have to either execute them each time or add them to a boot-up like script which will be executed on each boot.

   E.g. on Ubuntu/Debian the following content can be put into the file /etc/network/if-up.d/99-ccu-shim-network

   #!/bin/sh
   if [ "$IFACE" = "<INTERFACE>" ]; then
     if ! ip link show ccu-shim >/dev/null 2>&1; then
       ip link add ccu-shim link <INTERFACE> type macvlan mode bridge
       ip addr add <AUX-ADDRESS> dev ccu-shim
       ip link set ccu-shim up
       ip route add <IP-ADDRESS> dev ccu-shim protocol static
     fi
   fi

   NOTE: Make sure to set executable rights (chmod a+rx /etc/network/if-up.d/99-ccu-shim-network) to that file and to replace all <XXX> tags with the appropriate values from above.

Container Setup / Startup

For finally creating the docker container one can use different manual methods (docker pull, docker load and docker compose) or use the above mentioned self-container install-docker.sh solution. For the manual container setup/create one can use the following different solutions (make sure to replace all <XXX> placeholder accordingly):

Using docker pull:

1. Use docker pull to retrieve the latest version:

   docker pull ghcr.io/jens-maus/raspberrymatic:latest

2. Create the raspberrymatic docker container:

   docker create --name ccu \
                 --read-only \
                 --volume ccu_data:/usr/local:rw \
                 --volume /lib/modules:/lib/modules:ro \
                 --volume /run/udev/control:/run/udev/control \
                 --privileged --restart always --stop-timeout 30 \
                 --network ccu --ip <IP-ADDRESS> --hostname ccu \
                 ghcr.io/jens-maus/raspberrymatic:latest

3. Start the docker container:

   docker start -i ccu

   NOTE: Omit -i if you want to start the container non-interactive and put it in background.

Using docker load:

1. Download the latest docker/oci image version of RaspberryMatic:

   wget https://github.com/jens-maus/RaspberryMatic/releases/download/3.XX.XX.YYYYMMDD/RaspberryMatic-3.XX.XX.YYYYMMDD-oci_amd64.tgz

2. Use docker load to load the downloaded into the docker environment:

   docker load < RaspberryMatic-3.XX.XX.YYYYMMDD-oci_amd64.tgz

3. Run the raspberrymatic docker using cmd-line:

   docker create --name ccu \
                 --read-only \
                 --volume ccu_data:/usr/local:rw \
                 --volume /lib/modules:/lib/modules:ro \
                 --volume /run/udev/control:/run/udev/control \
                 --privileged --restart always --stop-timeout 30 \
                 --network ccu --ip <IP-ADDRESS> --hostname ccu \
                 ghcr.io/jens-maus/raspberrymatic:amd64-3.XX.XX.YYYYMMDD

4. Start the docker container:

   docker start -i ccu

   NOTE: Omit -i if you want to start the container non-interactive and put it in background.

Using docker compose

1. Add the following to your docker-compose.yml:

   version: "3.8"
   services:
     raspberrymatic:
       image: ghcr.io/jens-maus/raspberrymatic:latest
       container_name: ccu
       hostname: ccu
       read_only: true
       privileged: true
       restart: unless-stopped
       stop_grace_period: 30s
       volumes:
         - ccu_data:/usr/local:rw
         - /lib/modules:/lib/modules:ro
         - /run/udev/control:/run/udev/control
       networks:
         ccu:
           ipv4_address: <IP-ADDRESS>
   volumes:
     ccu_data:
   networks:
     ccu:
       name: ccu
       driver: macvlan
       driver_opts:
         parent: <INTERFACE>
       ipam:
         config:
           - subnet: <SUBNET>
             gateway: <GATEWAY>

2. Start your docker compose environment:

   docker compose up -d

Potential container startup errors

1. If you get ERROR: could not insert 'eq3_char_loop': Exec format error messages during startup the required kernel modules could not be correctly compiled on your docker host machine. To solve that issue you can try the following procedure:

   sudo apt remove --purge linux-headers-*
   sudo apt autoremove && sudo apt autoclean
   sudo apt install linux-headers-generic
   sudo apt install --reinstall pivccu-modules-dkms

2. In case radio communication takes sometimes up to several seconds or randomly fails ensure the container instance has enough cpu shares to process radio messages in time. This can be done by adding --cap=sys_nice and tweaking --cpu-reserved and/or --cpu-shares in your docker run command line or by adding the following in your docker-compose.yml:

      raspberrymatic:
        ...
        cap_add:
          - SYS_NICE         
        cpu_shares: 4096 # 4x more cpu cycles than standard containers during busy peaks
        deploy:
          update_config:
            order: stop-first
          resources:
            reservations:
              cpus: '0.6'
              memory: 256M

3. (partly obsolete): If during container startup you receive cpu.rt_runtime_us == 0 errors in the docker logs ccu output of your RaspberryMatic startup, your host system is using a linux kernel with CONFIG_RT_GROUP_SCHED real-time scheduling capabilities and did not assign enough real-time shares to the docker daemon, thus the RaspberryMatic container.

   • Modify/Create the /etc/docker/daemon.json file to contain/look like:

     {
       "cpu-rt-runtime": 950000
     }

     Afterwards, make sure you restart the docker daemon accordingly (e.g. sudo systemctl restart docker).
   • Depending on the method you are using to start the RaspberryMatic docker container make sure to either

     1. If using install-docker.sh:

        No action required as the latest install-docker.sh script should automatically set the necessary additional command-line options.

     2. If using docker pull or docker load:

        add the following command-line options to the corresponding docker run ... command:

        --cpu-rt-runtime 950000 --ulimit rtprio=99

     3. If using docker compose:

        add the following options to your docker-compose.yml file:

        cpu_rt_runtime: 950000
        ulimits:
          rtprio: 99

Container Usage

WebUI access to container

After startup, point your web browser to the IP of your hosting system and with port :80 and the usual CCU WebUI should be accessible:

http://<IP-ADDRESS>/

Shell access to container

After startup, you can either access the container via enabling SSH access to it in the CCU WebUI or you use the following docker exec command to be able to use the command-line:

docker exec -it ccu /bin/sh

Update to newer versions

The RaspberryMatic docker has to be updated from the host machine (the recommended "docker way"). Use either the same instructions like in using install-docker.sh and the other subsections (e.g. using docker pull) so that you retrieve the very latest version of the RaspberryMatic docker with the next execution.

Uninstallation

If you decide to get rid of the whole RaspberryMatic container installation or if you would like to cleanup before trying to install the container in a fresh environment you can do so by running the install-docker.sh script with an uninstall parameter:

wget -qO - https://raspberrymatic.de/install-docker.sh | sudo bash -s uninstall

This command will make sure to uninstall, remove and purge all relevant configuration files, docker container and docker network dependencies so that after a fresh reboot of your docker host system only the user data volume should still be present.

Appendix

Additional install-docker.sh settings

You can pass variables to the install-docker.sh script to change the defaults upon runtime:

• CCU_DATA_VOLUME
  • Name of the docker volume where CCU data will persist. It can be a local location as well such as a mounted NAS folder, cluster fs (glusterfs), etc.
  • Default: "ccu_data"
• CCU_OCI_REPO
  • Container repository to use
  • Default: "ghcr.io/jens-maus/raspberrymatic"
• CCU_OCI_TAG
  • Container image tag to use
  • Default: "latest"
• CCU_CONTAINER_NAME
  • Name of the container instance. Will also be used as --hostname parameter for docker run.
  • Default: "ccu"
• CCU_CONTAINER_IP
  • IP address for the container instance (e.g. 192.168.178.4). Have to be a non-DHCP assignable IP address from the same LAN subnet as the docker host.
  • Default: "" (will be interactively requested)
• CCU_CONTAINER_IP_AUX
  • Auxiliary ip address additionally assigned to the docker host system (e.g. 192.168.178.3). Have to be a non-DHCP assignable IP address from the same LAN subnet as the main IP address of the docker host.
  • Default: "" (will be interactively requested)
• CCU_NETWORK_NAME
  • Name for the docker macvlan network being created/maintained.
  • Default: "ccu"
• CCU_NETWORK_INTERFACE
  • Interface name (e.g. eth0, ensXX) of the main interface the docker host is running on and on which the same subnet is located.
  • Default: "" (will be automatically determined or interactively requested)
• CCU_NETWORK_SUBNET
  • Subnet (e.g. 192.168.178.0/24) of the main docker host and container ip address being used.
  • Default: "" (will be automatically determined or interactively requested)
• CCU_NETWORK_GATEWAY
  • Gateway ip address (e.g. 192.168.178.1) of the main docker host being used.
  • Default: "" (will be automatically determined or interactively requested)
• CCU_DOCKER_RUN_OPTIONS
  • Additional options for docker run
  • Default: ""
• CCU_DOCKER_PULL_OPTIONS
  • Additional options for docker pull run
  • Default: ""
• CCU_DOCKER_PULL_REFRESH
  • Do a docker pull to refresh image
  • Default: "true"
• CCU_DOCKER_STOP_TIMEOUT
  • time allowed to stop the container before Docker kills it
  • Default: 30 (seconds)