Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to Docker? #1757

Open
FezVrasta opened this issue Aug 16, 2019 · 36 comments

Comments

@FezVrasta
Copy link

@FezVrasta FezVrasta commented Aug 16, 2019

My RPi 4 (4GB) is finally arrived and I'm getting ready to migrate my old Home Assistant + DeCONZ setup to it.

I want to use Docker, since right now everything is installed on the host system and it's a hot mess.

I was wondering what's the procedure to migrate my DeCONZ config to my new RPI, considering I'm going to use Docker this time? I would really like not to have to re-pair all my devices again.

@manup

This comment has been minimized.

Copy link
Member

@manup manup commented Aug 17, 2019

Re-pairing isn't necessary. Basically you just need to transfer the zll.db file to the Docker environment. The easiest way is to create a backup file in the Phoscon App with your current setup and reload this configuration in the Phoscon App of your new Docker setup.

If something doesn't work right away please don't reset anything.

@FezVrasta

This comment has been minimized.

Copy link
Author

@FezVrasta FezVrasta commented Aug 18, 2019

Thanks.

I'm uploading the backup through the web UI but it's been stuck on the "Loading backup - This can take a moment..." for quite a few minutes already.

The process is running (2% CPU, 1% RAM)

What's the proper way to interrupt the process to try again?

@FezVrasta

This comment has been minimized.

Copy link
Author

@FezVrasta FezVrasta commented Aug 18, 2019

This is what I see in the console:

image

@FezVrasta

This comment has been minimized.

Copy link
Author

@FezVrasta FezVrasta commented Aug 18, 2019

I reloaded the page and tried again and I noticed this, I guess the backup restore process doesn't even start.

image

I tried with the old web UI and there at least it surfaces the error to the user:

image

@manup

This comment has been minimized.

Copy link
Member

@manup manup commented Aug 18, 2019

Hmm that's odd, I'll forward it to the developer of the backup page.
The 503 is however suspicious, is your setup working otherwise and the Zigbee network operational?

@FezVrasta

This comment has been minimized.

Copy link
Author

@FezVrasta FezVrasta commented Aug 18, 2019

I have no idea, I haven't tried to use it yet, I don't want to pair stuff manually ever again 😅

@manup

This comment has been minimized.

Copy link
Member

@manup manup commented Aug 18, 2019

I mean is your Zigbee network working, are devices connected?

@FezVrasta

This comment was marked as resolved.

Copy link
Author

@FezVrasta FezVrasta commented Aug 18, 2019

No, because I removed the RaspBee from my RPi2 and plugged it into the new RPi4 which is not yet configured.

But I just noticed the new RPi4 DeCONZ is not detecting the RaspBee, so I guess I first need to figure out why...

image

@FezVrasta

This comment was marked as resolved.

Copy link
Author

@FezVrasta FezVrasta commented Aug 18, 2019

This is my docker-compose configuration:

  deconz:
    image: marthoc/deconz
    container_name: deconz
    restart: unless-stopped
    network_mode: host
    environment:
      - DECONZ_WEB_PORT=8080
      - TZ=Europe/Budapest
      - DECONZ_UPNP=0
    devices:
      - /dev/ttyAMA0
    volumes:
      - ./services/deconz/data/config:/root/.local/share/dresden-elektronic/deCONZ
@manup

This comment has been minimized.

Copy link
Member

@manup manup commented Aug 18, 2019

Oh my bad just seeing the issue is about import (on the new machine I guess) not export.
I would say due how it looks the RaspBee isn't correctly installed yet and therefore "Not connected" is shown.

Have you configured the serial interface as described in the installation instructions?
https://phoscon.de/en/raspbee/install#docker

@FezVrasta

This comment was marked as resolved.

Copy link
Author

@FezVrasta FezVrasta commented Aug 18, 2019

I'm using DietPi, during the initial setup it asked me if I wanted to keep the serial port enabled and I said yes, so I guess that's all I need? (raspi-config is not a thing on DietPI from what I read)

@manup

This comment has been minimized.

Copy link
Member

@manup manup commented Aug 18, 2019

I don't know about DietPi but the serial port should not be used for any system logging, as Raspbian does it per default. Does DietPi even provide the /dev/ttyAMA0 interface?

@manup

This comment has been minimized.

Copy link
Member

@manup manup commented Aug 18, 2019

As a side note on Raspbian when the serial interface is configured properly /dev/ttyS0 is used instead of /dev/ttyAMA0.

@FezVrasta

This comment was marked as resolved.

Copy link
Author

@FezVrasta FezVrasta commented Aug 18, 2019

image

this is what I see, so I think the ttyAMA0 and ttyS0 are okay

@manup

This comment has been minimized.

Copy link
Member

@manup manup commented Aug 18, 2019

this is what I see, so I think the ttyAMA0 and ttyS0 are okay

Not if the interface is still shared with the login shell, but I don't know if it is the case here. On Raspbian it needs to be explicitly disabled.

What is the output of ls -l /dev/ttyAMA0?

@FezVrasta

This comment was marked as resolved.

Copy link
Author

@FezVrasta FezVrasta commented Aug 18, 2019

DietPi# ls -l /dev/ttyAMA0
crw--w---- 1 root tty 204, 64 Aug 18 11:43 /dev/ttyAMA0
@manup

This comment has been minimized.

Copy link
Member

@manup manup commented Aug 18, 2019

That's interesting :)
So unless you are running deCONZ as root it can't work.

You need to place your user into the tty group and give the /dev/ttyAMA0 group rw rights.

sudo gpasswd -a $USER tty
sudo chmod g+r /dev/ttyAMA0

Reboot afterwards. Still if the device is shared with the login shell it won't work.

@FezVrasta

This comment was marked as resolved.

Copy link
Author

@FezVrasta FezVrasta commented Aug 18, 2019

I'm running everything as root since I'm using Docker to run the various services.

I was reading this issue, is shell and console the same thing?
MichaIng/DietPi#343

@manup

This comment has been minimized.

Copy link
Member

@manup manup commented Aug 18, 2019

I'm running everything as root since I'm using Docker to run the various services.

How do you start the container? Docker doesn't require to be run as root.

I was reading this issue, is shell and console the same thing?
MichaIng/DietPi#343

I guess so, following looks interesting:
MichaIng/DietPi#343 (comment)

@FezVrasta

This comment was marked as resolved.

Copy link
Author

@FezVrasta FezVrasta commented Aug 18, 2019

I setup a systemd service that starts my docker-compose config. It doesn't require root but I'm just using it for now to make the setup easier. Once that's done I'll try to run it with an unprivileged user and see how it goes.

@FezVrasta

This comment was marked as resolved.

Copy link
Author

@FezVrasta FezVrasta commented Aug 18, 2019

Here's some log from the container...


deconz           | 12:04:44:464 dev /dev/ttyAMA0
deconz           | 12:04:45:055 device state timeout ignored in state 2
deconz           | 12:04:46:105 device state timeout ignored in state 2
deconz           | 12:04:47:156 device state timeout ignored in state 2
deconz           | 12:04:48:160 device state timeout ignored in state 2
deconz           | 12:04:49:160 device state timeout ignored in state 2
deconz           | 12:04:49:606 don't close database yet, keep open for 900 seconds
@manup

This comment has been minimized.

Copy link
Member

@manup manup commented Aug 18, 2019

It seems that the serial communication doesn't work. My guess is the above mentioned issue.
You can also check which programs are accessing the device:

sudo lsof /dev/ttyAMA0

Perhaps try to use /dev/ttyS0 as alternative.

@FezVrasta

This comment was marked as resolved.

Copy link
Author

@FezVrasta FezVrasta commented Aug 18, 2019

COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
hciattach 650 root    3u   CHR 204,64      0t0 1821 /dev/ttyAMA0

I'm starting to think it may be easier to use Raspbian 🙄

@manup

This comment has been minimized.

Copy link
Member

@manup manup commented Aug 18, 2019

https://linux.die.net/man/8/hciattach

hciattach is related to the bluetooth service.

I'm starting to think it may be easier to use Raspbian 🙄

I think it will work on DietPi but the setup requires extra steps which need to be figured out.

@FezVrasta

This comment was marked as resolved.

Copy link
Author

@FezVrasta FezVrasta commented Aug 18, 2019

Ok so let me try to disable the BT for now (but I'll need to enable it later anyways)...

@manup

This comment has been minimized.

Copy link
Member

@manup manup commented Aug 18, 2019

Bluetooth can be enabled, but it needs to be configured to use another serial interface.

@FezVrasta

This comment was marked as resolved.

Copy link
Author

@FezVrasta FezVrasta commented Aug 18, 2019

Ok, for now it's disabled, at least we have to worry about one less thing, now I don't get anything back when I run DietPi# lsof /dev/ttyAMA0.

from dietpi-config I see the UART is enabled, but I kept the console off:
image

Still, DeCONZ can't detect the RaspBee 😩

DietPi# ls -l /dev/ttyAMA0 
crw-rw---- 1 root dialout 204, 64 Aug 18 12:45 /dev/ttyAMA0
DietPi# ls -l /dev/ttyS0  
crw-rw---- 1 root dialout 4, 64 Aug 18 12:45 /dev/ttyS0
@manup

This comment has been minimized.

Copy link
Member

@manup manup commented Aug 18, 2019

Note you need to run lsof with sudo.

@FezVrasta

This comment was marked as resolved.

Copy link
Author

@FezVrasta FezVrasta commented Aug 18, 2019

I'm not sure what I'm doing, but setting the container with DECONZ_DEVICE=/dev/ttyS0 now it detects RaspBee, but still it doesn't tell me anything about the firmware...

image

deconz           | [marthoc/deconz] Starting deCONZ...
deconz           | [marthoc/deconz] Current deCONZ version: 2.05.66
deconz           | [marthoc/deconz] Web UI port: 8080
deconz           | [marthoc/deconz] Websockets port: 443
deconz           | [marthoc/deconz] VNC Disabled
deconz           | libpng warning: iCCP: known incorrect sRGB profile
deconz           | This plugin does not support propagateSizeHints()
deconz           | This plugin does not support propagateSizeHints()
deconz           | This plugin does not support propagateSizeHints()
deconz           | 13:02:43:161 HTTP Server listen on address 0.0.0.0, port: 8080, root: /usr/share/deCONZ/webapp/
deconz           | 13:02:43:176 CTRL. 3.16.213:02:43:263 dev /dev/ttyAMA0
deconz           | 13:02:43:263 COM: --dev: /dev/ttyS0 (RaspBee)
deconz           | 13:02:43:263 ZCLDB init file /root/.local/share/dresden-elektronik/deCONZ/zcldb.txt
deconz           | 13:02:43:422 parent process /bin/sh
deconz           | 13:02:43:422 gw run mode: docker
deconz           | 13:02:43:422 GW sd-card image version file does not exist: /root/.local/share/dresden-elektronik/deCONZ/gw-version
deconz           | 13:02:43:422 sd-card cid: 035344534c31364780ffffffff012200
deconz           | 13:02:43:424 DB sqlite version 3.16.2
deconz           | 13:02:43:426 DB PRAGMA page_count: 30
deconz           | 13:02:43:426 DB PRAGMA page_size: 4096
deconz           | 13:02:43:426 DB PRAGMA freelist_count: 0
deconz           | 13:02:43:426 DB file size 122880 bytes, free pages 0
deconz           | 13:02:43:426 DB PRAGMA user_version: 6
deconz           | 13:02:43:426 DB cleanup
deconz           | 13:02:43:427 DB create temporary views
deconz           | 13:02:43:434 don't close database yet, keep open for 900 seconds
deconz           | 13:02:43:435 started websocket server at port 443
deconz           | 13:02:43:438 discovery updated announce interval to 10 minutes
deconz           | 13:02:43:441 found node plugin: libde_rest_plugin.so - REST API Plugin
deconz           | 13:02:43:444 found node plugin: libde_signal_plugin.so - Signal Monitor Plugin
deconz           | 13:02:43:459 found node plugin: libstd_otau_plugin.so - STD OTAU Plugin
deconz           | 13:02:43:504 dev /dev/ttyAMA0
deconz           | 13:02:43:504 COM: --dev: /dev/ttyS0 (RaspBee)
deconz           | 13:02:43:620 Device firmware version 0x26280500
deconz           | 13:02:43:631 unlocked max nodes: 200
deconz           | 13:02:43:813 Device protocol version: 0x0108
deconz           | 13:02:43:828 new node - ext: 0x00212effff017929, nwk: 0x0000
deconz           | 13:02:43:915 don't close database yet, keep open for 900 seconds
deconz           | 13:02:43:923 don't close database yet, keep open for 900 seconds
deconz           | 13:02:43:925 don't close database yet, keep open for 900 seconds
deconz           | 13:02:43:932 don't close database yet, keep open for 900 seconds
deconz           | 13:02:43:934 don't close database yet, keep open for 900 seconds
deconz           | 13:02:43:941 don't close database yet, keep open for 900 seconds
deconz           | 13:02:43:942 don't close database yet, keep open for 900 seconds
deconz           | 13:02:43:943 don't close database yet, keep open for 900 seconds
deconz           | 13:02:43:952 don't close database yet, keep open for 900 seconds
deconz           | 13:02:43:953 don't close database yet, keep open for 900 seconds
deconz           | 13:02:43:954 don't close database yet, keep open for 900 seconds
deconz           | 13:02:43:962 don't close database yet, keep open for 900 seconds
deconz           | 13:02:43:977 don't close database yet, keep open for 900 seconds
deconz           | 13:02:43:993 don't close database yet, keep open for 900 seconds
deconz           | 13:02:44:007 don't close database yet, keep open for 900 seconds
deconz           | 13:02:44:008 don't close database yet, keep open for 900 seconds
deconz           | 13:02:44:016 don't close database yet, keep open for 900 seconds
deconz           | 13:02:44:017 don't close database yet, keep open for 900 seconds
deconz           | 13:02:44:025 don't close database yet, keep open for 900 seconds
deconz           | 13:02:44:026 don't close database yet, keep open for 900 seconds
deconz           | 13:02:44:035 don't close database yet, keep open for 900 seconds
deconz           | 13:02:44:037 don't close database yet, keep open for 900 seconds
deconz           | 13:02:44:037 don't close database yet, keep open for 900 seconds
deconz           | 13:02:44:045 don't close database yet, keep open for 900 seconds
deconz           | 13:02:44:046 don't close database yet, keep open for 900 seconds
deconz           | 13:02:44:054 don't close database yet, keep open for 900 seconds
deconz           | 13:02:44:055 don't close database yet, keep open for 900 seconds
deconz           | 13:02:44:063 don't close database yet, keep open for 900 seconds
deconz           | 13:02:44:208 Current channel 25
deconz           | 13:02:44:248 CTRL ANT_CTRL 0x03
deconz           | 13:02:44:327 Device protocol version: 0x0108
deconz           | 13:02:44:422 Current channel 25
deconz           | 13:02:44:450 CTRL ANT_CTRL 0x03
deconz           | 13:02:44:551 APS-DATA.indication from unknown node 0x0017880104E419E0
deconz           | 13:02:44:551 ZCL attribute report 0x0017880104E419E0 for cluster 0x0001, ep 0x02
deconz           | 13:02:44:569 ZCL attribute report 0x90FD9FFFFED16946 for cluster 0x0300, ep 0x01
deconz           | 13:02:44:585 ZCL attribute report 0x000B57FFFEEFB368 for cluster 0x0006, ep 0x01
deconz           | 13:02:47:146 ZCL attribute report 0x90FD9FFFFED16946 for cluster 0x0008, ep 0x01
deconz           | 13:02:47:450 New websocket 10.0.0.8:52448 (state: 3) 
deconz           | 13:02:48:553 dev /dev/ttyAMA0
deconz           | 13:02:48:554 GW update firmware found: /usr/share/deCONZ/firmware/deCONZ_Rpi_0x26330500.bin.GCF

Would you consider this a progress? 😅

Note you need to run lsof with sudo.

I'm running as root user

@FezVrasta

This comment was marked as resolved.

Copy link
Author

@FezVrasta FezVrasta commented Aug 18, 2019

I'm not sure if it matters, but I can't even rename the gateway, it will always return a 403 error. Is that because it needs a working RaspBee connected or it's unrelated?

I also upgraded the firmware just to be sure

@FezVrasta

This comment has been minimized.

Copy link
Author

@FezVrasta FezVrasta commented Aug 18, 2019

Oh dammit, I figured out I had a typo in the volume...

#1757 (comment)

It should have been:

- ./services/deconz/data/config:/root/.local/share/dresden-elektronik/deCONZ

Now it shows the firmware correctly, but still I can't restore the backup with that 503 error I reported above...

image

@FezVrasta

This comment has been minimized.

Copy link
Author

@FezVrasta FezVrasta commented Aug 18, 2019

Don't ask me how, but the configuration now appeared in the web app... 🤷‍♂️

@FezVrasta FezVrasta closed this Aug 18, 2019
@FezVrasta

This comment has been minimized.

Copy link
Author

@FezVrasta FezVrasta commented Aug 18, 2019

I restarted Docker and I lost all the configuration 🙄 I don't yet know if it's my fault, but the backup import is now again not working properly.

edit: I don't think it's my fault, because the /opt/docker-compose/services/deconz/data/config folder has some files in it, so the configuration should persist across restarts.

@FezVrasta FezVrasta reopened this Aug 18, 2019
@manup

This comment has been minimized.

Copy link
Member

@manup manup commented Aug 19, 2019

I restarted Docker and I lost all the configuration

How do you see all configuration is lost, are there no devices shown anymore?

Are you sure that the volume configuration is correct?
Your docker-compose configuration says:

 deconz:
    image: marthoc/deconz
    container_name: deconz
    restart: unless-stopped
    network_mode: host
    environment:
      - DECONZ_WEB_PORT=8080
      - TZ=Europe/Budapest
      - DECONZ_UPNP=0
    devices:
      - /dev/ttyAMA0
    volumes:
      - ./services/deconz/data/config:/root/.local/share/dresden-elektronic/deCONZ

Shouldn't that be /opt/docker-compose/services/deconz/data/config:/root/.local/share/dresden-elektronic/deCONZ?

@FezVrasta

This comment has been minimized.

Copy link
Author

@FezVrasta FezVrasta commented Aug 19, 2019

No, the volumes directive is correct, in fact I see files being written in the folder (I use the same format for all my containers).

I haven't restarted again, once I'll do so I'll see if it gets reset again.

Anyways, I'd keep this issue focused on the backup import feature which doesn't seem to work reliably. Is that okay to you?

@FezVrasta

This comment has been minimized.

Copy link
Author

@FezVrasta FezVrasta commented Aug 19, 2019

I restarted the container and the config didn't get lost. So the only issue is the import backup feature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.