NPM crashes on Raspberry Pi 3 B+ in the latest Docker image

[konstantint]

What happened

pi@raspberrypi:~ $ docker run -it koenkk/zigbee2mqtt:latest npm
Using '/app/data' as data directory
Creating configuration file...
#
# Fatal error in , line 0
# unreachable code
#
#
#
#FailureMessage Object: 0x7eace25c

If I do this manually from /bin/ash I also see the final message Illegal instruction (core dumped)
The problem does not happen on a different machine.

What did you expect to happen

Something like this (note the previous version tag):

pi@raspberrypi:~ $ docker run -it koenkk/zigbee2mqtt:1.18.3 npm                                                                    
Using '/app/data' as data directory       
Creating configuration file...                                   
                                
Usage: npm <command>                                             
                                                                 
where <command> is one of:
 ...

Debug info

Raspberry version: 3B+, quite recent Raspberry Pi OS

pi@raspberrypi:~ $ uname -a
Linux raspberrypi 5.10.36-v7+ #1418 SMP Thu May 13 18:15:03 BST 2021 armv7l GNU/Linux

[Ivano62]

The same problem on Raspberry PI 3 model B. I run in docker-compose with that conf:
version: '3'
services:
zigbee2mqtt:
container_name: zigbee2mqtt:latest
image: koenkk/zigbee2mqtt
volumes:
- "/home/pi/zigbee2mqtt:/app/data"
devices:
- "/dev/ttyAMA0"
environment:
- "TZ=Europe/Rome"
network_mode: host
restart: unless-stopped

and get this error:

**Fatal error in , line 0
unreachable code

FailureMessage Object: 0x7edbf25c**

With 1.18.3 version works with any problem.

Thanks

[LordMyschkin]

same problem on rpi4 here

[Paraphraser]

Same problem on RPI4. I pinned to 1.18.3 and it started working again.

[ssiergl]

Hmm...working here without issues on RPI4, debian buster 64bit latest docker version

Sounds like an platform compile issue or that the wrong platform was pulled from updated alpine image

Related to #7591

[Ivano62]

I honestly don't know what the problem is, the sure thing is that by reinstalling version 1.18.3 everything works correctly. So I think it depends on the latest version

[ruttromp]

same issue on rpi4 since the update to 1.19.0 using the docker koenkk/zigbee2mqtt:latest image

[cyberdie]

Same problem here with Pi4 and docker run:
docker run -d --network=host --name zigbee-gateway -e "TZ=Europe/Madrid" --restart unless-stopped -v /home/pi/dockerdata/zigbee2mqtt/:/app/data --device=/dev/ttyACM0 koenkk/zigbee2mqtt:latest

docker keeps restarting with errors:
`

Fatal error in , line 0

unreachable code

#FailureMessage Object: 0xbedd725c
Using '/app/data' as data directory

Fatal error in , line 0

unreachable code

#FailureMessage Object: 0xbea2a25c
Using '/app/data' as data directory
'

[sineverba]

@Koenkk I'm not sure of trouble or resolution. I simply not update until I will see a green flag on these issues... but....

With another project I had troubles with alpine 3.13 on RPI (I'm referring to this commit: 085983d)

I needed (on my own project) to revert to 3.12.

Maybe someone can try to create a docker with 3.12 instead of 3.13?

[ssiergl]

What docker version are you guys using?

Changelog of alpine 3.13 requires minimum docker Version 19.03.9
Update of musl changes time64 requirements. I assume, did not test myself, that there is the problem.
So basically you guys should check your OS and docker version against the requirements mentioned in official alpine changelog: https://wiki.alpinelinux.org/wiki/Release_Notes_for_Alpine_3.13.0

@Koenkk IMO the docker minimum requirement should be mentioned in release changelog

[sineverba]

What docker version are you guys using?

Changelog of alpine 3.13 requires minimum docker Version 19.03.9
Update of musl changes time64 requirements. I assume, did not test myself, that there ist the problem.
So basically you guys should check your OS and docker version against the requirements mentioned in official alpine changelog: https://wiki.alpinelinux.org/wiki/Release_Notes_for_Alpine_3.13.0

@Koenkk IMO the docker minimum requirement should be mentioned in release changelog

If this is the issue, IMHO need to revert to 3.12. Moving to 3.13 cuts out all Raspbian users, e.g.

[cyberdie]

In my case (PI4):
Linux HomeServer 5.10.17-v7l+ #1403 SMP Mon Feb 22 11:33:35 GMT 2021 armv7l GNU/Linux
Docker version 20.10.6, build 370c289

[m0wlheld]

Problem confirmed, rollback to image 1.18.3 works.

Env:
Raspberry Pi 3B+
Linux denise 5.10.11-v7+ #1399 SMP Thu Jan 28 12:06:05 GMT 2021 armv7l GNU/Linux
Docker version 20.10.6, build 370c289

[ssiergl]

Could you check if your libseccomp has the required version?

Like stated here:

To check if your host libseccomp is time64-compatible, invoke scmp_sys_resolver -a x86 clock_gettime64 for x86 containers, or scmp_sys_resolver -a arm clock_gettime64 for armhf or armv7 containers. If 403 is returned, time64 is supported. If -1 is returned, time64 is not supported. Note that Docker must still be at least version 19.03.9, regardless of the result of this command. In order to run under old Docker or libseccomp versions, the moby default seccomp profile should be downloaded and on line 2, defaultAction changed to SCMP_ACT_TRACE, then --seccomp-profile=default.json can be passed to dockerd, or --security-opt=seccomp=default.json passed to docker create or docker run. This will cause the system calls to return ENOSYS instead of EPERM, allowing the container to fall back to 32-bit time system calls. In this case, the container will not be compatible with dates past 2038.

[Ivano62]

I honestly don't know what the problem is, the sure thing is that by reinstalling version 1.18.3 everything works correctly. So I think it depends on the latest version

I use docker-compose, in any case
Env:
Raspberry Pi 3 model B
Linux raspberry 5.10.17-v7+ #1421 SMP Thu May 27 13:59:01 BST 2021 armv7l GNU/Linux
Docker version 20.10.6, build 370c289
RaspBee II

[ssiergl]

What docker version are you guys using?
Changelog of alpine 3.13 requires minimum docker Version 19.03.9
Update of musl changes time64 requirements. I assume, did not test myself, that there ist the problem.
So basically you guys should check your OS and docker version against the requirements mentioned in official alpine changelog: https://wiki.alpinelinux.org/wiki/Release_Notes_for_Alpine_3.13.0
@Koenkk IMO the docker minimum requirement should be mentioned in release changelog

If this is the issue, IMHO need to revert to 3.12. Moving to 3.13 cuts out all Raspbian users, e.g.

Well basically this will just move the Problem to another time point. Alpine will not revert this changes for sure.

[sineverba]

I'm a Docker "fanatic", Alpine "addicted" but I think sometime we can leave Alpine alone and change to another base dist.
I know the container size issues, but sometimes I prefer more space used to get more stability (and less trouble :) )

[ssiergl]

I'm a Docker "fanatic", Alpine "addicted" but I think sometime we can leave Alpine alone and change to another base dist.
I know the container size issues, but sometimes I prefer more space used to get more stability (and less trouble :) )

Well true that :D
I personally always use debian buster minimal for building projects (more or less same reason...less breaking changes). But this is a base decision which needs to be discussed with @Koenkk

[Paraphraser]

In my case (Pi4):

$ docker --version
Docker version 20.10.6, build 370c289
$ uname -a
Linux new-dev 5.10.17-v7l+ #1421 SMP Thu May 27 14:00:13 BST 2021 armv7l GNU/Linux

Both are "latest and greatest".

I agree with @sineverba - in my situation, image size is less important than stability and reliability. On the other hand, there's probably no such thing as a perfect base distro.

[m0wlheld]

less important than stability and reliability

It does not work at all ...

[ssiergl]

In my case (Pi4):

$ docker --version
Docker version 20.10.6, build 370c289
$ uname -a
Linux new-dev 5.10.17-v7l+ #1421 SMP Thu May 27 14:00:13 BST 2021 armv7l GNU/Linux

Both are "latest and greatest".

I agree with @sineverba - in my situation, image size is less important than stability and reliability. On the other hand, there's probably no such thing as a perfect base distro.

No the perfect base distro does not exist.....but we basically try to use distros which are not always on bleeding edge but stable. As long they care about security I'm fine with this.

Just for all.....this is my configuration....docker and libseccomp version fits on debian buster but well I'm on aarch64

[konstantint]

pi@raspberrypi:~$ docker --version
Docker version 20.10.6, build 370c289
pi@raspberrypi:~$ sudo apt install seccomp
...
pi@raspberrypi:~$ scmp_sys_resolver -a arm clock_gettime64
-1

So yes, the Alpine 3.13 requirements related to libseccomp are most probably the cause here. Let us try the workarounds suggested in the Alpine docs:

pi@raspberrypi:~$ docker run -it --security-opt=seccomp=unconfined koenkk/zigbee2mqtt:latest npm
Using '/app/data' as data directory
Creating configuration file...

Usage: npm <command>
...

I.e. no crash. Similarly:

pi@raspberrypi:~$ wget https://github.com/moby/moby/raw/master/profiles/seccomp/default.json
pi@raspberrypi:~$ sed -i 's/SCMP_ACT_ERRNO/SCMP_ACT_TRACE/' default.json
pi@raspberrypi:~$ docker run -it --security-opt=seccomp=default.json koenkk/zigbee2mqtt:latest npm

also works fine. Finally:

$ sudo cp default.json /etc/docker/seccomp-profile.json
$ cat | sudo tee /etc/docker/daemon.json <<EOF
{
   "seccomp-profile": "/etc/docker/seccomp-profile.json"
}
EOF
$ sudo systemctl restart docker

At this point my Raspberry lost connectivity and I had to hard-reset it by plugging the cord. It came back up fine, so let's proceed:

$ docker run -it koenkk/zigbee2mqtt:latest npm

Works fine

Summary

• If you updated to the latest zigbee2mqtt image for no good reason, just roll back to 1.18.3.
• If you really need the latest image NOW, consider the workaround with /etc/docker/seccomp-profile.json above (NB: don't blindly copy my shell commands - the cat | tee step will overwrite whatever you might have had in /etc/docker/daemon.json. I did not have that file at all but your case may be different). Beware of your machine losing connectivity or possibly crashing on systemctl restart docker (not sure why it happened). Alternatively, you could also specify the security policy in the docker run without restarting the daemon. I presume this flag should configurable via docker-compose as well although I never tried.
• On the zigbee2mqtt side, I'd suggest not updating to the latest Alpine for now unless it is really needed somewhy.
• In the longer term, I presume the issue with libseccomp will be resolved in the majority of host systems which should make it possible to upgrade to the newer Alpine as well.

[m0wlheld]

Just for all.....this is my configuration....docker and libseccomp version fits on debian buster but well I'm on aarch64

Could this be a arm64/armv7 issue? Most RPI3 will probably run a 32 bit kernel.

[LouisMT]

This is indeed an issue with Alpine 3.13 on 32-bit Debian hosts. I've run into this issue using another Docker image a while ago. Basically, Alpine 3.13 uses 64 bit timestamps now. But the libseccomp2 package on Debian is very old and doesn't recognize the calls Docker makes.

So the easiest fix is: upgrade the libseccomp2 package. This issue has been reported months ago in the Debian repositories, but it hasn't gained any traction. So I wouldn't wait on Debian to fix the problem for you. And I think many more Docker images will eventually upgrade to Alpine 3.13 at some point, so reverting to 3.12 isn't a long term solution.

So, in my opinion the best fix is to manually upgrade the package. I've written down how to do so here: xirixiz/dsmr-reader-docker#203 (comment).

The fixed package is also in the Debian Backports repository. I recommend using the Backports method, as it's an official Debian repository that contains updated versions of stable Debian packages.

[ssiergl]

I doubt as well that libseccomp will be updated in the near future on any of the distros mentioned in alpine changelog. The current version they are using are from 2019 and older although a lot newer maintained versions in 2.4 and 2.5 releases exist

Whatever for now I am as well ok to revert to 3.12....still it is officially supported till may next year. I do not see an urgent reason to update the base image. @Koenkk or any special stuff planed where you need the new packages?

@LouisMT do not forget that a lot not experienced people are using zigbee2mqtt with a raspberry pi...not sure if it is advisable to force them to install or upgrade a package beforehand IMO

Nevertheless.....alpine 3.13 was released end last year....so no idea about the long term solution...either changing base image to debian buster...which should be inline with people distros which mostly are debian derivates
Or force people to use aarch64 based OS like I'm using 👯

[sineverba]

[...]

@LouisMT do not forget that a lot not experienced people are using zigbee2mqtt with a raspberry pi...not sure if it is advisable to force them to install or upgrade a package beforehand IMO

[...]

This.

[LouisMT]

Don't get me wrong: I don't mean everyone should upgrade the package. But, a few images I'm using have already upgraded to 3.13, and not all maintainers are willing to downgrade back to 3.12. Or it takes a long time.

So, if you want to save yourself from breaking containers in the future, this is the easiest and best fix you can perform yourself in my opinion. Otherwise, you depend on the maintainer of all the images you're using, or on Alpine/Debian to fix the issue. And something tells me they won't be fixing this (soon).

But you're right, in the case of this image, I think it's best to either downgrade the Alpine base or to change base entirely, to fix the issue for everyone.

[Koenkk]

Thanks for confirming, created a new release (1.19.1)

[Koenkk]

Alpine 3.12 will become unsupported starting from May so I'm forced to upgrade to a more recent Alpine version.

Could someone check if the latest Raspberry Pi OS already includes libseccomp >= 2.4.2? Command: dpkg --list | grep libseccomp

[m0wlheld]

ii  libseccomp2:armhf               2.3.3-4                        armhf        high level interface to Linux seccomp filter

Raspbian GNU/Linux 10 (buster)

[Paraphraser]

If you are running Buster, you need to add libseccomp2 from backports. See IOTstack patch 2 for an example of how to do that.

If you are running Bullseye the problem just goes away.

I'm running several Pis with a mixture of Buster and Bullseye, with docker containers that are known to break on Buster if libseccomp2 is obsolete so I can state all this from certain knowledge.

See also IOTstack issue 401.

Bullseye example:

Buster example with libseccomp2 updated from backports:

[Koenkk]

@Paraphraser that sounds reasonable

@m0wlheld out of interest, why did you not update to bullseye eye? Seems it is out since December 2021.

[m0wlheld]

No specific reason. For my smart home controller, I prefer stability over fancyness. Upgrading now …

[Koenkk]

I've upgraded the container to Alpine 3.15 and Node.js 16 (available in the dev branch now). Wrote some docs on how to install libseccomp2: link (Thanks @Paraphraser!)

[jvitali]

Works like a charm in RPi 4:
Linux raspi 5.15.30-v8+ #1536 SMP PREEMPT Mon Mar 28 13:53:14 BST 2022 aarch64 GNU/Linux

Thank you :)

[SoftwareSchlosser]

Just installed everything + libseccomp2 backport on my Raspberry Pi 3 Model B Rev 1.2 with Raspbian 10 / buster but it gave the same error (...unreachable code...).
Also tried patch 1 and 3 mentioned here without success.
What could I have done wrong?

node version: v10.24.0
npm version: 5.8.0

dpkg info:
ii libseccomp2:armhf 2.5.1-1~bpo10+1
uname -a
Linux pi 5.10.103-v7+ #1529 SMP Tue Mar 8 12:21:37 GMT 2022 armv7l GNU/Linux

[git1106]

Just installed everything + libseccomp2 backport on my Raspberry Pi 3 Model B Rev 1.2 with Raspbian 10 / buster but it gave the same error (...unreachable code...).

Same problem here:

g@raspberrypi:~/try/zigbee2mqtt $ sudo docker run --device=/dev/serial/by-id/usb-dresden_elektronik_ingenieurtechnik_GmbH_ConBee_II_DE2440423-if00 -v $(pwd)/data:/app/data -v /run/udev:/run/udev:ro koenkk/zigbee2mqtt
Using '/app/data' as data directory


#
# Fatal error in , line 0
# unreachable code
#
#
#
#FailureMessage Object: 0x7ed1464c
g@raspberrypi:~/try/zigbee2mqtt $ grep Model /proc/cpuinfo 
Model		: Raspberry Pi 3 Model B Rev 1.2
g@raspberrypi:~/try/zigbee2mqtt $ lsb_release -a
No LSB modules are available.
Distributor ID:	Raspbian
Description:	Raspbian GNU/Linux 10 (buster)
Release:	10
Codename:	buster
g@raspberrypi:~/try/zigbee2mqtt $ apt-cache policy docker.io
docker.io:
  Installed: 18.09.1+dfsg1-7.1+rpi1+rpt1
  Candidate: 18.09.1+dfsg1-7.1+rpi1+rpt1
  Version table:
 *** 18.09.1+dfsg1-7.1+rpi1+rpt1 500
        500 http://archive.raspberrypi.org/debian buster/main armhf Packages
        100 /var/lib/dpkg/status
     18.09.1+dfsg1-7.1+rpi1+deb10u3 500
        500 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
g@raspberrypi:~/try/zigbee2mqtt $ apt-cache policy libseccomp2
libseccomp2:
  Installed: 2.5.1-1~bpo10+1
  Candidate: 2.5.1-1~bpo10+1
  Version table:
 *** 2.5.1-1~bpo10+1 100
        100 http://httpredir.debian.org/debian buster-backports/main armhf Packages
        100 /var/lib/dpkg/status
     2.3.3-4 500
        500 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
g@raspberrypi:~/try/zigbee2mqtt $ sudo docker image inspect db89c5f5d19f | head -9
[
    {
        "Id": "sha256:db89c5f5d19f79680a18960da8f7feecd2ee8cfb3bed6d005f28fb5dc42fb523",
        "RepoTags": [
            "koenkk/zigbee2mqtt:latest"
        ],
        "RepoDigests": [
            "koenkk/zigbee2mqtt@sha256:599b31ca43edf702763bd834b9e8b14b62ef7eed8512947d8ca27271cbbe367e"

Just FYI. I don't think this is important to me, I'll try to move to bullseye.

[git1106]

Just installed everything + libseccomp2 backport on my Raspberry Pi 3 Model B Rev 1.2 with Raspbian 10 / buster but it gave the same error (...unreachable code...).

Same problem here:

[...]

Actually --security-opt=seccomp=unconfinedmakes it work. Maybe not a good idea on a production system.

[SoftwareSchlosser]

Moved to bullseye yesterday, thank you anyway :)

[guiramos]

konstantint

This guy is a freaking genius!!!