Mailu on arm exits with code 132 #2541

Minion3665 · 2022-11-18T21:55:42Z

Environment & Version

Environment

docker-compose
kubernetes
docker swarm

Version

Version: master

Description

Mailu fails to run on a raspberry pi 4

Replication Steps

Follow https://mailu.io/master/arm_images.html to create a docker compose configuration for mailu
Put that configuration onto a raspberry pi 4
Run docker compose up

Observed behaviour

Every mailu container (i.e. everything except redis) exits with code 132

Expected behaviour

I expect the mailu containers to start and run mailu

Extra notes

This seems to have been introduced between 8a90f83 and 68bb8da as reverting to the former and building manually works

Logs

{16:49}/opt/mailu ➭ docker compose up                                                              (PI) 
[+] Running 11/11
 ⠿ Network mailu_default        Created                                                             0.1s
 ⠿ Container mailu-resolver-1   Created                                                             0.3s
 ⠿ Container mailu-webdav-1     Created                                                             3.1s
 ⠿ Container mailu-front-1      Created                                                             3.1s
 ⠿ Container mailu-fetchmail-1  Created                                                             3.1s
 ⠿ Container mailu-redis-1      Created                                                             3.1s
 ⠿ Container mailu-admin-1      Created                                                             0.3s
 ⠿ Container mailu-imap-1       Created                                                             0.3s
 ⠿ Container mailu-smtp-1       Created                                                             0.3s
 ⠿ Container mailu-antispam-1   Created                                                             0.3s
 ⠿ Container mailu-webmail-1    Created                                                             0.1s
Attaching to mailu-admin-1, mailu-antispam-1, mailu-fetchmail-1, mailu-front-1, mailu-imap-1, mailu-redis-1, mailu-resolver-1, mailu-smtp-1, mailu-webdav-1, mailu-webmail-1
mailu-redis-1      | 1:C 18 Nov 2022 21:49:33.978 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
mailu-redis-1      | 1:C 18 Nov 2022 21:49:33.978 # Redis version=7.0.5, bits=64, commit=00000000, modified=0, pid=1, just started
mailu-redis-1      | 1:C 18 Nov 2022 21:49:33.979 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
mailu-redis-1      | 1:M 18 Nov 2022 21:49:33.985 * monotonic clock: POSIX clock_gettime
mailu-redis-1      | 1:M 18 Nov 2022 21:49:33.988 * Running mode=standalone, port=6379.
mailu-redis-1      | 1:M 18 Nov 2022 21:49:33.989 # Server initialized
mailu-redis-1      | 1:M 18 Nov 2022 21:49:33.990 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
mailu-redis-1      | 1:M 18 Nov 2022 21:49:33.996 * Loading RDB produced by version 7.0.5
mailu-redis-1      | 1:M 18 Nov 2022 21:49:33.996 * RDB age 94 seconds
mailu-redis-1      | 1:M 18 Nov 2022 21:49:33.996 * RDB memory usage when created 1.11 Mb
mailu-redis-1      | 1:M 18 Nov 2022 21:49:34.001 * Done loading RDB, keys loaded: 2480, keys expired: 0.
mailu-redis-1      | 1:M 18 Nov 2022 21:49:34.001 * DB loaded from disk: 0.005 seconds
mailu-redis-1      | 1:M 18 Nov 2022 21:49:34.001 * Ready to accept connections
mailu-resolver-1 exited with code 0
mailu-fetchmail-1 exited with code 132
mailu-webdav-1 exited with code 132
mailu-resolver-1 exited with code 132
mailu-front-1 exited with code 132
mailu-admin-1 exited with code 132
mailu-smtp-1 exited with code 0
mailu-fetchmail-1 exited with code 0
mailu-imap-1 exited with code 132
mailu-antispam-1 exited with code 132
mailu-webdav-1 exited with code 132
mailu-resolver-1 exited with code 132
mailu-admin-1 exited with code 132
mailu-fetchmail-1 exited with code 132
mailu-smtp-1 exited with code 132
mailu-webmail-1 exited with code 132
mailu-front-1 exited with code 132
mailu-imap-1 exited with code 132
mailu-antispam-1 exited with code 132
mailu-resolver-1 exited with code 132
mailu-admin-1 exited with code 132
mailu-webdav-1 exited with code 132
mailu-webmail-1 exited with code 0
mailu-smtp-1 exited with code 132
mailu-fetchmail-1 exited with code 132
mailu-imap-1 exited with code 132
mailu-antispam-1 exited with code 132
mailu-front-1 exited with code 132
mailu-resolver-1 exited with code 132
mailu-admin-1 exited with code 132
mailu-webmail-1 exited with code 132
mailu-webdav-1 exited with code 132
mailu-smtp-1 exited with code 132
mailu-antispam-1 exited with code 132
mailu-fetchmail-1 exited with code 132
mailu-imap-1 exited with code 132
mailu-admin-1 exited with code 132
^CGracefully stopping... (press Ctrl+C again to force)
[+] Running 4/6
 ⠿ Container mailu-webmail-1    Stopped                                                             0.3s
 ⠿ Container mailu-fetchmail-1  Stopped                                                             0.1s
 ⠿ Container mailu-webdav-1     Stopped                                                             0.2s
 ⠹ Container mailu-admin-1      Stopping                                                            0.3s
 ⠹ Container mailu-smtp-1       Stopping                                                            0.3s
 ⠿ Container mailu-antispam-1   Stopped                                                             0.3s
 ⠋ Container mailu-imap-1       Stopping                                                            0.0s

The text was updated successfully, but these errors were encountered:

leolivier · 2022-11-19T10:37:03Z

Hi, same issue here also with latest arm64 version.
I think the issue is in the resolver which exits for an unknown reason with code 0 and as other containers depend on it, they can't start (on my side, everything exits with code 0 and not 132).
And by the way, still no way to go back to the previous version, so my server is not working and I can't rollback to the working version :/
EDIT: first exit with code 0 but subsequent with code 132...

pi@raspberrypi /docker/mailu $ docker compose up
[+] Running 10/10
 ⠿ Container mailu.resolver   Created                                                                                                                                                                    0.9s
 ⠿ Container mailu.fetchmail  Created                                                                                                                                                                    1.1s
 ⠿ Container mailu.front      Created                                                                                                                                                                    1.1s
 ⠿ Container mailu.antivirus  Created                                                                                                                                                                    1.1s
 ⠿ Container mailu.redis      Created                                                                                                                                                                    1.0s
 ⠿ Container mailu.admin      Created                                                                                                                                                                    1.1s
 ⠿ Container mailu.smtp       Created                                                                                                                                                                    1.0s
 ⠿ Container mailu.imap       Created                                                                                                                                                                    1.1s
 ⠿ Container mailu.antispam   Created                                                                                                                                                                    1.0s
 ⠿ Container mailu.webmail    Created                                                                                                                                                                    0.6s
Attaching to mailu.admin, mailu.antispam, mailu.antivirus, mailu.fetchmail, mailu.front, mailu.imap, mailu.redis, mailu.resolver, mailu.smtp, mailu.webmail
mailu.redis      | 1:C 19 Nov 2022 10:34:30.835 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
mailu.redis      | 1:C 19 Nov 2022 10:34:30.835 # Redis version=7.0.5, bits=64, commit=00000000, modified=0, pid=1, just started
mailu.redis      | 1:C 19 Nov 2022 10:34:30.835 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
mailu.redis      | 1:M 19 Nov 2022 10:34:30.837 * monotonic clock: POSIX clock_gettime
mailu.redis      | 1:M 19 Nov 2022 10:34:30.840 * Running mode=standalone, port=6379.
mailu.redis      | 1:M 19 Nov 2022 10:34:30.841 # Server initialized
mailu.redis      | 1:M 19 Nov 2022 10:34:30.841 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
mailu.redis      | 1:M 19 Nov 2022 10:34:30.846 * Loading RDB produced by version 7.0.5
mailu.redis      | 1:M 19 Nov 2022 10:34:30.846 * RDB age 151 seconds
mailu.redis      | 1:M 19 Nov 2022 10:34:30.846 * RDB memory usage when created 4.55 Mb
mailu.redis      | 1:M 19 Nov 2022 10:34:30.910 * Done loading RDB, keys loaded: 26291, keys expired: 0.
mailu.redis      | 1:M 19 Nov 2022 10:34:30.910 * DB loaded from disk: 0.064 seconds
mailu.redis      | 1:M 19 Nov 2022 10:34:30.910 * Ready to accept connections
mailu.resolver exited with code 0
mailu.fetchmail exited with code 0
mailu.antivirus exited with code 0
mailu.front exited with code 0
mailu.resolver exited with code 132
mailu.admin exited with code 0
mailu.imap exited with code 132
mailu.antispam exited with code 132
mailu.smtp exited with code 132
mailu.antivirus exited with code 132
mailu.fetchmail exited with code 132
mailu.front exited with code 132
mailu.resolver exited with code 132
mailu.webmail exited with code 132
mailu.antispam exited with code 132
mailu.smtp exited with code 132
mailu.imap exited with code 132
mailu.admin exited with code 132
mailu.fetchmail exited with code 132

leolivier · 2022-11-19T10:57:38Z

Found at https://stackoverflow.com/questions/60930359/docker-containers-exit-code-132 and at RasaHQ/rasa#4602 that in an amd64 context, it might be due to building with amd64-avx and not simply amd64.
On openresty/docker-openresty#39, more or less the same but with sse4.2

I don't know if the same kind of issue might happen in arm64 context and if something was changed in the build recently ?

leolivier · 2022-11-19T11:02:23Z

Likely to be linked to #2538 I'm afraid. It changes the C++ compilation flags for ARM build...
EDIT: or maybe this previous MR on the same topic: #2535

EDIT2: Maybe I'm wrong but it looks like the CI jobs are not running anything on ARM!
See https://github.com/Mailu/Mailu/actions/runs/3487823441/jobs/5835996825: all jobs related to ARM seem to be skipped.

nextgens · 2022-11-19T14:33:50Z

It's most likely one of the compiler flags at

Mailu/core/base/Dockerfile

Line 26 in 68bb8da

... I'm investigating.

nextgens · 2022-11-19T16:38:01Z

This works on the aarch64 machine I have access to.

root@a2:~/dev/storage# docker ps 
CONTAINER ID   IMAGE                           COMMAND                  CREATED              STATUS                        PORTS                                                                                                                                                                                                                                NAMES
1a54d32de8da   mailu/postfix:master-arm        "/bin/sh -c /start.py"   About a minute ago   Up About a minute (healthy)   25/tcp, 10025/tcp                                                                                                                                                                                                                    storage-smtp-1
18430bca9627   mailu/admin:master-arm          "/bin/sh -c /start.py"   About a minute ago   Up About a minute (healthy)   80/tcp                                                                                                                                                                                                                               storage-admin-1
53f074c8614c   mailu/roundcube:master-arm      "/bin/sh -c /start.py"   About a minute ago   Up About a minute (healthy)   80/tcp                                                                                                                                                                                                                               storage-webmail-1
2d5b09459747   mailu/dovecot:master-arm        "/bin/sh -c /start.py"   About a minute ago   Up About a minute (healthy)   110/tcp, 143/tcp, 993/tcp, 2525/tcp, 4190/tcp                                                                                                                                                                                        storage-imap-1
6dcc523d4c15   mailu/rspamd:master-arm         "/bin/sh -c /start.py"   About a minute ago   Up About a minute (healthy)   11332/tcp, 11334-11335/tcp                                                                                                                                                                                                           storage-antispam-1
4ec576a01421   mailu/nginx:master-arm          "/bin/sh -c /start.py"   About a minute ago   Up About a minute (healthy)   0.0.0.0:25->25/tcp, 127.0.0.1:110->110/tcp, 0.0.0.0:80->80/tcp, 127.0.0.1:143->143/tcp, 127.0.0.1:465->465/tcp, 127.0.0.1:587->587/tcp, 127.0.0.1:993->993/tcp, 10025/tcp, 0.0.0.0:443->443/tcp, 127.0.0.1:995->995/tcp, 10143/tcp   storage-front-1
79f0c980563f   redis:alpine                    "docker-entrypoint.s…"   About a minute ago   Up About a minute             6379/tcp                                                                                                                                                                                                                             storage-redis-1
7c0b8f50aa11   mailu/unbound:master-arm        "/bin/sh -c /start.py"   About a minute ago   Up About a minute (healthy)   53/tcp, 53/udp                                                                                                                                                                                                                       storage-resolver-1
2f5722a4fdfe   moby/buildkit:buildx-stable-1   "buildkitd"              2 weeks ago          Up 2 weeks                                                                                                                                                                                                                                                         buildx_buildkit_competent_margulis0
root@a2:~/dev/storage# docker exec -ti storage-admin-1 /bin/sh -c 'uname -a; ps'
Linux 18430bca9627 5.15.0-1016-oracle #20-Ubuntu SMP Mon Aug 8 07:08:08 UTC 2022 aarch64 Linux
PID   USER     TIME  COMMAND
    1 root      0:00 python3 /start.py
    8 root      0:02 {gunicorn} /app/venv/bin/python3 /app/venv/bin/gunicorn --threads 4 -b :80 --access-logf
    9 root      0:00 {gunicorn} /app/venv/bin/python3 /app/venv/bin/gunicorn --threads 4 -b :80 --access-logf
   58 root      0:00 ps
root@a2:~/dev/storage# docker exec -ti storage-admin-1 /bin/bash -c 'apk add lsof;lsof |grep /usr/lib/libhardened_malloc.so'
OK: 70 MiB in 43 packages
python3    1              root  mem       REG      8,1          14542605 /usr/lib/libhardened_malloc.so (path dev=0,89)
gunicorn   8              root  mem       REG      8,1          14542605 /usr/lib/libhardened_malloc.so (path dev=0,89)
gunicorn   9              root  mem       REG      8,1          14542605 /usr/lib/libhardened_malloc.so (path dev=0,89)
gunicorn   9  10 gunicorn root  mem       REG      8,1          14542605 /usr/lib/libhardened_malloc.so (path dev=0,89)
gunicorn   9  11 gunicorn root  mem       REG      8,1          14542605 /usr/lib/libhardened_malloc.so (path dev=0,89)
gunicorn   9  12 gunicorn root  mem       REG      8,1          14542605 /usr/lib/libhardened_malloc.so (path dev=0,89)
gunicorn   9  13 gunicorn root  mem       REG      8,1          14542605 /usr/lib/libhardened_malloc.so (path dev=0,89)
bash     172              root  mem       REG      8,1          14542605 /usr/lib/libhardened_malloc.so (path dev=0,89)
lsof     179              root  mem       REG      8,1          14542605 /usr/lib/libhardened_malloc.so (path dev=0,89)
grep     180              root  mem       REG      8,1          14542605 /usr/lib/libhardened_malloc.so (path dev=0,89)
lsof     181              root  mem       REG      8,1          14542605 /usr/lib/libhardened_malloc.so (path dev=0,89)

nextgens · 2022-11-19T16:39:30Z

@leolivier @Minion3665 we have recently added two things: libhardened_malloc via LD_PRELOAD and a bunch of compiler hardening flags; if you could find what is causing the problems that'd be great

leolivier · 2022-11-19T16:43:45Z

@nextgens, how can we do that?
What I can tell is that I update my images every week, and the images of last Friday (11/11) were working (I don't remember if they were updated at that time, so it's maybe older images actually, but I did a pull on 11/11).
If you provide some images with or without the flags, I can try them quickly and tell you if they are working or not.

EDIT: Maybe you can give me the name/tag of your personal docker hub images as you did when we worked together on the ARM support ?

leolivier · 2022-11-19T16:46:59Z

This works on the aarch64 machine I have access to.

Is it a raspberry pi? I have switched recently my RPi 4 to arm64 version, but the images were working before I updated today...

nextgens · 2022-11-19T17:13:40Z

Just recompile the images with the proposed changes:

export MAILU_VERSION=local-arm
docker buildx bake -f tests/build.hcl --set '*.platform=linux/arm64/v8'
docker-compose up -d

For speed sake, you may want to try only one of the failing images (don't forget MAILU_VERSION=local-arm)

docker buildx bake -f tests/build.hcl --set '*.platform=linux/arm64/v8' resolver

and edit your docker-compose.yml to ensure that image is used in place of the others

nextgens · 2022-11-19T17:15:59Z

This works on the aarch64 machine I have access to.

Is it a raspberry pi? I have switched recently my RPi 4 to arm64 version, but the images were working before I updated today...

That's an ampere A1 instance on oracle's cloud.

leolivier · 2022-11-19T17:21:12Z

docker buildx bake --no-cache -f tests/build.hcl --set '*.platform=linux/arm64/v8' resolver

But if I do that on the master branch, I will have the same result as the hub image, won't I?
Anyway, I will try it but I don't get the idea...

Minion3665 · 2022-11-19T17:24:18Z

docker buildx bake --no-cache -f tests/build.hcl --set '*.platform=linux/arm64/v8' resolver

But if I do that on the master branch, I will have the same result as the hub image, won't I? Anyway, I will try it but I don't get the idea...

I believe the idea is to do it after modifying the code (e.g. I checked-out a week-old commit and did it, my results are on docker hub under minion3665 and are working for me). Hopefully if we build enough we can pinpoint where the error is

nextgens · 2022-11-19T17:25:17Z

Somewhere in between line 21 and 25 on

Mailu/core/base/Dockerfile

Line 26 in 68bb8da

is my guess.

leolivier · 2022-11-19T17:37:18Z

I confirm @Minion3665 images are working for me too.
I will use them until the bug is fixed.
@Minion3665 are you trying to rebuild the different combinations of commits? If you do it, I won't as I would need to recreate my whole docker devt environment which I destroyed recently...

nextgens · 2022-11-19T17:58:44Z

@leolivier commenting line by line line 21 to 25 and finding what exactly is breaking things would be helpful.

nextgens · 2022-11-19T18:05:25Z

I've retried rebuilding everything for armv7 and that works too on this CPU:

root@a2:~/dev/storage# docker run --rm -it mailu/nginx:local-arm7 /bin/bash
WARNING: The requested image's platform (linux/arm/v7) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
bash-5.1# ldd /bin/hostname 
	/lib/ld-musl-armhf.so.1 (0xf7ef3000)
	libc.musl-armv7.so.1 => /lib/ld-musl-armhf.so.1 (0xf7ef3000)

Minion3665 · 2022-11-19T18:07:15Z

I confirm @Minion3665 images are working for me too. I will use them until the bug is fixed. @Minion3665 are you trying to rebuild the different combinations of commits? If you do it, I won't as I would need to recreate my whole docker devt environment which I destroyed recently...

I am not right now, however I may do soon (i.e. when I finish some other work I have to do) so depending on how quickly you want this fixed you can either try rebuilding or not.

nextgens · 2022-11-19T18:08:52Z

It would be good if we could identify a test-case that would break on the builder: there's no point in them running the test-suite if I can't make it crash there.

leolivier · 2022-11-19T18:10:06Z

I am not right now, however I may do soon (i.e. when I finish some other work I have to do) so depending on how quickly you want this fixed you can either try rebuilding or not.
Actually, your current front image has an issue and does not start for me (I think it's linked to webmail) so I have no working version...

leolivier · 2022-11-19T18:13:13Z

It would be good if we could identify a test-case that would break on the builder: there's no point in them running the test-suite if I can't make it crash there.

As I said, I destroyed my devt envt so I just tried a git clone on my RPi4 directly then export MAILU_VERSION and docker buildx bake on the Pi. I have this strange 132 error which does seem to be linked to any cpp variable:

ERROR: failed to solve: process "/bin/sh -c set -euxo pipefail   ; apk add --no-cache py3-pip   ; python3 -m venv ${VIRTUAL_ENV}   ; ${VIRTUAL_ENV}/bin/pip install --no-cache-dir -r requirements-build.txt   ; apk del -r py3-pip   ; rm -f /tmp/*.pem" did not complete successfully: exit code: 132

nextgens · 2022-11-19T18:15:29Z

132 means 'illegal instruction'; it could be anything.

Remove line 21 to 25 in Mailu/core/base/Dockerfile and retry.

leolivier · 2022-11-19T18:16:33Z

I did that already but it keeps stopping on the above error...

nextgens · 2022-11-19T18:19:45Z

Including line 21 with LD_PRELOAD ? That could well be what makes pip crash.

nextgens · 2022-11-19T18:22:08Z

If that still doesn't work, try rebuilding without buildx:

cd core/base
docker build -t test/test .

leolivier · 2022-11-19T18:28:27Z

LD_PRELOAD didn't work. I tried docker build w/o buildx but it fails too because TARGETPLATFORM is not set :/

+ uname -m
+ machine=aarch64
/bin/sh: TARGETPLATFORM: parameter not set
The command '/bin/sh -c set -euxo pipefail   ; addgroup -Sg ${MAILU_GID} mailu   ; adduser -Sg ${MAILU_UID} -G mailu -h /app -g "mailu app" -s /bin/bash mailu   ; apk add --no-cache bash ca-certificates curl python3 tzdata   ; machine="$(uname -m)"   ; ! [[ "${TARGETPLATFORM}" != linux/arm/v7 && \( "${machine}" == x86_64 || "${machine}" == armv8* || "${machine}" == aarch64 \) ]]     || apk add --no-cache --repository=http://dl-cdn.alpinelinux.org/alpine/edge/testing hardened-malloc' returned a non-zero code: 2

I must confess that I worked on a lot of other things since I worked on this topic and I have forgotten everything, so I'm not efficient at all :(

nextgens · 2022-11-19T18:30:43Z

try with --build-arg TARGETPLATFORM=linux/arm64/v8

leolivier · 2022-11-19T18:33:39Z

try with --build-arg TARGETPLATFORM=linux/arm64/v8

I added that and I'm back to the same issue:

The command '/bin/sh -c set -euxo pipefail   ; apk add --no-cache py3-pip   ; python3 -m venv ${VIRTUAL_ENV}   ; ${VIRTUAL_ENV}/bin/pip install --no-cache-dir -r requirements-build.txt   ; apk del -r py3-pip   ; rm -f /tmp/*.pem' returned a non-zero code: 132

nextgens · 2022-11-19T19:00:23Z

This may be a different bug/problem. Looking at the diff in between the two commits mentioned in the issue:

git diff 8a90f83..68bb8da -- core/base/Dockerfile
diff --git a/core/base/Dockerfile b/core/base/Dockerfile
index d5be6a90..6f31e21c 100644
--- a/core/base/Dockerfile
+++ b/core/base/Dockerfile
@@ -8,11 +8,21 @@ ENV TZ=Etc/UTC LANG=C.UTF-8
 
 ARG MAILU_UID=1000
 ARG MAILU_GID=1000
+ARG TARGETPLATFORM
 
 RUN set -euxo pipefail \
   ; addgroup -Sg ${MAILU_GID} mailu \
   ; adduser -Sg ${MAILU_UID} -G mailu -h /app -g "mailu app" -s /bin/bash mailu \
-  ; apk add --no-cache bash ca-certificates curl python3 tzdata
+  ; apk add --no-cache bash ca-certificates curl python3 tzdata \
+  ; machine="$(uname -m)" \
+  ; ! [[ "${TARGETPLATFORM}" != linux/arm/v7 && \( "${machine}" == x86_64 || "${machine}" == armv8* || "${machine}" == aarch64 \) ]] \
+    || apk add --no-cache --repository=http://dl-cdn.alpinelinux.org/alpine/edge/testing hardened-malloc
+
+ENV LD_PRELOAD=/usr/lib/libhardened_malloc.so
+ENV CXXFLAGS="-g -O2 -fdebug-prefix-map=/app=. -fstack-protector-strong -Wformat -Werror=format-security -fstack-clash-protection -fexceptions"
+ENV CFLAGS="-g -O2 -fdebug-prefix-map=/app=. -fstack-protector-strong -Wformat -Werror=format-security -fstack-clash-protection -fexceptions"
+ENV CPPFLAGS="-Wdate-time -D_FORTIFY_SOURCE=2"
+ENV LDFLAGS="-Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now"
 
 WORKDIR /app

leolivier · 2022-11-20T12:15:05Z

I finally was able to restore my devt environment.
I removed all lines from 21 to 25 in the Dockerfile and rebuilt resolver and it works.
I will now 1rts rebuild everything with that context to have my whole server working (hopefully) then I will try to put back these lines one by one and keep you updated once there is an issue on resolver.

leolivier · 2022-11-20T14:51:27Z

Hi again, 1rst result:
Adding only the line NV LD_PRELOAD=/usr/lib/libhardened_malloc.so breaks the build itself:

ERROR: failed to solve: process "/bin/sh -c set -euxo pipefail   ; apk add --no-cache py3-pip   ; python3 -m venv ${VIRTUAL_ENV}   ; ${VIRTUAL_ENV}/bin/pip install --no-cache-dir -r requirements-build.txt   ; apk del -r py3-pip   ; rm -f /tmp/*.pem" did not complete successfully: exit code: 137

(Note the 137 code, not 132 - which means by the way RF KILL if this is an errno, not illegal instruction)

I will now try with other lines, w/o the hardened malloc

leolivier · 2022-11-20T15:06:28Z

I confirm that only removing the line on hardened malloc solves the issue! (tested only on resolver for now but I think it's enough)

nextgens · 2022-11-20T15:13:01Z

@leolivier please check whether LD_PRELOAD=/usr/lib/libhardened_malloc-light.so works

@see Mailu#2541 Support is going to be a nightmare if RPI4 is not working.

nextgens · 2022-11-20T17:44:49Z

@leolivier @Minion3665 the PR has been merged and new images should have been pushed; please confirm they work.

leolivier · 2022-11-20T17:55:23Z

/usr/lib/libhardened_malloc-light.so

Same 137 error...

leolivier · 2022-11-20T18:05:18Z

@leolivier @Minion3665 the PR has been merged and new images should have been pushed; please confirm they work.

Thanks for the new images!
Everything looks ok except snappymail which seems to be frozen at start. I will investigate a bit more on that one.

I know this is still beta but would it be possible to work on having specific tags on each new versions so we can rollback to the previous version when such an issue arises?

EDIT: finally snappymail also starts but awfully slowly... It takes more than 5mns to start and run nginx. Looks like it was stuck on the "chmod -R a+rX /var/www/webmail/" command inside start.py which is weird because the command is very quick if I run it manually... Anyway, after this slow start, it works normally !

Minion3665 · 2022-11-20T19:46:58Z

@leolivier @Minion3665 the PR has been merged and new images should have been pushed; please confirm they work.

Thank you very much, everything is working perfectly on my end

I know this is still beta but would it be possible to work on having specific tags on each new versions so we can rollback to the previous version when such an issue arises?

The problem is that each version is a commit, if it were done on existing pages I suspect this would lead to a lot of tags very quickly which'd be quite a bit of noise for anyone looking at tags on docker hub. If this were done perhaps a second org/similar would be an advantage

nextgens · 2022-11-21T07:50:34Z

@leolivier @Minion3665 the PR has been merged and new images should have been pushed; please confirm they work.

Thanks for the new images! Everything looks ok

Thank you for the feedback.

except snappymail which seems to be frozen at start. I will investigate a bit more on that one.

Massive work has been done on #2526 ; it may improve things. By default snappymail uses a file-based cache... and we switch to APCu.

I know this is still beta but would it be possible to work on having specific tags on each new versions so we can rollback to the previous version when such an issue arises?

We've talked about it in the past and the answer is no. When the ARM images will hit a stable release then that will be possible... but we don't want people running different shades of master. To solve your specific problem, I suggest you look into persisting the "running" images before attempting an update; what I usually do is stop the containers, rename them and then upgrade. If the upgrade works I then dispose of the stopped containers... and if it doesn't I just restart them.

leolivier · 2022-11-21T10:49:23Z

Thanks for the suggestion @nextgens, I will try that

@see Mailu#2541 Support is going to be a nightmare if RPI4 is not working.

nextgens · 2022-11-24T16:57:56Z

@leolivier

docker images --format "{{.Repository}}:{{.Tag}}"|grep :master|while read image; do docker tag "$image" "${image/:master/:working}"; done

leolivier · 2022-11-25T07:04:02Z

Thank you @nextgens

nextgens · 2023-04-14T08:15:11Z

@leolivier can you post the output of cat /proc/cpuinfo from one of those rpi4 please?

leolivier · 2023-04-14T08:54:58Z

@nextgens here it is:

pi@raspberrypi ~ $ cat /proc/cpuinfo
processor       : 0
BogoMIPS        : 108.00
Features        : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3

processor       : 1
BogoMIPS        : 108.00
Features        : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3

processor       : 2
BogoMIPS        : 108.00
Features        : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3

processor       : 3
BogoMIPS        : 108.00
Features        : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3

Hardware        : BCM2835
Revision        : c03112
Serial          : 10000000adfa00dd
Model           : Raspberry Pi 4 Model B Rev 1.2

2771: Sanitize logs as appropriate r=mergify[bot] a=nextgens ## What type of PR? enhancement ## What does this PR do? - Sanitize logs as appropriate. - change the healthcheck of radicale to something less verbose - disable hardened-malloc if we detect a processor not supporting the AVX extension set Should we backport something like that? It could be argued it's a bugfix. ### Related issue(s) - close #2644 - close #2764 - #2541 ## Prerequisites Before we can consider review and merge, please make sure the following list is done and checked. If an entry in not applicable, you can check it or remove it from the list. - [ ] In case of feature or enhancement: documentation updated accordingly - [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file. Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>

2791: LD_PRELOAD may not be in ENV r=mergify[bot] a=nextgens ## What type of PR? bug-fix ## What does this PR do? In front, config.py can be called several times. LD_PRELOAD may have already been removed from ENV ### Related issue(s) - close #2789 - #2541 ## Prerequisites Before we can consider review and merge, please make sure the following list is done and checked. If an entry in not applicable, you can check it or remove it from the list. - [ ] In case of feature or enhancement: documentation updated accordingly - [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file. Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>

2791: LD_PRELOAD may not be in ENV r=nextgens a=nextgens ## What type of PR? bug-fix ## What does this PR do? In front, config.py can be called several times. LD_PRELOAD may have already been removed from ENV ### Related issue(s) - close #2789 - #2541 ## Prerequisites Before we can consider review and merge, please make sure the following list is done and checked. If an entry in not applicable, you can check it or remove it from the list. - [ ] In case of feature or enhancement: documentation updated accordingly - [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file. Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>

Minion3665 changed the title ~~Mailu on arm exists with code 132~~ Mailu on arm exits with code 132 Nov 18, 2022

nextgens added the type/bug Bug. Not working as intended label Nov 19, 2022

nextgens added a commit to nextgens/Mailu that referenced this issue Nov 20, 2022

Disable libhardened-malloc for non x86.

db9ed1f

@see Mailu#2541 Support is going to be a nightmare if RPI4 is not working.

nextgens mentioned this issue Nov 20, 2022

Disable libhardened-malloc for non x86. #2547

Merged

bors bot closed this as completed in 31c6c26 Nov 20, 2022

nextgens added a commit to nextgens/Mailu that referenced this issue Nov 23, 2022

Disable libhardened-malloc for non x86.

3d4e1b2

@see Mailu#2541 Support is going to be a nightmare if RPI4 is not working.

nextgens mentioned this issue Apr 14, 2023

Sanitize logs as appropriate #2771

Merged

2 tasks

nextgens mentioned this issue Apr 20, 2023

LD_PRELOAD may not be in ENV #2791

Merged

2 tasks

Mailu on arm exits with code 132 #2541

Mailu on arm exits with code 132 #2541

Comments

Minion3665 commented Nov 18, 2022

Environment & Version

Environment

Version

Description

Replication Steps

Observed behaviour

Expected behaviour

Extra notes

Logs

leolivier commented Nov 19, 2022 • edited Loading

leolivier commented Nov 19, 2022

leolivier commented Nov 19, 2022 • edited Loading

nextgens commented Nov 19, 2022

nextgens commented Nov 19, 2022

nextgens commented Nov 19, 2022

leolivier commented Nov 19, 2022 • edited Loading

leolivier commented Nov 19, 2022

nextgens commented Nov 19, 2022 • edited Loading

nextgens commented Nov 19, 2022

leolivier commented Nov 19, 2022 • edited Loading

Minion3665 commented Nov 19, 2022

nextgens commented Nov 19, 2022

leolivier commented Nov 19, 2022

nextgens commented Nov 19, 2022

nextgens commented Nov 19, 2022

Minion3665 commented Nov 19, 2022

nextgens commented Nov 19, 2022

leolivier commented Nov 19, 2022

leolivier commented Nov 19, 2022 • edited Loading

nextgens commented Nov 19, 2022

leolivier commented Nov 19, 2022

nextgens commented Nov 19, 2022

nextgens commented Nov 19, 2022

leolivier commented Nov 19, 2022

nextgens commented Nov 19, 2022

leolivier commented Nov 19, 2022

nextgens commented Nov 19, 2022

leolivier commented Nov 20, 2022

leolivier commented Nov 20, 2022

leolivier commented Nov 20, 2022

nextgens commented Nov 20, 2022

nextgens commented Nov 20, 2022

leolivier commented Nov 20, 2022

leolivier commented Nov 20, 2022 • edited Loading

Minion3665 commented Nov 20, 2022

nextgens commented Nov 21, 2022 • edited Loading

leolivier commented Nov 21, 2022

nextgens commented Nov 24, 2022

leolivier commented Nov 25, 2022

nextgens commented Apr 14, 2023

leolivier commented Apr 14, 2023

leolivier commented Nov 19, 2022 •

edited

Loading

leolivier commented Nov 19, 2022 •

edited

Loading

leolivier commented Nov 19, 2022 •

edited

Loading

nextgens commented Nov 19, 2022 •

edited

Loading

leolivier commented Nov 19, 2022 •

edited

Loading

leolivier commented Nov 19, 2022 •

edited

Loading

leolivier commented Nov 20, 2022 •

edited

Loading

nextgens commented Nov 21, 2022 •

edited

Loading