segmentation fault on arm64 during rabbitmq-plugins enable --offline rabbitmq_shovel #593
Comments
|
Are you running this build on a real arm64 system, or an emulated one? |
|
It's a gitlab runner, I assume it's a virtual host |
Please do your part to be helpful. I'm assuming there is some way to get information about the CI environment from GitLab. When a CI job runs on GitHub, for instance, the log shows the build information: GitHub CI build headerIf something similar is available from GitLab, or if you can provide a link to the build that failed, it would help us out. |
|
This is likely a duplicate of #583. erlang/otp#6340 is also highly relevant. |
|
I've managed to get some info: The initial log lines are: after that, there is some build-specific lines like pulling the repo and then the installation of docker buildx: Using docker image sha256:6972c414f322dfa40324df3c503d4b217ccdec6d576e408ed10437f508f4181b for docker:19.03.13 with digest docker@sha256:84165ff897a6e328c702f8da72046efb7e00c6aa3d3b2c5057f276707bcedf33 ...
$ mkdir $HOME/.docker
$ echo -e '{ # collapsed multi-line command
{
"experimental": "enabled"
}
$ docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
Unable to find image 'multiarch/qemu-user-static:latest' locally
latest: Pulling from multiarch/qemu-user-static
f5b7ce95afea: Pulling fs layer
22ee763f0af7: Pulling fs layer
30abb83a18eb: Pulling fs layer
0657daef200b: Pulling fs layer
998118dcc3de: Pulling fs layer
0657daef200b: Waiting
998118dcc3de: Waiting
22ee763f0af7: Verifying Checksum
22ee763f0af7: Download complete
30abb83a18eb: Verifying Checksum
30abb83a18eb: Download complete
f5b7ce95afea: Verifying Checksum
f5b7ce95afea: Download complete
0657daef200b: Verifying Checksum
0657daef200b: Download complete
f5b7ce95afea: Pull complete
22ee763f0af7: Pull complete
30abb83a18eb: Pull complete
0657daef200b: Pull complete
998118dcc3de: Verifying Checksum
998118dcc3de: Download complete
998118dcc3de: Pull complete
Digest: sha256:16abbe859db6a7d142d90125f3260008f698a224ccd0e776d554cfc46fd025b6
Status: Downloaded newer image for multiarch/qemu-user-static:latest
Setting /usr/bin/qemu-alpha-static as binfmt interpreter for alpha
Setting /usr/bin/qemu-arm-static as binfmt interpreter for arm
Setting /usr/bin/qemu-armeb-static as binfmt interpreter for armeb
Setting /usr/bin/qemu-sparc-static as binfmt interpreter for sparc
Setting /usr/bin/qemu-sparc32plus-static as binfmt interpreter for sparc32plus
Setting /usr/bin/qemu-sparc64-static as binfmt interpreter for sparc64
Setting /usr/bin/qemu-ppc-static as binfmt interpreter for ppc
Setting /usr/bin/qemu-ppc64-static as binfmt interpreter for ppc64
Setting /usr/bin/qemu-ppc64le-static as binfmt interpreter for ppc64le
Setting /usr/bin/qemu-m68k-static as binfmt interpreter for m68k
Setting /usr/bin/qemu-mips-static as binfmt interpreter for mips
Setting /usr/bin/qemu-mipsel-static as binfmt interpreter for mipsel
Setting /usr/bin/qemu-mipsn32-static as binfmt interpreter for mipsn32
Setting /usr/bin/qemu-mipsn32el-static as binfmt interpreter for mipsn32el
Setting /usr/bin/qemu-mips64-static as binfmt interpreter for mips64
Setting /usr/bin/qemu-mips64el-static as binfmt interpreter for mips64el
Setting /usr/bin/qemu-sh4-static as binfmt interpreter for sh4
Setting /usr/bin/qemu-sh4eb-static as binfmt interpreter for sh4eb
Setting /usr/bin/qemu-s390x-static as binfmt interpreter for s390x
Setting /usr/bin/qemu-aarch64-static as binfmt interpreter for aarch64
Setting /usr/bin/qemu-aarch64_be-static as binfmt interpreter for aarch64_be
Setting /usr/bin/qemu-hppa-static as binfmt interpreter for hppa
Setting /usr/bin/qemu-riscv32-static as binfmt interpreter for riscv32
Setting /usr/bin/qemu-riscv64-static as binfmt interpreter for riscv64
Setting /usr/bin/qemu-xtensa-static as binfmt interpreter for xtensa
Setting /usr/bin/qemu-xtensaeb-static as binfmt interpreter for xtensaeb
Setting /usr/bin/qemu-microblaze-static as binfmt interpreter for microblaze
Setting /usr/bin/qemu-microblazeel-static as binfmt interpreter for microblazeel
Setting /usr/bin/qemu-or1k-static as binfmt interpreter for or1k
Setting /usr/bin/qemu-hexagon-static as binfmt interpreter for hexagon
$ mkdir -p $HOME/.docker/cli-plugins/
$ wget -O $HOME/.docker/cli-plugins/docker-buildx $BUILDX_URL
Connecting to storage.googleapis.com (172.253.123.128:443)
saving to '/root/.docker/cli-plugins/docker-buildx'
docker-buildx 0% | | 430k 0:02:09 ETA
docker-buildx 54% |***************** | 29.8M 0:00:01 ETA
docker-buildx 100% |********************************| 54.8M 0:00:00 ETA
'/root/.docker/cli-plugins/docker-buildx' saved
$ chmod a+x $HOME/.docker/cli-plugins/docker-buildx
$ docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
$ if [ "$CI_COMMIT_REF_NAME" == "master" ]; then # collapsed multi-line command
$ IMAGE_NAME="${CI_REGISTRY_IMAGE}/rabbitmq"
$ IMAGE="${IMAGE_NAME}:${TAG}"
$ IMAGE_LATEST="${IMAGE_NAME}:latest"
$ if docker pull $IMAGE; then # collapsed multi-line commandafter that, the docker gets built and fails as mentioned: docker buildx create --use
heuristic_shaw
$ docker buildx build --platform linux/amd64,linux/arm64/v8 --build-arg BUILDKIT_INLINE_CACHE=1 $CACHE_ARG --push -t $IMAGE -t $IMAGE_LATEST -f dockerfiles/rabbitmq/Dockerfile dockerfiles/rabbitmq
time="2022-12-02T18:03:53Z" level=warning msg="invalid non-bool value for BUILDX_NO_DEFAULT_LOAD: "
#1 [internal] booting buildkit
#1 pulling image moby/buildkit:buildx-stable-1
#1 pulling image moby/buildkit:buildx-stable-1 3.6s done
#1 creating container buildx_buildkit_heuristic_shaw0
#1 creating container buildx_buildkit_heuristic_shaw0 2.5s done
#1 DONE 6.2s
#3 [internal] load build definition from Dockerfile
#3 transferring dockerfile:
#3 transferring dockerfile: 495B done
#3 DONE 0.1s
#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.1s
#5 [linux/amd64 internal] load metadata for docker.io/library/rabbitmq:3.11...
#5 DONE 1.2s
#4 [linux/arm64 internal] load metadata for docker.io/library/rabbitmq:3.11...
#4 DONE 1.2s
#12 [internal] load build context
#12 DONE 0.0s
#17 [linux/amd64 1/10] FROM docker.io/library/rabbitmq:3.11-management@sha2...
#17 resolve docker.io/library/rabbitmq:3.11-management@sha256:d63d5721a955527fef1df0d2c12cd7a87ab6e3a8f4de232d77f0025e181db23a 0.0s done
#17 DONE 0.0s
#6 [linux/arm64 1/10] FROM docker.io/library/rabbitmq:3.11-management@sha2...
#6 resolve docker.io/library/rabbitmq:3.11-management@sha256:d63d5721a955527fef1df0d2c12cd7a87ab6e3a8f4de232d77f0025e181db23a 0.0s done
#6 DONE 0.0s
#27 importing cache manifest from registry.gitlab.com/xxxxxxxxx...
#27 DONE 0.3s
#12 [internal] load build context
#12 transferring context: 13.57kB done
#12 DONE 0.1s
#6 [linux/arm64 1/10] FROM docker.io/library/rabbitmq:3.11-management@sha2...
#6 DONE 0.3s
#17 [linux/amd64 1/10] FROM docker.io/library/rabbitmq:3.11-management@sha2...
#17 sha256:e9c0e835b826e5cc1ab0b795342ee081f3f6721ef21e478bc752a24ae6d165ca 1.05MB / 11.77MB 0.2s
#17 sha256:715c1c578966eb68312919ec19789d5c5c5c62e643cfbaed23cf9b125f55a4eb 833B / 833B 0.1s done
#17 sha256:ed8f0defe00e7b3134ec8c7c13a9d296ad5e27affd1903f894109153bcf20036 501B / 501B 0.2s done
#17 sha256:e9c0e835b826e5cc1ab0b795342ee081f3f6721ef21e478bc752a24ae6d165ca 8.23MB / 11.77MB 0.3s
#17 sha256:98b33de64338626964a040181538fb88c9767d1993018750e02631d99cb5d274 107B / 107B 0.1s done
#17 sha256:da2156e3127fb9d3e13f39f3f2f213d11b8d51f6ccadf2cec51b08997d3df45a 273B / 273B 0.1s done
#17 sha256:e9c0e835b826e5cc1ab0b795342ee081f3f6721ef21e478bc752a24ae6d165ca 11.77MB / 11.77MB 0.5s
#17 sha256:32e52a38037e5a39d190cdbd275aa4830c4a7b34af77ea3b0f01ba777875a324 2.08kB / 2.08kB 0.2s
#17 sha256:e9c0e835b826e5cc1ab0b795342ee081f3f6721ef21e478bc752a24ae6d165ca 11.77MB / 11.77MB 0.5s done
#17 sha256:32e52a38037e5a39d190cdbd275aa4830c4a7b34af77ea3b0f01ba777875a324 2.08kB / 2.08kB 0.2s done
#17 sha256:0cfc7bb1390335bc5366c6a0e74be6ba0e29e0e509a3da7e80647a8768afc4ed 8.39MB / 17.84MB 0.3s
#17 sha256:de99a1ffedd324479cb1f1ed33f971553d21b568ab1bc7429e06d779a34722a8 0B / 52.30MB 0.2s
#17 sha256:d5e775568c003580578d8757a1a3785c36c52424de3a5dc4a665eb5b2f392bdb 1.84MB / 1.84MB 0.1s done
#17 sha256:0cfc7bb1390335bc5366c6a0e74be6ba0e29e0e509a3da7e80647a8768afc4ed 17.84MB / 17.84MB 0.5s
#17 sha256:de99a1ffedd324479cb1f1ed33f971553d21b568ab1bc7429e06d779a34722a8 3.15MB / 52.30MB 0.3s
#17 sha256:eaead16dc43bb8811d4ff450935d607f9ba4baffda4fc110cc402fa43f601d83 2.01MB / 28.58MB 0.2s
#17 sha256:0cfc7bb1390335bc5366c6a0e74be6ba0e29e0e509a3da7e80647a8768afc4ed 17.84MB / 17.84MB 0.6s done
#17 sha256:de99a1ffedd324479cb1f1ed33f971553d21b568ab1bc7429e06d779a34722a8 14.68MB / 52.30MB 0.6s
#17 sha256:eaead16dc43bb8811d4ff450935d607f9ba4baffda4fc110cc402fa43f601d83 8.39MB / 28.58MB 0.3s
#17 sha256:eaead16dc43bb8811d4ff450935d607f9ba4baffda4fc110cc402fa43f601d83 12.58MB / 28.58MB 0.5s
#17 sha256:de99a1ffedd324479cb1f1ed33f971553d21b568ab1bc7429e06d779a34722a8 19.92MB / 52.30MB 0.8s
#17 sha256:eaead16dc43bb8811d4ff450935d607f9ba4baffda4fc110cc402fa43f601d83 19.92MB / 28.58MB 0.6s
#17 sha256:de99a1ffedd324479cb1f1ed33f971553d21b568ab1bc7429e06d779a34722a8 27.26MB / 52.30MB 0.9s
#17 sha256:de99a1ffedd324479cb1f1ed33f971553d21b568ab1bc7429e06d779a34722a8 37.75MB / 52.30MB 1.1s
#17 sha256:eaead16dc43bb8811d4ff450935d607f9ba4baffda4fc110cc402fa43f601d83 28.31MB / 28.58MB 0.8s
#17 sha256:de99a1ffedd324479cb1f1ed33f971553d21b568ab1bc7429e06d779a34722a8 44.04MB / 52.30MB 1.2s
#17 sha256:eaead16dc43bb8811d4ff450935d607f9ba4baffda4fc110cc402fa43f601d83 28.58MB / 28.58MB 1.0s done
#17 extracting sha256:eaead16dc43bb8811d4ff450935d607f9ba4baffda4fc110cc402fa43f601d83
#17 sha256:de99a1ffedd324479cb1f1ed33f971553d21b568ab1bc7429e06d779a34722a8 48.23MB / 52.30MB 1.4s
#17 sha256:de99a1ffedd324479cb1f1ed33f971553d21b568ab1bc7429e06d779a34722a8 52.30MB / 52.30MB 1.5s
#17 sha256:de99a1ffedd324479cb1f1ed33f971553d21b568ab1bc7429e06d779a34722a8 52.30MB / 52.30MB 1.8s done
#17 extracting sha256:eaead16dc43bb8811d4ff450935d607f9ba4baffda4fc110cc402fa43f601d83 3.6s done
#17 DONE 5.4s
#6 [linux/arm64 1/10] FROM docker.io/library/rabbitmq:3.11-management@sha2...
#6 sha256:d7b58f2a665d2108e75d1361fe38549159c5b8ef60671018b97499cbbec508ae 11.73MB / 11.73MB 1.2s done
#6 sha256:b3051569353899b1f92064e3942a5694fc7775f8fd8e644cf12021be2ab5de30 835B / 835B 0.1s done
#6 sha256:393fec3d83dae7e40db04617215e575bedd01ab8a0b6452b43e51ea9ce5ad165 501B / 501B 0.1s done
#6 sha256:f09e3d0ad471ce95ed54a9c349ecacd27890c883bb8e95a2422e8bb026ef28fe 107B / 107B 0.2s done
#6 sha256:ccdad4bc939f28af8e65ea17cedb5038d7c60190106e6359d804bd06fe00b700 274B / 274B 0.1s done
#6 sha256:1ceaa0deba1f91d7e74aaeb7eae8735c30bffbda30cca819a8cfa0139362bce0 17.84MB / 17.84MB 1.0s done
#6 sha256:51aa5245f5c90da1e1d00b96f1be695c6627e081951aa9714827beb8ca566215 2.09kB / 2.09kB 0.2s done
#6 sha256:9f860bc38ec0cd69bbd3ef3eca48b40fe62c8db1de95cd38c07b202ceb339dd2 50.40MB / 50.40MB 1.6s done
#6 sha256:d6ec0ff8e83c114f957234a0898fc6a507348b7b69eb13e034486b56449ec13f 1.80MB / 1.80MB 0.3s done
#6 sha256:4e7e0215f4adc2c48ad9cb3b3781e21d474b477587f85682c2e2975ae91dce9d 27.20MB / 27.20MB 0.9s done
#6 extracting sha256:4e7e0215f4adc2c48ad9cb3b3781e21d474b477587f85682c2e2975ae91dce9d
#6 extracting sha256:4e7e0215f4adc2c48ad9cb3b3781e21d474b477587f85682c2e2975ae91dce9d 2.5s done
#6 extracting sha256:d6ec0ff8e83c114f957234a0898fc6a507348b7b69eb13e034486b56449ec13f
#6 extracting sha256:d6ec0ff8e83c114f957234a0898fc6a507348b7b69eb13e034486b56449ec13f 0.5s done
#6 extracting sha256:9f860bc38ec0cd69bbd3ef3eca48b40fe62c8db1de95cd38c07b202ceb339dd2
#6 extracting sha256:9f860bc38ec0cd69bbd3ef3eca48b40fe62c8db1de95cd38c07b202ceb339dd2 4.0s done
#6 extracting sha256:51aa5245f5c90da1e1d00b96f1be695c6627e081951aa9714827beb8ca566215
#6 extracting sha256:51aa5245f5c90da1e1d00b96f1be695c6627e081951aa9714827beb8ca566215 0.0s done
#6 extracting sha256:1ceaa0deba1f91d7e74aaeb7eae8735c30bffbda30cca819a8cfa0139362bce0
#6 extracting sha256:1ceaa0deba1f91d7e74aaeb7eae8735c30bffbda30cca819a8cfa0139362bce0 1.0s done
#6 extracting sha256:ccdad4bc939f28af8e65ea17cedb5038d7c60190106e6359d804bd06fe00b700 0.0s done
#6 extracting sha256:f09e3d0ad471ce95ed54a9c349ecacd27890c883bb8e95a2422e8bb026ef28fe 0.0s done
#6 extracting sha256:393fec3d83dae7e40db04617215e575bedd01ab8a0b6452b43e51ea9ce5ad165 0.0s done
#6 extracting sha256:b3051569353899b1f92064e3942a5694fc7775f8fd8e644cf12021be2ab5de30
#6 extracting sha256:b3051569353899b1f92064e3942a5694fc7775f8fd8e644cf12021be2ab5de30 0.0s done
#6 extracting sha256:d7b58f2a665d2108e75d1361fe38549159c5b8ef60671018b97499cbbec508ae
#6 extracting sha256:d7b58f2a665d2108e75d1361fe38549159c5b8ef60671018b97499cbbec508ae 1.1s done
#6 DONE 1.1s
#7 [linux/arm64 2/10] RUN rabbitmq-plugins enable --offline rabbitmq_shove...
#7 ...
#17 [linux/amd64 1/10] FROM docker.io/library/rabbitmq:3.11-management@sha2...
#17 extracting sha256:d5e775568c003580578d8757a1a3785c36c52424de3a5dc4a665eb5b2f392bdb 0.6s done
#17 extracting sha256:de99a1ffedd324479cb1f1ed33f971553d21b568ab1bc7429e06d779a34722a8 4.1s done
#17 extracting sha256:32e52a38037e5a39d190cdbd275aa4830c4a7b34af77ea3b0f01ba777875a324 0.0s done
#17 extracting sha256:0cfc7bb1390335bc5366c6a0e74be6ba0e29e0e509a3da7e80647a8768afc4ed 1.1s done
#17 extracting sha256:da2156e3127fb9d3e13f39f3f2f213d11b8d51f6ccadf2cec51b08997d3df45a 0.0s done
#17 extracting sha256:98b33de64338626964a040181538fb88c9767d1993018750e02631d99cb5d274 0.0s done
#17 extracting sha256:ed8f0defe00e7b3134ec8c7c13a9d296ad5e27affd1903f894109153bcf20036 0.0s done
#17 extracting sha256:715c1c578966eb68312919ec19789d5c5c5c62e643cfbaed23cf9b125f55a4eb 0.0s done
#17 extracting sha256:e9c0e835b826e5cc1ab0b795342ee081f3f6721ef21e478bc752a24ae6d165ca 1.2s done
#17 DONE 1.2s
#18 [linux/amd64 2/10] RUN rabbitmq-plugins enable --offline rabbitmq_shove...
#18 2.633 Enabling plugins on node rabbit@buildkitsandbox:
#18 2.635 rabbitmq_shovel
#18 3.742 The following plugins have been configured:
#18 3.744 rabbitmq_management
#18 3.744 rabbitmq_management_agent
#18 3.745 rabbitmq_prometheus
#18 3.746 rabbitmq_shovel
#18 3.747 rabbitmq_web_dispatch
#18 3.747 Applying plugin configuration to rabbit@buildkitsandbox...
#18 3.751 The following plugins have been enabled:
#18 3.752 rabbitmq_shovel
#18 3.753
#18 3.753 set 5 plugins.
#18 3.754 Offline change; changes will take effect at broker restart.
#18 DONE 5.0s
#7 [linux/arm64 2/10] RUN rabbitmq-plugins enable --offline rabbitmq_shove...
#7 2.689 Segmentation fault (core dumped)
#7 ERROR: process "/bin/sh -c rabbitmq-plugins enable --offline rabbitmq_shovel" did not complete successfully: exit code: 139
------
> [linux/arm64 2/10] RUN rabbitmq-plugins enable --offline rabbitmq_shovel:
------
failed to solve: rpc error: code = Unknown desc = process "/bin/sh -c rabbitmq-plugins enable --offline rabbitmq_shovel" did not complete successfully: exit code: 139
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: exit code 1 |
|
I also verified this happens with RabbitMQ 3.9, 3.10. The latest RabbitMQ it does NOT happen with is 3.8. Is there a workaround to build single-arch somehow? e.g. arm64/v8 only? |
|
@sagimann please enclose large amounts of log output in |
|
Closing due to lack of activity and lack of exact instructions for reproduction. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi, just tried building the following docker for arm64:
and it fails at the second command line execution:
The text was updated successfully, but these errors were encountered: