Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error cross building for riscv64 arch #4316

Closed
crazy-max opened this issue Oct 10, 2023 · 6 comments · Fixed by #4348
Closed

Error cross building for riscv64 arch #4316

crazy-max opened this issue Oct 10, 2023 · 6 comments · Fixed by #4348
Labels
kind/bug

Comments

@crazy-max
Copy link
Member

crazy-max commented Oct 10, 2023 •
edited

https://github.com/moby/buildkit/actions/runs/6463544011/job/17546872986?pr=4313

$ docker buildx bake --set *.platform=linux/riscv64
...
 > [linux/amd64->riscv64 runc 3/3] RUN --mount=from=runc-src,src=/usr/src/runc,target=. --mount=target=/root/.cache,type=cache   CGO_ENABLED=1 xx-go build -mod=vendor -ldflags '-extldflags -static' -tags 'apparmor seccomp netgo cgo static_build osusergo' -o /usr/bin/runc ./ &&   xx-verify --static /usr/bin/runc:
63.14 # github.com/opencontainers/runc
63.14 /usr/local/go/pkg/tool/linux_amd64/link: running riscv64-alpine-linux-musl-clang failed: exit status 1
63.14 clang-16: error: unable to execute command: Segmentation fault (core dumped)
63.14 clang-16: error: linker command failed due to signal (use -v to see invocation)
63.14

With -v:

$ docker buildx bake --set *.platform=linux/riscv64
...
> [runc 3/3] RUN --mount=from=runc-src,src=/usr/src/runc,target=. --mount=target=/root/.cache,type=cache   CGO_ENABLED=1 xx-go build -mod=vendor -ldflags '-v -extldflags -static' -tags 'apparmor seccomp netgo cgo static_build osusergo' -o /usr/bin/runc ./ &&   xx-verify --static /usr/bin/runc:
2.069 # github.com/opencontainers/runc
2.069 HEADER = -H5 -T0x11000 -R0x10000
2.069 host link: "riscv64-alpine-linux-musl-clang" "-o" "/tmp/go-build802415401/b001/exe/a.out" "-static" "-Qunused-arguments" "-Wl,--compress-debug-sections=zlib" "/tmp/go-link-2510713925/go.o" "/tmp/go-link-2510713925/000000.o" "/tmp/go-link-2510713925/000001.o" "/tmp/go-link-2510713925/000002.o" "/tmp/go-link-2510713925/000003.o" "/tmp/go-link-2510713925/000004.o" "/tmp/go-link-2510713925/000005.o" "/tmp/go-link-2510713925/000006.o" "/tmp/go-link-2510713925/000007.o" "/tmp/go-link-2510713925/000008.o" "/tmp/go-link-2510713925/000009.o" "/tmp/go-link-2510713925/000010.o" "/tmp/go-link-2510713925/000011.o" "/tmp/go-link-2510713925/000012.o" "/tmp/go-link-2510713925/000013.o" "/tmp/go-link-2510713925/000014.o" "/tmp/go-link-2510713925/000015.o" "/tmp/go-link-2510713925/000016.o" "/tmp/go-link-2510713925/000017.o" "/tmp/go-link-2510713925/000018.o" "/tmp/go-link-2510713925/000019.o" "/tmp/go-link-2510713925/000020.o" "-O2" "-g" "-O2" "-g" "-L/riscv64-alpine-linux-musl/usr/lib" "-lseccomp" "-O2" "-g" "-L/riscv64-alpine-linux-musl/usr/lib" "-lseccomp" "-O2" "-g" "-lpthread" "-no-pie" "-static"   
2.069 /usr/local/go/pkg/tool/linux_amd64/link: running riscv64-alpine-linux-musl-clang failed: exit status 1
2.069 clang-16: error: unable to execute command: Segmentation fault
2.069 clang-16: error: linker command failed due to signal (use -v to see invocation)
2.069

This looks similar to #3625 and happens since October 06: https://github.com/moby/buildkit/actions/workflows/buildkit.yml?query=event%3Aschedule

image

I tried with crazymax/xx:binutils-2.40 (tonistiigi/xx#121) using https://github.com/crazy-max/xx/releases/tag/prebuilt%2Fld-2.40-14 but no dice.

Only change I see that might be linked is:

(1/41) Upgrading musl (1.2.4-r1 -> 1.2.4-r2)
@crazy-max
Copy link
Member Author

crazy-max commented Oct 10, 2023 •
edited

I forgot that we were using edge repo when installing alpine packages through xx-apk: https://github.com/tonistiigi/xx/blob/d53654c2ac7058ec5290b6d051b9d0cc3a7d06b9/src/xx-apk#L49-L53

The actual diff is with the musl-dev package.

It works with: https://github.com/moby/buildkit/actions/runs/6417563430/job/17423592621#step:7:310

#23 0.092 fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz
#23 0.180 fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/community/x86_64/APKINDEX.tar.gz
#23 0.442 (1/40) Installing ncurses-terminfo-base (6.4_p20230506-r0)
#23 0.449 (2/40) Installing libncursesw (6.4_p20230506-r0)
#23 0.456 (3/40) Installing readline (8.2.1-r1)
#23 0.461 (4/40) Installing bash (5.2.15-r5)
#23 0.475 Executing bash-5.2.15-r5.post-install
#23 0.478 (5/40) Installing libffi (3.4.4-r2)
#23 0.481 (6/40) Installing libgcc (12.2.1_git20220924-r10)
#23 0.484 (7/40) Installing libstdc++ (12.2.1_git20220924-r10)
#23 0.502 (8/40) Installing xz-libs (5.4.3-r0)
#23 0.507 (9/40) Installing libxml2 (2.11.4-r0)
#23 0.528 (10/40) Installing zstd-libs (1.5.5-r4)
#23 0.538 (11/40) Installing llvm16-libs (16.0.6-r1)
#23 1.451 (12/40) Installing clang16-libs (16.0.6-r1)
#23 1.931 (13/40) Installing fortify-headers (1.1-r3)
#23 1.934 (14/40) Installing binutils (2.40-r7)
#23 1.996 (15/40) Installing libgomp (12.2.1_git20220924-r10)
#23 2.001 (16/40) Installing libatomic (12.2.1_git20220924-r10)
#23 2.004 (17/40) Installing gmp (6.2.1-r3)
#23 2.011 (18/40) Installing isl26 (0.26-r1)
#23 2.030 (19/40) Installing mpfr4 (4.2.0_p12-r0)
#23 2.040 (20/40) Installing mpc1 (1.3.1-r1)
#23 2.045 (21/40) Installing gcc (12.2.1_git20220924-r10)
#23 2.934 (22/40) Installing libstdc++-dev (12.2.1_git20220924-r10)
#23 3.219 (23/40) Installing clang16-libclang (16.0.6-r1)
#23 3.486 (24/40) Installing clang16 (16.0.6-r1)
#23 3.561 (25/40) Installing libmagic (5.45-r0)
#23 3.589 (26/40) Installing file (5.45-r0)
#23 3.600 (27/40) Installing brotli-libs (1.0.9-r14)
#23 3.609 (28/40) Installing libunistring (1.1-r1)
#23 3.624 (29/40) Installing libidn2 (2.3.4-r1)
#23 3.628 (30/40) Installing nghttp2-libs (1.55.1-r0)
#23 3.631 (31/40) Installing libcurl (8.3.0-r0)
#23 3.640 (32/40) Installing libexpat (2.5.0-r1)
#23 3.643 (33/40) Installing pcre2 (10.42-r1)
#23 3.652 (34/40) Installing git (2.40.1-r0)
#23 3.719 (35/40) Installing scudo-malloc (16.0.6-r1)
#23 3.723 (36/40) Installing lld-libs (16.0.6-r0)
#23 3.772 (37/40) Installing lld (16.0.6-r0)
#23 3.775 (38/40) Installing make (4.4.1-r1)
#23 3.781 (39/40) Installing musl-dev (1.2.4-r1)
#23 3.849 (40/40) Installing pkgconf (1.9.5-r0)
...
#35 0.707 + apk  --root /riscv64-alpine-linux-musl add musl-dev gcc libseccomp-dev libseccomp-static
#35 0.712 fetch https://dl-cdn.alpinelinux.org/alpine/edge/main/riscv64/APKINDEX.tar.gz
#35 0.811 fetch https://dl-cdn.alpinelinux.org/alpine/edge/community/riscv64/APKINDEX.tar.gz
#35 1.100 (1/20) Installing musl (1.2.4_git20230717-r2)
#35 1.109 (2/20) Installing jansson (2.14-r4)
#35 1.112 (3/20) Installing zlib (1.3-r0)
#35 1.115 (4/20) Installing zstd-libs (1.5.5-r7)
#35 1.127 (5/20) Installing binutils (2.41-r0)
#35 1.190 (6/20) Installing libgomp (13.1.1_git20230722-r2)
#35 1.196 (7/20) Installing libatomic (13.1.1_git20230722-r2)
#35 1.201 (8/20) Installing libgcc (13.1.1_git20230722-r2)
#35 1.205 (9/20) Installing gmp (6.3.0-r0)
#35 1.212 (10/20) Installing isl26 (0.26-r1)
#35 1.228 (11/20) Installing mpfr4 (4.2.1-r0)
#35 1.235 (12/20) Installing mpc1 (1.3.1-r1)
#35 1.238 (13/20) Installing libstdc++ (13.1.1_git20230722-r2)
#35 1.257 (14/20) Installing gcc (13.1.1_git20230722-r2)
#35 2.011 (15/20) Installing linux-headers (6.5-r0)
#35 2.108 (16/20) Installing libseccomp (2.5.4-r3)
#35 2.112 (17/20) Installing pkgconf (2.0.3-r0)
#35 2.116 (18/20) Installing libseccomp-dev (2.5.4-r3)
#35 2.119 (19/20) Installing libseccomp-static (2.5.4-r3)
#35 2.125 (20/20) Installing musl-dev (1.2.4_git20230717-r2)

#35 2.125 (20/20) Installing musl-dev (1.2.4_git20230717-r2)

But fails with: https://github.com/moby/buildkit/actions/runs/6430248332/job/17460882400#step:7:258

#26 0.104 fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz
#26 0.208 fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/community/x86_64/APKINDEX.tar.gz
#26 0.491 (1/41) Upgrading musl (1.2.4-r1 -> 1.2.4-r2)
#26 0.504 (2/41) Installing ncurses-terminfo-base (6.4_p20230506-r0)
#26 0.510 (3/41) Installing libncursesw (6.4_p20230506-r0)
#26 0.518 (4/41) Installing readline (8.2.1-r1)
#26 0.524 (5/41) Installing bash (5.2.15-r5)
#26 0.541 Executing bash-5.2.15-r5.post-install
#26 0.544 (6/41) Installing libffi (3.4.4-r2)
#26 0.548 (7/41) Installing libgcc (12.2.1_git20220924-r10)
#26 0.551 (8/41) Installing libstdc++ (12.2.1_git20220924-r10)
#26 0.589 (9/41) Installing xz-libs (5.4.3-r0)
#26 0.595 (10/41) Installing libxml2 (2.11.4-r0)
#26 0.610 (11/41) Installing zstd-libs (1.5.5-r4)
#26 0.622 (12/41) Installing llvm16-libs (16.0.6-r1)
#26 1.662 (13/41) Installing clang16-libs (16.0.6-r1)
#26 2.187 (14/41) Installing fortify-headers (1.1-r3)
#26 2.190 (15/41) Installing binutils (2.40-r7)
#26 2.270 (16/41) Installing libgomp (12.2.1_git20220924-r10)
#26 2.277 (17/41) Installing libatomic (12.2.1_git20220924-r10)
#26 2.280 (18/41) Installing gmp (6.2.1-r3)
#26 2.288 (19/41) Installing isl26 (0.26-r1)
#26 2.312 (20/41) Installing mpfr4 (4.2.0_p12-r0)
#26 2.323 (21/41) Installing mpc1 (1.3.1-r1)
#26 2.332 (22/41) Installing gcc (12.2.1_git20220924-r10)
#26 3.373 (23/41) Installing libstdc++-dev (12.2.1_git20220924-r10)
#26 3.732 (24/41) Installing clang16-libclang (16.0.6-r1)
#26 4.032 (25/41) Installing clang16 (16.0.6-r1)
#26 4.119 (26/41) Installing libmagic (5.45-r0)
#26 4.153 (27/41) Installing file (5.45-r0)
#26 4.155 (28/41) Installing brotli-libs (1.0.9-r14)
#26 4.168 (29/41) Installing libunistring (1.1-r1)
#26 4.190 (30/41) Installing libidn2 (2.3.4-r1)
#26 4.196 (31/41) Installing nghttp2-libs (1.55.1-r0)
#26 4.201 (32/41) Installing libcurl (8.3.0-r0)
#26 4.213 (33/41) Installing libexpat (2.5.0-r1)
#26 4.218 (34/41) Installing pcre2 (10.42-r1)
#26 4.228 (35/41) Installing git (2.40.1-r0)
#26 4.303 (36/41) Installing scudo-malloc (16.0.6-r1)
#26 4.309 (37/41) Installing lld-libs (16.0.6-r0)
#26 4.365 (38/41) Installing lld (16.0.6-r0)
#26 4.370 (39/41) Installing make (4.4.1-r1)
#26 4.376 (40/41) Installing musl-dev (1.2.4-r2)
#26 4.455 (41/41) Installing pkgconf (1.9.5-r0)
...
#32 0.857 + apk  --root /riscv64-alpine-linux-musl add musl-dev gcc libseccomp-dev libseccomp-static
#32 0.863 fetch https://dl-cdn.alpinelinux.org/alpine/edge/main/riscv64/APKINDEX.tar.gz
#32 0.980 fetch https://dl-cdn.alpinelinux.org/alpine/edge/community/riscv64/APKINDEX.tar.gz
#32 1.387 (1/20) Installing musl (1.2.4_git20230717-r3)
#32 1.402 (2/20) Installing jansson (2.14-r4)
#32 1.407 (3/20) Installing zlib (1.3-r0)
#32 1.414 (4/20) Installing zstd-libs (1.5.5-r7)
#32 1.434 (5/20) Installing binutils (2.41-r0)
#32 1.538 (6/20) Installing libgomp (13.1.1_git20230722-r2)
#32 1.546 (7/20) Installing libatomic (13.1.1_git20230722-r2)
#32 1.551 (8/20) Installing libgcc (13.1.1_git20230722-r2)
#32 1.556 (9/20) Installing gmp (6.3.0-r0)
#32 1.566 (10/20) Installing isl26 (0.26-r1)
#32 1.594 (11/20) Installing mpfr4 (4.2.1-r0)
#32 1.606 (12/20) Installing mpc1 (1.3.1-r1)
#32 1.611 (13/20) Installing libstdc++ (13.1.1_git20230722-r2)
#32 1.645 (14/20) Installing gcc (13.1.1_git20230722-r2)
#32 2.836 (15/20) Installing linux-headers (6.5-r0)
#32 3.016 (16/20) Installing libseccomp (2.5.4-r3)
#32 3.023 (17/20) Installing pkgconf (2.0.3-r0)
#32 3.030 (18/20) Installing libseccomp-dev (2.5.4-r3)
#32 3.034 (19/20) Installing libseccomp-static (2.5.4-r3)
#32 3.042 (20/20) Installing musl-dev (1.2.4_git20230717-r3)

#32 3.042 (20/20) Installing musl-dev (1.2.4_git20230717-r3)

Probably related to this change: alpinelinux/aports@a22cf87

@crazy-max crazy-max mentioned this issue Oct 11, 2023
Merged
@crazy-max
Copy link
Member Author

Opened an issue on alpine tracker: https://gitlab.alpinelinux.org/alpine/aports/-/issues/15350

@crazy-max
Copy link
Member Author

Added a repro with a simple project here: https://github.com/crazy-max/alpine-edge-musl-segfault/tree/main/jq

@jedevc jedevc mentioned this issue Oct 13, 2023
Merged
This was referenced Oct 13, 2023
Closed
Merged
@crazy-max
Copy link
Member Author

crazy-max commented Oct 16, 2023 •
edited 

$ docker run --rm -it alpine:edge sh
> cd /opt
> apk add git clang llvm lld file
> git clone https://github.com/tonistiigi/xx
> cp ./xx/src/xx-* /usr/bin/
> ln -s /usr/bin/xx-cc /usr/bin/xx-clang

> TARGETPLATFORM=linux/arm/v7 xx-apk add musl-dev gcc
> TARGETPLATFORM=linux/arm/v7 xx-clang --setup-target-triple
> armv7-alpine-linux-musleabihf-clang
clang-16: error: no input files

> TARGETPLATFORM=linux/riscv64 xx-apk add musl-dev gcc
> TARGETPLATFORM=linux/riscv64 xx-clang --setup-target-triple
> riscv64-alpine-linux-musl-clang
/usr/bin/riscv64-alpine-linux-musl-ld: /usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3c
/riscv64-alpine-linux-musl/usr/lib/Scrt1.o: in function `.L0 ':
Scrt1.c:(.text._start_c+0x18): undefined reference to `main'
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)

> riscv64-alpine-linux-musl-clang -v
Alpine clang version 16.0.6
Target: riscv64-alpine-linux-musl
Thread model: posix
InstalledDir: /usr/bin
Configuration file: /usr/lib/llvm16/bin/riscv64-alpine-linux-musl.cfg
System configuration file directory: /etc/clang16
Found candidate GCC installation: /riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1
Selected GCC installation: /riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1
 "/usr/bin/riscv64-alpine-linux-musl-ld" --sysroot=/riscv64-alpine-linux-musl/ -pie -z now -z relro --hash-style=gnu --build-id --eh-frame-hdr -m elf64lriscv -X -dynamic-linker /lib/ld-musl-riscv64.so.1 -o a.out /riscv64-alpine-linux-musl/usr/lib/Scrt1.o /riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1/crti.o /riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1/crtbeginS.o -L/riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1 -L/riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1/../../../../riscv64-alpine-linux-musl/lib -L/riscv64-alpine-linux-musl/lib -L/riscv64-alpine-linux-musl/usr/lib --as-needed -rpath-link /riscv64-alpine-linux-musl/usr/lib -lssp_nonshared -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1/crtendS.o /riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1/crtn.o
/usr/bin/riscv64-alpine-linux-musl-ld: /usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3c
/riscv64-alpine-linux-musl/usr/lib/Scrt1.o: in function `.L0 ':
Scrt1.c:(.text._start_c+0x18): undefined reference to `main'
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
/usr/bin/riscv64-alpine-linux-musl-ld: /riscv64-alpine-linux-musl/usr/lib/Scrt1.o: unsupported relocation type 0x3d
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)

This was referenced Oct 16, 2023
Merged
Merged
@crazy-max
Copy link
Member Author

Built musl-dev-1.2.4_git20230717-r2 on my sifive mb https://github.com/crazy-max/alpine-edge-musl-segfault/tree/main/aports-musl but it still segfault. You can check this target and invoke docker buildx bake build-riscv64 in jq folder. Here are the logs:

#15 [build-riscv64 2/8] RUN apk add --no-cache file bash clang lld musl-dev pkgconfig git make go
#15 0.031 fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/aarch64/APKINDEX.tar.gz
#15 0.152 fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/community/aarch64/APKINDEX.tar.gz
#15 0.283 (1/44) Upgrading musl (1.2.4-r1 -> 1.2.4-r2)
#15 0.301 (2/44) Installing ncurses-terminfo-base (6.4_p20230506-r0)
#15 0.316 (3/44) Installing libncursesw (6.4_p20230506-r0)
#15 0.328 (4/44) Installing readline (8.2.1-r1)
#15 0.339 (5/44) Installing bash (5.2.15-r5)
#15 0.361 Executing bash-5.2.15-r5.post-install
#15 0.363 (6/44) Installing libffi (3.4.4-r2)
#15 0.372 (7/44) Installing libgcc (12.2.1_git20220924-r10)
#15 0.380 (8/44) Installing libstdc++ (12.2.1_git20220924-r10)
#15 0.399 (9/44) Installing xz-libs (5.4.3-r0)
#15 0.409 (10/44) Installing libxml2 (2.11.4-r0)
#15 0.425 (11/44) Installing zstd-libs (1.5.5-r4)
#15 0.441 (12/44) Installing llvm16-libs (16.0.6-r1)
#15 1.020 (13/44) Installing clang16-libs (16.0.6-r1)
#15 1.323 (14/44) Installing fortify-headers (1.1-r3)
#15 1.332 (15/44) Installing binutils (2.40-r7)
#15 1.402 (16/44) Installing libgomp (12.2.1_git20220924-r10)
#15 1.413 (17/44) Installing libatomic (12.2.1_git20220924-r10)
#15 1.421 (18/44) Installing gmp (6.2.1-r3)
#15 1.436 (19/44) Installing isl26 (0.26-r1)
#15 1.457 (20/44) Installing mpfr4 (4.2.0_p12-r0)
#15 1.469 (21/44) Installing mpc1 (1.3.1-r1)
#15 1.479 (22/44) Installing gcc (12.2.1_git20220924-r10)
#15 2.163 (23/44) Installing libstdc++-dev (12.2.1_git20220924-r10)
#15 2.490 (24/44) Installing clang16-libclang (16.0.6-r1)
#15 2.696 (25/44) Installing clang16 (16.0.6-r1)
#15 2.772 (26/44) Installing libmagic (5.45-r0)
#15 2.806 (27/44) Installing file (5.45-r0)
#15 2.816 (28/44) Installing ca-certificates (20230506-r0)
#15 2.838 (29/44) Installing brotli-libs (1.0.9-r14)
#15 2.853 (30/44) Installing libunistring (1.1-r1)
#15 2.871 (31/44) Installing libidn2 (2.3.4-r1)
#15 2.880 (32/44) Installing nghttp2-libs (1.57.0-r0)
#15 2.889 (33/44) Installing libcurl (8.4.0-r0)
#15 2.905 (34/44) Installing libexpat (2.5.0-r1)
#15 2.916 (35/44) Installing pcre2 (10.42-r1)
#15 2.931 (36/44) Installing git (2.40.1-r0)
#15 2.988 (37/44) Installing binutils-gold (2.40-r7)
#15 3.016 (38/44) Installing go (1.20.10-r0)
#15 4.074 (39/44) Installing scudo-malloc (16.0.6-r1)
#15 4.084 (40/44) Installing lld-libs (16.0.6-r0)
#15 4.126 (41/44) Installing lld (16.0.6-r0)
#15 4.137 (42/44) Installing make (4.4.1-r1)
#15 4.147 (43/44) Installing musl-dev (1.2.4-r2)
#15 4.216 (44/44) Installing pkgconf (1.9.5-r0)
#15 4.231 Executing busybox-1.36.1-r2.trigger
#15 4.236 Executing ca-certificates-20230506-r0.trigger
#15 4.267 OK: 643 MiB in 58 packages
#15 DONE 4.8s

#16 [build-riscv64 3/8] COPY --link --from=tonistiigi/xx:1.2.1 / /
#16 merging done
#16 DONE 0.1s

#17 [build-riscv64 4/8] COPY --from=alpine-musl / /root/packages
#17 DONE 0.0s

#18 [build-riscv64 5/8] RUN xx-apk add --allow-untrusted /root/packages/main/riscv64/*.apk
#18 0.037 fetch https://dl-cdn.alpinelinux.org/alpine/edge/main/riscv64/APKINDEX.tar.gz
#18 0.135 fetch https://dl-cdn.alpinelinux.org/alpine/edge/community/riscv64/APKINDEX.tar.gz
#18 0.283 (1/1) Installing alpine-keys (2.4-r1)
#18 0.296 OK: 0 MiB in 1 packages
#18 0.320 + apk  --root /riscv64-alpine-linux-musl add --allow-untrusted /root/packages/main/riscv64/musl-1.2.4_git20230717-r2.apk /root/packages/main/riscv64/musl-dbg-1.2.4_git20230717-r2.apk /root/packages/main/riscv64/musl-dev-1.2.4_git20230717-r2.apk /root/packages/main/riscv64/musl-libintl-1.2.4_git20230717-r2.apk /root/packages/main/riscv64/musl-utils-1.2.4_git20230717-r2.apk
#18 0.323 fetch https://dl-cdn.alpinelinux.org/alpine/edge/main/riscv64/APKINDEX.tar.gz
#18 0.404 fetch https://dl-cdn.alpinelinux.org/alpine/edge/community/riscv64/APKINDEX.tar.gz
#18 0.607 (1/6) Installing musl (1.2.4_git20230717-r2)
#18 0.611 (2/6) Installing musl-dbg (1.2.4_git20230717-r2)
#18 0.624 (3/6) Installing musl-dev (1.2.4_git20230717-r2)
#18 0.709 (4/6) Installing musl-libintl (1.2.4_git20230717-r2)
#18 0.709 (5/6) Installing scanelf (1.3.7-r2)
#18 0.719 (6/6) Installing musl-utils (1.2.4_git20230717-r2)
#18 0.720 OK: 29 MiB in 7 packages
#18 DONE 0.8s

#19 [build-riscv64 6/8] RUN xx-apk add gcc jq-dev oniguruma-dev
#19 0.042 + apk  --root /riscv64-alpine-linux-musl add gcc jq-dev oniguruma-dev
#19 0.205 (1/18) Installing jansson (2.14-r4)
#19 0.245 (2/18) Installing zlib (1.3-r0)
#19 0.259 (3/18) Installing zstd-libs (1.5.5-r7)
#19 0.284 (4/18) Installing binutils (2.41-r0)
#19 0.348 (5/18) Installing libgomp (13.1.1_git20230722-r3)
#19 0.360 (6/18) Installing libatomic (13.1.1_git20230722-r3)
#19 0.368 (7/18) Installing libgcc (13.1.1_git20230722-r3)
#19 0.377 (8/18) Installing gmp (6.3.0-r0)
#19 0.392 (9/18) Installing isl26 (0.26-r1)
#19 0.409 (10/18) Installing mpfr4 (4.2.1-r0)
#19 0.421 (11/18) Installing mpc1 (1.3.1-r1)
#19 0.430 (12/18) Installing libstdc++ (13.1.1_git20230722-r3)
#19 0.452 (13/18) Installing gcc (13.1.1_git20230722-r3)
#19 0.990 (14/18) Installing oniguruma (6.9.9-r0)
#19 1.024 (15/18) Installing jq (1.7-r1)
#19 1.036 (16/18) Installing pkgconf (2.0.3-r0)
#19 1.047 (17/18) Installing jq-dev (1.7-r1)
#19 1.064 (18/18) Installing oniguruma-dev (6.9.9-r0)
#19 1.079 OK: 142 MiB in 25 packages
#19 DONE 1.2s

#20 [build-riscv64 7/8] WORKDIR /src
#20 DONE 0.0s

#21 [build-riscv64 8/8] RUN --mount=type=bind,source=.,rw   CGO_ENABLED=1 xx-go build -ldflags '-v -extldflags -static' -o ./jq &&   file ./jq
#21 3.989 # jq
#21 3.989 HEADER = -H5 -T0x11000 -R0x10000
#21 3.989 host link: "riscv64-alpine-linux-musl-clang" "-o" "/tmp/go-build3231769279/b001/exe/a.out" "-static" "-Qunused-arguments" "-Wl,--compress-debug-sections=zlib" "/tmp/go-link-1721160904/go.o" "/tmp/go-link-1721160904/000000.o" "/tmp/go-link-1721160904/000001.o" "/tmp/go-link-1721160904/000002.o" "/tmp/go-link-1721160904/000003.o" "/tmp/go-link-1721160904/000004.o" "/tmp/go-link-1721160904/000005.o" "/tmp/go-link-1721160904/000006.o" "/tmp/go-link-1721160904/000007.o" "/tmp/go-link-1721160904/000008.o" "/tmp/go-link-1721160904/000009.o" "/tmp/go-link-1721160904/000010.o" "/tmp/go-link-1721160904/000011.o" "/tmp/go-link-1721160904/000012.o" "/tmp/go-link-1721160904/000013.o" "-O2" "-g" "-L" "/src" "-ljq" "-lonig" "-O2" "-g" "-lpthread" "-no-pie" "-static"
#21 3.989 /usr/lib/go/pkg/tool/linux_arm64/link: running riscv64-alpine-linux-musl-clang failed: exit status 1
#21 3.989 clang-16: error: unable to execute command: Segmentation fault
#21 3.989 clang-16: error: linker command failed due to signal (use -v to see invocation)
#21 3.989
#21 ERROR: process "/bin/sh -c CGO_ENABLED=1 xx-go build -ldflags '-v -extldflags -static' -o ./jq &&   file ./jq" did not complete successfully: exit code: 1
------
 > [build-riscv64 8/8] RUN --mount=type=bind,source=.,rw   CGO_ENABLED=1 xx-go build -ldflags '-v -extldflags -static' -o ./jq &&   file ./jq:
3.989 # jq
3.989 HEADER = -H5 -T0x11000 -R0x10000
3.989 host link: "riscv64-alpine-linux-musl-clang" "-o" "/tmp/go-build3231769279/b001/exe/a.out" "-static" "-Qunused-arguments" "-Wl,--compress-debug-sections=zlib" "/tmp/go-link-1721160904/go.o" "/tmp/go-link-1721160904/000000.o" "/tmp/go-link-1721160904/000001.o" "/tmp/go-link-1721160904/000002.o" "/tmp/go-link-1721160904/000003.o" "/tmp/go-link-1721160904/000004.o" "/tmp/go-link-1721160904/000005.o" "/tmp/go-link-1721160904/000006.o" "/tmp/go-link-1721160904/000007.o" "/tmp/go-link-1721160904/000008.o" "/tmp/go-link-1721160904/000009.o" "/tmp/go-link-1721160904/000010.o" "/tmp/go-link-1721160904/000011.o" "/tmp/go-link-1721160904/000012.o" "/tmp/go-link-1721160904/000013.o" "-O2" "-g" "-L" "/src" "-ljq" "-lonig" "-O2" "-g" "-lpthread" "-no-pie" "-static"
3.989 /usr/lib/go/pkg/tool/linux_arm64/link: running riscv64-alpine-linux-musl-clang failed: exit status 1
3.989 clang-16: error: unable to execute command: Segmentation fault
3.989 clang-16: error: linker command failed due to signal (use -v to see invocation)
3.989
------
Dockerfile:12
--------------------
  11 |     WORKDIR /src
  12 | >>> RUN --mount=type=bind,source=.,rw \
  13 | >>>   CGO_ENABLED=1 xx-go build -ldflags '-v -extldflags -static' -o ./jq && \
  14 | >>>   file ./jq
  15 |
--------------------
ERROR: failed to solve: process "/bin/sh -c CGO_ENABLED=1 xx-go build -ldflags '-v -extldflags -static' -o ./jq &&   file ./jq" did not complete successfully: exit code: 1

With -v:

#18 [stage-3 6/6] RUN --mount=type=bind,source=.,rw   CGO_ENABLED=1 xx-go build -ldflags '-extldflags "-v -static"' -o ./jq &&   file ./jq
#18 3.507 # jq
#18 3.507 /usr/lib/go/pkg/tool/linux_arm64/link: running riscv64-alpine-linux-musl-clang failed: exit status 1
#18 3.507 Alpine clang version 16.0.6
#18 3.507 Target: riscv64-alpine-linux-musl
#18 3.507 Thread model: posix
#18 3.507 InstalledDir: /usr/bin
#18 3.507 Configuration file: /usr/lib/llvm16/bin/riscv64-alpine-linux-musl.cfg
#18 3.507 Found candidate GCC installation: /riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1
#18 3.507 Selected GCC installation: /riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1
#18 3.507  "/usr/bin/riscv64-alpine-linux-musl-ld" --sysroot=/riscv64-alpine-linux-musl/ -z now -z relro --hash-style=gnu --build-id --eh-frame-hdr -m elf64lriscv -X -static -o $WORK/b001/exe/a.out /riscv64-alpine-linux-musl/usr/lib/crt1.o /riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1/crti.o /riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1/crtbeginT.o -L/src -L/riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1 -L/riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1/../../../../riscv64-alpine-linux-musl/lib -L/riscv64-alpine-linux-musl/lib -L/riscv64-alpine-linux-musl/usr/lib --as-needed -rpath-link /riscv64-alpine-linux-musl/usr/lib --compress-debug-sections=zlib /tmp/go-link-2675570171/go.o /tmp/go-link-2675570171/000000.o /tmp/go-link-2675570171/000001.o /tmp/go-link-2675570171/000002.o /tmp/go-link-2675570171/000003.o /tmp/go-link-2675570171/000004.o /tmp/go-link-2675570171/000005.o /tmp/go-link-2675570171/000006.o /tmp/go-link-2675570171/000007.o /tmp/go-link-2675570171/000008.o /tmp/go-link-2675570171/000009.o /tmp/go-link-2675570171/000010.o /tmp/go-link-2675570171/000011.o /tmp/go-link-2675570171/000012.o /tmp/go-link-2675570171/000013.o -ljq -lonig -lpthread -lssp_nonshared --start-group -lgcc -lgcc_eh -lc --end-group /riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1/crtend.o /riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1/crtn.o
#18 3.507 clang-16: error: unable to execute command: Segmentation fault
#18 3.507 clang-16: error: linker command failed due to signal (use -v to see invocation)
#18 3.507
#18 ERROR: process "/bin/sh -c CGO_ENABLED=1 xx-go build -ldflags '-extldflags \"-v -static\"' -o ./jq &&   file ./jq" did not complete successfully: exit code: 1
------
 > [stage-3 6/6] RUN --mount=type=bind,source=.,rw   CGO_ENABLED=1 xx-go build -ldflags '-extldflags "-v -static"' -o ./jq &&   file ./jq:
3.507 Target: riscv64-alpine-linux-musl
3.507 Thread model: posix
3.507 InstalledDir: /usr/bin
3.507 Configuration file: /usr/lib/llvm16/bin/riscv64-alpine-linux-musl.cfg
3.507 Found candidate GCC installation: /riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1
3.507 Selected GCC installation: /riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1
3.507  "/usr/bin/riscv64-alpine-linux-musl-ld" --sysroot=/riscv64-alpine-linux-musl/ -z now -z relro --hash-style=gnu --build-id --eh-frame-hdr -m elf64lriscv -X -static -o $WORK/b001/exe/a.out /riscv64-alpine-linux-musl/usr/lib/crt1.o /riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1/crti.o /riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1/crtbeginT.o -L/src -L/riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1 -L/riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1/../../../../riscv64-alpine-linux-musl/lib -L/riscv64-alpine-linux-musl/lib -L/riscv64-alpine-linux-musl/usr/lib --as-needed -rpath-link /riscv64-alpine-linux-musl/usr/lib --compress-debug-sections=zlib /tmp/go-link-2675570171/go.o /tmp/go-link-2675570171/000000.o /tmp/go-link-2675570171/000001.o /tmp/go-link-2675570171/000002.o /tmp/go-link-2675570171/000003.o /tmp/go-link-2675570171/000004.o /tmp/go-link-2675570171/000005.o /tmp/go-link-2675570171/000006.o /tmp/go-link-2675570171/000007.o /tmp/go-link-2675570171/000008.o /tmp/go-link-2675570171/000009.o /tmp/go-link-2675570171/000010.o /tmp/go-link-2675570171/000011.o /tmp/go-link-2675570171/000012.o /tmp/go-link-2675570171/000013.o -ljq -lonig -lpthread -lssp_nonshared --start-group -lgcc -lgcc_eh -lc --end-group /riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1/crtend.o /riscv64-alpine-linux-musl/usr/lib/gcc/riscv64-alpine-linux-musl/13.1.1/crtn.o
3.507 clang-16: error: unable to execute command: Segmentation fault
3.507 clang-16: error: linker command failed due to signal (use -v to see invocation)
3.507
------
Dockerfile:35
--------------------
  34 |     WORKDIR /src
  35 | >>> RUN --mount=type=bind,source=.,rw \
  36 | >>>   CGO_ENABLED=1 xx-go build -ldflags '-extldflags "-v -static"' -o ./jq && \
  37 | >>>   file ./jq
  38 |
--------------------
ERROR: failed to solve: process "/bin/sh -c CGO_ENABLED=1 xx-go build -ldflags '-extldflags \"-v -static\"' -o ./jq &&   file ./jq" did not complete successfully: exit code: 1

So this might not be musl but ld (/usr/bin/riscv64-alpine-linux-musl-ld).

Opened tonistiigi/xx#127 to build binutils 2.41 and see if it solves this issue on our side (lurking at alpinelinux/aports@ed1730d#diff-42a05e20f42ec112988cfb8d78c1a0797d882c7fcf503b1287bdc6729c407aebL17)

@crazy-max
Copy link
Member Author

Another project looks to have the same issue and on their side rolling back to musl-dev-1.2.4_git20230717-r2 seems to solve the issue: https://gitlab.alpinelinux.org/alpine/aports/-/issues/15350#note_345819

@crazy-max crazy-max mentioned this issue Oct 18, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix riscv64 build crazy-max/buildkit
1 participant
@crazy-max