Build failure: `pkgsStatic.perl` (Perl interpreter statically linked to Musl libc) #295608

YoshiRulz · 2024-03-13T13:05:53Z

Steps To Reproduce

Steps to reproduce the behavior:

e.g. nix build -I nixpkgs=flake:github:NixOS/nixpkgs/23.11 nixpkgs#pkgsStatic.perl538

It still affects master as of 2024-03-12. edit: Changed title for clarity; pkgsMusl.perl is fine.

Build log

tail of build log

make -C dist/threads-shared/ PERL_CORE=1 LIBPERL=libperl.a LINKTYPE=static static
make[2]: Entering directory '/build/perl-5.38.2/dist/threads-shared'
"../../miniperl_top" "-I../../lib" "../../lib/ExtUtils/xsubpp"  -typemap '/build/perl-5.38.2/dist/threads-shared/../../lib/ExtUtils/typemap'  shared.xs > shared.xsc
mv shared.xsc shared.c
x86_64-unknown-linux-musl-gcc -c   -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -Wall -Wno-use-after-free -fwrapv -fno-strict-aliasing   -DVERSION=\"1.68\" -DXS_VERSION=\"1.68\" -fPIC -Wno-unused-function "-I../.."   shared.c
In file included from /nix/store/5m5zr8z4m8wx4v2c6w3fwcg07igrwzjy-libxcrypt-static-x86_64-unknown-linux-musl-4.4.36/include/crypt.h:22,
                 from ../../reentr.h:124,
                 from ../../op.h:700,
                 from ../../perl.h:4542,
                 from shared.xs:131:
/nix/store/0nfn0979vnb1w48rzm5mkx9zvbrkmxpp-x86_64-unknown-linux-musl-gcc-13.2.0/x86_64-unknown-linux-musl/sys-include/sys/cdefs.h:1:2: warning: #warning usage of non-standard #include <sys/cdefs.h> is deprecated [-Wcpp]
    1 | #warning usage of non-standard #include <sys/cdefs.h> is deprecated
      |  ^~~~~~~
In file included from ../../perl.h:7816:
../../sv_inline.h:566:1: warning: 'always_inline' function might not be inlinable [-Wattributes]
  566 | Perl_SvPVXtrue(pTHX_ SV *sv)
      | ^~~~~~~~~~~~~~
../../sv_inline.h:536:1: warning: 'always_inline' function might not be inlinable [-Wattributes]
  536 | Perl_newSV_type_mortal(pTHX_ const svtype type)
      | ^~~~~~~~~~~~~~~~~~~~~~
rm -f "../../lib/auto/threads/shared/shared.a"
x86_64-unknown-linux-musl-ar cr "../../lib/auto/threads/shared/shared.a" shared.o
x86_64-unknown-linux-musl-ranlib "../../lib/auto/threads/shared/shared.a"
chmod 755 ../../lib/auto/threads/shared/shared.a
make[2]: Leaving directory '/build/perl-5.38.2/dist/threads-shared'
./miniperl_top statars B Devel::Peek Fcntl File::DosGlob File::Glob Hash::Util Hash::Util::FieldHash I18N::Langinfo Opcode POSIX PerlIO::encoding PerlIO::mmap PerlIO::scalar PerlIO::via SDBM_File Sys::Hostname XS::APItest XS::Typemap attributes mro re Compress::Raw::Bzip2 Compress::Raw::Zlib Digest::MD5 Digest::SHA Encode Filter::Util::Call IPC::SysV MIME::Base64 Math::BigInt::FastCalc List::Util Socket Sys::Syslog Time::Piece Unicode::Collate Data::Dumper Devel::PPPort IO Cwd Storable Time::HiRes Unicode::Normalize threads threads::shared > static.list
./miniperl_top extlibs B Devel::Peek Fcntl File::DosGlob File::Glob Hash::Util Hash::Util::FieldHash I18N::Langinfo Opcode POSIX PerlIO::encoding PerlIO::mmap PerlIO::scalar PerlIO::via SDBM_File Sys::Hostname XS::APItest XS::Typemap attributes mro re Compress::Raw::Bzip2 Compress::Raw::Zlib Digest::MD5 Digest::SHA Encode Filter::Util::Call IPC::SysV MIME::Base64 Math::BigInt::FastCalc List::Util Socket Sys::Syslog Time::Piece Unicode::Collate Data::Dumper Devel::PPPort IO Cwd Storable Time::HiRes Unicode::Normalize threads threads::shared > ext.libs
x86_64-unknown-linux-musl-gcc -Wl,-E -o perl perlmain.o libperl.a lib/auto/B/B.a lib/auto/Devel/Peek/Peek.a lib/auto/Fcntl/Fcntl.a lib/auto/File/DosGlob/DosGlob.a lib/auto/File/Glob/Glob.a lib/auto/Hash/Util/Util.a lib/auto/Hash/Util/FieldHash/FieldHash.a lib/auto/I18N/Langinfo/Langinfo.a lib/auto/Opcode/Opcode.a lib/auto/POSIX/POSIX.a lib/auto/PerlIO/encoding/encoding.a lib/auto/PerlIO/mmap/mmap.a lib/auto/PerlIO/scalar/scalar.a lib/auto/PerlIO/via/via.a lib/auto/SDBM_File/SDBM_File.a lib/auto/Sys/Hostname/Hostname.a lib/auto/XS/APItest/APItest.a lib/auto/XS/Typemap/Typemap.a lib/auto/attributes/attributes.a lib/auto/mro/mro.a lib/auto/re/re.a lib/auto/Compress/Raw/Bzip2/Bzip2.a lib/auto/Compress/Raw/Zlib/Zlib.a lib/auto/Digest/MD5/MD5.a lib/auto/Digest/SHA/SHA.a lib/auto/Encode/Encode.a lib/auto/Filter/Util/Call/Call.a lib/auto/IPC/SysV/SysV.a lib/auto/MIME/Base64/Base64.a lib/auto/Math/BigInt/FastCalc/FastCalc.a lib/auto/List/Util/Util.a lib/auto/Socket/Socket.a lib/auto/Sys/Syslog/Syslog.a lib/auto/Time/Piece/Piece.a lib/auto/Unicode/Collate/Collate.a lib/auto/Data/Dumper/Dumper.a lib/auto/Devel/PPPort/PPPort.a lib/auto/IO/IO.a lib/auto/Cwd/Cwd.a lib/auto/Storable/Storable.a lib/auto/Time/HiRes/HiRes.a lib/auto/Unicode/Normalize/Normalize.a lib/auto/threads/threads.a lib/auto/threads/shared/shared.a -lm -lcrypt -lpthread -ldl -lm -L/nix/store/ljqvlm46wa0v3r39kii8k2gzwk6jxybs-zlib-static-x86_64-unknown-linux-musl-1.3.1/lib -lz -lrt -lpthread
/nix/store/sqii1hjj7rr1d5c5wwxlswqrv39nispr-x86_64-unknown-linux-musl-binutils-2.41/bin/x86_64-unknown-linux-musl-ld: lib/auto/re/re.a(re_comp.o): in function `Perl_reg_add_data':
re_comp.c:(.text+0xa390): multiple definition of `Perl_reg_add_data'; libperl.a(regcomp.o):regcomp.c:(.text+0x70a0): first defined here
/nix/store/sqii1hjj7rr1d5c5wwxlswqrv39nispr-x86_64-unknown-linux-musl-binutils-2.41/bin/x86_64-unknown-linux-musl-ld: lib/auto/re/re.a(re_comp.o): in function `Perl_populate_anyof_bitmap_from_invlist':
re_comp.c:(.text+0xa8e0): multiple definition of `Perl_populate_anyof_bitmap_from_invlist'; libperl.a(regcomp.o):regcomp.c:(.text+0x7b60): first defined here
/nix/store/sqii1hjj7rr1d5c5wwxlswqrv39nispr-x86_64-unknown-linux-musl-binutils-2.41/bin/x86_64-unknown-linux-musl-ld: lib/auto/re/re.a(re_comp.o): in function `Perl_add_above_Latin1_folds':
re_comp.c:(.text+0xab10): multiple definition of `Perl_add_above_Latin1_folds'; libperl.a(regcomp.o):regcomp.c:(.text+0x7ce0): first defined here
/nix/store/sqii1hjj7rr1d5c5wwxlswqrv39nispr-x86_64-unknown-linux-musl-binutils-2.41/bin/x86_64-unknown-linux-musl-ld: lib/auto/re/re.a(re_comp.o): in function `Perl_set_ANYOF_arg':
re_comp.c:(.text+0xb020): multiple definition of `Perl_set_ANYOF_arg'; libperl.a(regcomp.o):regcomp.c:(.text+0x8160): first defined here
/nix/store/sqii1hjj7rr1d5c5wwxlswqrv39nispr-x86_64-unknown-linux-musl-binutils-2.41/bin/x86_64-unknown-linux-musl-ld: lib/auto/re/re.a(re_comp.o): in function `Perl_get_ANYOFM_contents':
re_comp.c:(.text+0xcf50): multiple definition of `Perl_get_ANYOFM_contents'; libperl.a(regcomp.o):regcomp.c:(.text+0x9d70): first defined here
/nix/store/sqii1hjj7rr1d5c5wwxlswqrv39nispr-x86_64-unknown-linux-musl-binutils-2.41/bin/x86_64-unknown-linux-musl-ld: lib/auto/re/re.a(re_comp.o): in function `Perl_get_ANYOFHbbm_contents':
re_comp.c:(.text+0xd020): multiple definition of `Perl_get_ANYOFHbbm_contents'; libperl.a(regcomp.o):regcomp.c:(.text+0x9e40): first defined here
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:154: perl] Error 1
make[1]: Leaving directory '/build/perl-5.38.2'
make: *** [Makefile:87: all] Error 2

full log

Additional context

I'm aiming for pkgsStatic.wine64Packages.minimal, which may be a fool's errand but it's worth a try.

Notify maintainers

@stigtsp @zakame @dasJ (and Eelco?)

Metadata

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.38, NixOS, 23.05 (Stoat), 23.05.1875.8df7a67abaf`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.13.3`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Add a 👍 reaction to issues you find important.

The text was updated successfully, but these errors were encountered:

bltavares · 2024-03-27T13:26:06Z

There is an upstream issue, with a patch reported to fix it.

The commit has been already applied to Perl 5.39.2, so I'm unsure if it's better to attempt to patch 5.38 (current version), or to attempt to upgrade nixpkgs to 5.39.9 instead.

stigtsp · 2024-03-27T13:52:17Z

The commit has been already applied to Perl 5.39.2, so I'm unsure if it's better to attempt to patch 5.38 (current version), or to attempt to upgrade nixpkgs to 5.39.9 instead.

Nice find! I think we should conditionally patch this for static builds ~~to 5.38 will stay around for a while~~.

This commit backports a commit introduced on Perl 5.39.2 to allow `pkgs.pkgsStatic.perl` to compile without raising errors about duplicate symbols being defined during the linking process. This fixes NixOS#295608 **Tested on:** - M1 Mac (aarch64-darwin) - NixOS VM (x86_64-linux) **Test method**: ```sh nix repl -I nixpkgs=$PWD nix repl> pkgs = import <nixpkgs> {} nix repl> :b pkgs.perl nix repl> :b pkgs.pkgsStatic.perl ```

bltavares · 2024-03-27T18:05:27Z

I'm not very familiar with nix nor nixpkgs contribution process, but I'm trying it locally at this moment (and the world is being recompiled, so it's some sign of success, I guess).

If I'm able to have a successful build I'll submit a PR soon.

stigtsp · 2024-03-27T18:16:48Z

I'm happy to help out of course. Do you have a branch somewhere with those changes?

bltavares · 2024-03-27T18:29:18Z

@stigtsp thanks for offering to help.

I've applied the patch regardless of it being a stdenv.hostPlatform.isStatic as upstream has changed for all stdlibs, but it can be checked if it would be better.

stigtsp · 2024-03-27T18:51:47Z

If you'd want to set up a PR I think that would be best, also we should be able to avoid mass rebuilds as mentioned here: 1cbba6d#r140306610

This commit backports a commit introduced on Perl 5.39.2 to allow `pkgs.pkgsStatic.perl` to compile without raising errors about duplicate symbols being defined during the linking process. This fixes NixOS#295608 **Tested on:** - M1 Mac (aarch64-darwin) - NixOS VM (x86_64-linux) **Test method**: ```sh nix repl -I nixpkgs=$PWD nix repl> pkgs = import <nixpkgs> {} nix repl> :b pkgs.perl nix repl> :b pkgs.pkgsStatic.perl ``` Despite the upstream commit being scoped to all environments, we are only applying this patch only on the `pkgsStatic` context to avoid rebuilding all of NixOS packages by changing Perl.

bltavares · 2024-03-28T00:46:01Z

I'm still waiting on my Linux VM to finish compiling the world, which sounds like a good scenario.

Meanwhile I came across a chain of ~~yaks~~ issues on MacOS that prevents pkgsStatic.perl to cross-compile due to case-sensitiveness of the filesystem, even after enabling it. #113985 (comment) -> NixOS/nix#1446 -> NixOS/nix#9318

PS: Another ~~yak~~ issue appeared after being able to move nix-daemon to a TMPDIR that is case-sensitive:

Another yak to be shaved

       > Guessing build platform ... aarch64-apple-darwin
       > Configuring for a native build
       > Checking perl version ... perl5-5.38.2
       > Checking for cc ... aarch64-apple-darwin-clang
       > Checking for ar ... aarch64-apple-darwin-ar
       > Checking for nm ... aarch64-apple-darwin-nm
       > Checking for ranlib ... aarch64-apple-darwin-ranlib
       > Checking for readelf ... none found
       > ERROR: Cannot find readelf

I'm waiting for the Linux build to conclude in order to open a Pull Request.

…usl. This commit backports a commit introduced on Perl 5.39.2 to allow `pkgs.pkgsStatic.perl` to compile without raising errors about duplicate symbols being defined during the linking process. This fixes NixOS#295608 | Tested on | Status | | --- | --- | | M1 Mac (aarch64-darwin) | Failed due to other issues* | - NixOS VM (x86_64-linux) | Compiling | **Test method**: ```sh nix repl -I nixpkgs=$PWD nix repl> pkgs = import <nixpkgs> {} nix repl> :b pkgs.perl nix repl> :b pkgs.pkgsStatic.perl ``` Despite the upstream commit being scoped to all environments, we are only applying this patch only on the `pkgsStatic` context to avoid rebuilding all of NixOS packages by changing Perl. *MacOS: `pkgsStatic.perl` still fails to compile after this change due to MacOS case-insensitiviness affecting the build recipe under /tmp, as `perl-cross/configure` is copied over `perl/Configure` file, as well as not finding `readelf` for the `configurePhase`. NixOS#113985 (comment) NixOS/nix#1446 NixOS/nix#9318 NixOS/nix#8995

This commit backports a commit introduced on Perl 5.39.2 to allow `pkgs.pkgsStatic.perl` to compile without raising errors about duplicate symbols being defined during the linking process. This fixes NixOS#295608 | Tested on | Status | | --- | --- | | M1 Mac (aarch64-darwin) | Failed due to other issues* | | NixOS VM (x86_64-linux) | Compiled | | Nix (aarch64-linux) | Compiled | **Test method**: ```sh nix repl -I nixpkgs=$PWD nix repl> pkgs = import <nixpkgs> {} nix repl> :b pkgs.perl nix repl> :b pkgs.pkgsStatic.perl ``` Despite the upstream commit being scoped to all environments, we are only applying this patch only on the `pkgsStatic` context to avoid rebuilding all of NixOS packages by changing Perl. It includes an additional build flag [also used upstream](Perl/perl5#22052) for configuring static builds. *MacOS: `pkgsStatic.perl` still fails to compile after this change due to MacOS case-insensitiviness affecting the build recipe under /tmp, as `perl-cross/configure` is copied over `perl/Configure` file, as well as not finding `readelf` for the `configurePhase`. NixOS#113985 (comment) NixOS/nix#1446 NixOS/nix#9318 NixOS/nix#8995

YoshiRulz added the 0.kind: build failure label Mar 13, 2024

YoshiRulz changed the title ~~Build failure: pkgsStatic.perl (Perl interpreter w/ Musl libc)~~ Build failure: pkgsStatic.perl (Perl interpreter statically linked to Musl libc) Mar 13, 2024

t4ccer added the 6.topic: static label Mar 19, 2024

bltavares linked a pull request Mar 28, 2024 that will close this issue

(pkgsStatic.perl): Patch Perl to avoid duplicate symbols with musl #299623

Open

13 tasks

jcaesar mentioned this issue May 27, 2024

qemu: add user space emulator only variant #314998

Open

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build failure: `pkgsStatic.perl` (Perl interpreter statically linked to Musl libc) #295608

Build failure: `pkgsStatic.perl` (Perl interpreter statically linked to Musl libc) #295608

YoshiRulz commented Mar 13, 2024 •

edited

bltavares commented Mar 27, 2024

stigtsp commented Mar 27, 2024 •

edited

bltavares commented Mar 27, 2024

stigtsp commented Mar 27, 2024

bltavares commented Mar 27, 2024

stigtsp commented Mar 27, 2024

bltavares commented Mar 28, 2024 •

edited

Build failure: pkgsStatic.perl (Perl interpreter statically linked to Musl libc) #295608

Build failure: pkgsStatic.perl (Perl interpreter statically linked to Musl libc) #295608

Comments

YoshiRulz commented Mar 13, 2024 • edited

Steps To Reproduce

Build log

Additional context

Notify maintainers

Metadata

bltavares commented Mar 27, 2024

stigtsp commented Mar 27, 2024 • edited

bltavares commented Mar 27, 2024

stigtsp commented Mar 27, 2024

bltavares commented Mar 27, 2024

stigtsp commented Mar 27, 2024

bltavares commented Mar 28, 2024 • edited

Build failure: `pkgsStatic.perl` (Perl interpreter statically linked to Musl libc) #295608

Build failure: `pkgsStatic.perl` (Perl interpreter statically linked to Musl libc) #295608

YoshiRulz commented Mar 13, 2024 •

edited

stigtsp commented Mar 27, 2024 •

edited

bltavares commented Mar 28, 2024 •

edited