Installer does not play well with Arch Linux’s makepkg

[ghost]

I am trying to update the AUR package for Arch to 2019.07.1, but I am running into issues with the install step. More specifically: the packaging script runs the command:

make DESTDIR="$pkgdir" install

in a fakeroot environment (Arch packages are normally built as a normal user). However:

Issue 1: install-core-dist.p6 seems overly pessimistic about its ability to write to the installation directory.

'/tmp/makepkg/rakudo/src/rakudo-2019.07.1/perl6-m' '/tmp/makepkg/rakudo/src/rakudo-2019.07.1/tools/build/install-core-dist.p6' '/tmp/makepkg/rakudo/pkg/rakudo/usr/share/perl6/core'
No writeable path found, /tmp/makepkg/rakudo/pkg/rakudo/usr/share/perl6/core not writeable
  in block <unit> at /tmp/makepkg/rakudo/src/rakudo-2019.07.1/tools/build/install-core-dist.p6 line 33

This is printed despite the fact that some files still managed to be written there.

Issue 2: if I try to run the make install command in a normal shell to circumvent this problem, then this happens:

$ make DESTDIR=/tmp/makepkg/rakudo/pkg/rakudo/ install
[…]
'/tmp/makepkg/rakudo/src/rakudo-2019.07.1/perl6-m' '/tmp/makepkg/rakudo/src/rakudo-2019.07.1/tools/build/install-core-dist.p6' '/tmp/makepkg/rakudo/pkg/rakudo//usr/share/perl6/core'
installed!
rm -f -- /usr/share/nqp/lib/Perl6/Actions.moarvm
rm: cannot remove '/usr/share/nqp/lib/Perl6/Actions.moarvm': Permission denied

That is, the installation script tries to touch the existing system install of Rakudo, despite the DESTDIR parameter. I did get warnings about these files during the build step, but I was hoping that the install script would honor DESTDIR.

===NOTICE===
  Found stale files in /usr/share/nqp/lib/Perl6.
  These files were left by a previous install and cause breakage
  in this Rakudo version. The files will be removed during install.
  
  Will remove: /usr/share/nqp/lib/Perl6/Actions.moarvm
  Will remove: /usr/share/nqp/lib/Perl6/BOOTSTRAP.moarvm
  Will remove: /usr/share/nqp/lib/Perl6/Compiler.moarvm
  Will remove: /usr/share/nqp/lib/Perl6/Grammar.moarvm
  Will remove: /usr/share/nqp/lib/Perl6/Metamodel.moarvm
  Will remove: /usr/share/nqp/lib/Perl6/ModuleLoader.moarvm
  Will remove: /usr/share/nqp/lib/Perl6/Ops.moarvm
  Will remove: /usr/share/nqp/lib/Perl6/Optimizer.moarvm
  Will remove: /usr/share/nqp/lib/Perl6/Pod.moarvm
  Will remove: /usr/share/nqp/lib/Perl6/World.moarvm

This used to work in 2019.03. Should I attempt to bisect the issue or is it already relatively clear what could have introduced the bugs?

[AlexDaniel]

This used to work in 2019.03. Should I attempt to bisect the issue or is it already relatively clear what could have introduced the bugs?

The build system was changed completely, I don't think there's any point in bisecting it.

[ghost]

Ah, I see, thanks.

For issue 2, it seems that this would be the line to change:

rakudo/tools/lib/NQP/Config/Rakudo.pm

Line 509 in 082c09e

$self->{config}->{clean_old_p6_libs} .= "\t\$(RM_F) $file\n";

For issue 1, I am not sure.

[vrurg]

It would make sense of bisecting install-core-dist.p6. It's not changed.

With regard to the second problem, I wonder what is the installation prefix?

[ghost]

The prefix passed to Configure.pl is /usr because that’s where the installed package will end up. However, in this use-case of building a package, the installation script should not care about files that are already there. Those belong to the previous version of the package and will be correctly removed by the package manager when installing the new version. If passed a DESTDIR, the install script should only care about that.

[vrurg]

Ping @PatZim – I think this one is better for you to take care about.

[jonathonf]

Is it possible that it's checking using the user's real rather than "effective" UID?

builder   8326  0.0  0.0   4084  2980 pts/0    T    17:46   0:00 /bin/sh /usr/sbin/fakeroot make DESTDIR=/build/pkg/rakudo install
builder   8334  0.0  0.0   2244   148 ?        Ss   17:46   0:00 /usr/bin/faked
builder   8341  0.5  0.0  15208  6188 pts/0    T    17:46   0:00 make DESTDIR=/build/pkg/rakudo install
builder   8352  0.1  0.0  63624 20916 pts/0    Tl   17:46   0:00 /usr/bin/moar --libpath=/build/src/rakudo-2019.07.1/blib --libpath=/usr/share/nqp/lib perl6.moarvm --nqp-lib=/build/src/rakudo-2019.07.1/blib -e for @*ARGS

[dod38fr]

Hi, I've seen a similar issue on Debian/unstable

With libuv 1.30.1, rakudo fails installation with:

No writeable path found, /home/domi/debian-dev/build-area/rakudo-2019.07.1/debian/rakudo/usr/lib/perl6/core not writeable

We've seen this problem occurs with rakudo 2018.12 and 2019.07
(See rakudo 2018.12 build log)

Note that rakudo 2018.12 did build fine with libuv 1.24.1

Hope this helps

[jonathonf]

Ah, Debian's nqp uses the system libs, the AUR package uses bundled libs. Let me check what happens if I use system libs instead.

[dod38fr]

For what it's worth, here are some details on the setup that shows this issue with rakudo 2018.12:

• moarvm and nqp 2018.12 installed from stable repo (i.e. compiled with libuv 1.24.1)
• rakudo package builds fine with libuv1 and libuv1-dev installed from stable (i.e. version 1.24.1)
• rakudo package fails to build with libuv1 and libuv1-dev installed from unstable (i.e. version 1.30.1)

The diff between successful and failed logs shows no significant diff except the error at the end.

[vrurg]

@spider-mario could you, please, try the latest HEAD? I have merged a potential fix for the second issue but can't test it properly.

[vrurg]

As to the first issue, I highly suspect it to be related to #3170.

[ghost]

Thanks for the fix for the second issue, I will try to test it soon.

I doubt that the first issue is related to that Windows bug, though: in the Windows bug, the install step actually tries to perform the operation but fails, which could be due to Windows’ stricter locking of files that are in use. In the first issue, however, it doesn’t even try because it thinks that the destination path is not writable (despite having previously written to it).

[dod38fr]

Hi. @robertlemmen found a bad interaction between fakeroot and newer libuv that triggers build failure on Debian (TLDR version is: new libuv uses statx instead of stat. statx is not emulated by fakeroot, so the actual statx is called instead of a fake one). Since Arch uses fakeroot to build package, this may also impact builds on Arch linux.

You find more details there and there

Many thanks to @robertlemmen for the investigation :-)

[niner]

Since this is a bug in fakeroot, not in rakudo, can we remove the blocker label?

[vrurg]

@niner I think it makes sense. Removing.

[patrickbkr]

A recent MoarVM change might also make a difference wrt to Issue 1.

[jonathonf]

@spider-mario The above MoarVM patch fixes "Issue 1" for me with 2019.07.1, though it needs to be adjusted slightly to apply on the 2019.07.1 release code:

https://gist.github.com/jonathonf/88ccc50f12d4f5c0575da2924ee8f2fb

Pop that into moarvm's prepare(), rebuild moarvm and nqp, then rakudo will also build.

The solution for "Issue 2" would probably be to conflict with the earlier rakudo version, or just build in a clean chroot.

[ghost]

Sorry for the time it took me to test everything. I have backported c9428a5 to Rakudo 2019.07.1 and it appears to solve the second issue, as does MoarVM/MoarVM@36c35db fix the first one.

Many thanks to everyone involved!