DESTDIR support would be awesome

[Rhialto]

It would be awesome if there were DESTDIR support in the installation
procedure of rakudo-star.

Very long version:

I've been trying to package rakudo-star for pkgsrc[1], a framework for
building third-party software on NetBSD and other UNIX-like systems, (or
for now more specifically, pkgsrc-wip[2]).

[1] www.pkgsrc.org
[2] git://wip.pkgsrc.org/pkgsrc-wip.git

The general way packages in pkgsrc work is that anyone non-root can
build them, since they are installed to a $DESTDIR and later copied to
the final installation $PREFIX. I think other package management systems
use a similar approach.

However, the build process of rakudo-star makes this difficult. I
noticed that it builds something, installs it to its final destination,
and then uses it to build some more things. This is generally not
compatible with installing via a staging directory (since running
programs from the staging directory is not generally guaranteed to work).

So I tried two approaches to get around this. Both failed at some point
with files being searched in other locations than they were. However in
both cases I got the feeling that it was almost working, and somebody
who knows what they were doing could make it work fairly easily.

Approach [A].

I noticed that the --sdkroot option was used in a lot of places quite
similar to $DESTDIR: the pattern of forming path names
${SDKROOT}${PREFIX}/bin/...

So here I tried to use the --sdkroot option to have the software built
and installed into the staging area just before being used.
Approximately like this:

PREFIX=/usr/pkg
export DESTDIR=/for/instance/tmp/something
Configure.pl --prefix=${PREFIX} --backends=moar --gen-moar --sdkroot=${DESTDIR}

I noticed that not all invocations of sub-Configures were receiving a
copy of the --skdroot option, so I tried to add some code for it to
nqp/tools/lib/NQP/Configure.pm to recognize and pass on the --sdkroot
option.

To make moar work, finding moar.so, I also had to add an

export LD_LIBRARY_PATH=${DESTDIR}${PREFIX}/lib

but my attempt eventually stranded as below and I gave up at this point
and tried another approach.

...
/usr/pkg/bin/perl -MExtUtils::Command -e cp 3rdparty/dyncall/dyncall/*.h /home/tmp/wip/rakudo-star/work.x86_64/.destdir/usr/pkg/include/dyncall
/usr/pkg/bin/perl -MExtUtils::Command -e cp 3rdparty/dyncall/dyncallback/*.h /home/tmp/wip/rakudo-star/work.x86_64/.destdir/usr/pkg/include/dyncall
Building NQP ...
/usr/pkg/bin/perl Configure.pl --prefix=/usr/pkg --sdkroot=/home/tmp/wip/rakudo-star/work.x86_64/.destdir --backends=moar --make-install
Found /home/tmp/wip/rakudo-star/work.x86_64/.destdir/usr/pkg/bin/moar version 2016.04, which is new enough.
Cleaning up ...
/usr/pkg/bin/perl -MExtUtils::Command -e mkpath gen/moar/stage1/gen
/usr/pkg/bin/perl tools/build/gen-cat.pl moar src/how/Archetypes.nqp src/how/RoleToRoleApplier.nqp  src/how/NQPConcreteRoleHOW.nqp src/how/RoleToClassApplier.nqp  src/how/NQPCurriedRoleHOW.nqp src/how/NQPParametricRoleHOW.nqp  src/how/NQPClassHOW.nqp src/how/NQPNativeHOW.nqp  src/how/NQPAttribute.nqp src/how/NQPModuleHOW.nqp  src/how/EXPORTHOW.nqp > gen/moar/stage1/nqpmo.nqp
/usr/pkg/bin/moar --libpath=src/vm/moar/stage0 src/vm/moar/stage0/nqp.moarvm --bootstrap --setting=NULL --no-regex-lib --target=mbc  --output=gen/moar/stage1/nqpmo.moarvm gen/moar/stage1/nqpmo.nqp
sh: /usr/pkg/bin/moar: not found
*** Error code 127

Stop.
make: stopped in /home/tmp/wip/rakudo-star/work.x86_64/rakudo-star-2016.04/nqp
Command failed (status 256): make
Command failed (status 256): /usr/pkg/bin/perl Configure.pl --prefix=/usr/pkg --sdkroot=/home/tmp/wip/rakudo-star/work.x86_64/.destdir --backends=moar --make-install
*** Error code 2

Approach [B].

Here I thought I would do a bootstrap process, where I first build a
rakudo to some temporary directory, using its normal procedure, and then
use that one to build and install to $DESTDIR. I thought it should work
since I noticed that $DESTDIR was actually used in many places in
Makefiles etc.

These are the command I used:

SDKROOT=/tmp/sdkroot
mkdir ${SDKROOT:Q}

PREFIX=/usr/pkg
export DESTDIR=/for/instance/tmp/something

# Build for sdkroot
perl Configure.pl --prefix=${SDKROOT:Q} --gen-moar

make rakudo-install

# Configure and build for real
perl Configure.pl --prefix=${PREFIX:Q} --gen-moar --sdkroot=${SDKROOT:Q}

make install

This however went along nicely, until

# Configure and build for real

...

compiling src/instrument/crossthreadwrite.o
compiling src/moar.o
compiling src/platform/posix/mmap.o
compiling src/platform/posix/time.o
compiling src/platform/posix/sys.o
compiling src/jit/emit_posix_x64.o
linking libmoar.so
linking moar
/usr/pkg/bin/perl build/mk-moar-pc.pl pkgconfig/moar.pc
/usr/pkg/bin/perl -MExtUtils::Command -e mkpath /usr/pkg/bin
/usr/pkg/bin/perl -MExtUtils::Command -e cp moar /usr/pkg/bin
/usr/pkg/bin/perl -MExtUtils::Command -e chmod 755 /usr/pkg/bin/moar
Cannot chmod 755 /usr/pkg/bin/moar:No such file or directory at /usr/pkg/lib/perl5/5.22.0/ExtUtils/Command.pm line 280.
*** Error code 2

Stop.
make[2]: stopped in /home/tmp/wip/rakudo-star/work.x86_64/rakudo-star-2016.04/MoarVM
Building NQP ...
/usr/pkg/bin/perl Configure.pl --prefix=/usr/pkg --backends=moar --make-install
Can't exec "/usr/pkg/bin/moar": No such file or directory at tools/lib/NQP/Configure.pm line 450.

===SORRY!===
No suitable MoarVM (moar executable) found using the --prefix
(You can get a MoarVM built automatically with --gen-moar.)

Command failed (status 512): /usr/pkg/bin/perl Configure.pl --prefix=/usr/pkg --backends=moar --make-install
*** Error code 2

It seems that $DESTDIR is not taken into account there in the perl
commands, and that the --sdkroot option is not used when trying to run
/usr/pkg/bin/moar.

At this point, I gave up again, seeing that the build scripts are a bit
too complicated to grok properly in a few hours.

So, after this introduction, this is more or less my question: Do you
developers agree that it is useful to have an easy installation
procedure via a staging directory? And if yes, what can you to realise
this?

Personally, I prefer approach [A] since it is not doing the work twice.
But it may need some tricks to be able to run programs from the staging
directory instead of the installation directory.

I noticed issue #65 is basically asking the same thing, but I've expanded on that by giving the results of some approaches that I tried.

#65
#43
voidlinux/void-packages#3536

[lefth]

$DESTDIR support is also needed before rakudo-star can be packaged for Archlinux, or even installed by Arch's build-script system for installing third-party packages. Of course rakudobrew and tarball installs still work, but many users hesitate to install software that's neither in the repos nor the user repository of build scripts.

[Tyil]

The current iteration of Rakudo Star uses a Bash script instead of a Makefile. This script supports -p <destination> when installing. This should cover this particular use case.