nqp: cannot build with pre-installed moar

[dod38fr]

Hello

On Debian/sid, the The command perl Configure.pl --backends=moar --with-moar=/usr/bin/moarworks fine (well, it does not complain).

But make fails with:
(02:07:41 PM) domidumont: make: /home/domi/debian-dev/perl6-stuff/nqp/install/bin/moar: Command not found

Full log:

perl Configure.pl --backends=moar --with-moar=/usr/bin/moar
Found /usr/bin/moar version 2015.03, which is new enough.
Cleaning up ...
You can now use 'make' to build NQP.
After that, 'make test' will run some tests and
'make install' will install NQP.
  domi@ylum:/home/domi/debian-dev/perl6-stuff/nqp$ make
/usr/bin/perl -MExtUtils::Command -e mkpath gen/moar/stage1/gen
/usr/bin/perl tools/build/gen-cat.pl moar src/how/Archetypes.nqp src/how/RoleToRoleApplier.nqp src/how/NQPConcreteRoleHOW.nqp src/how/RoleToClassApplier.nqp src/how/NQPCurriedRoleHOW.nqp src/how/NQPParametricRoleHOW.nqp src/how/NQPClassHOW.nqp src/how/NQPNativeHOW.nqp src/how/NQPAttribute.nqp src/how/NQPModuleHOW.nqp src/how/EXPORTHOW.nqp  > gen/moar/stage1/nqpmo.nqp
/home/domi/debian-dev/perl6-stuff/nqp/install/bin/moar --libpath=src/vm/moar/stage0 src/vm/moar/stage0/nqp.moarvm --bootstrap --setting=NULL --no-regex-lib --target=mbc \
    --output=gen/moar/stage1/nqpmo.moarvm gen/moar/stage1/nqpmo.nqp
make: /home/domi/debian-dev/perl6-stuff/nqp/install/bin/moar: Command not found
Makefile:218: recipe for target 'gen/moar/stage1/nqpmo.moarvm' failed
make: *** [gen/moar/stage1/nqpmo.moarvm] Error 127

Any clue ?

[FROGGS]

This seems to do: perl Configure.pl --prefix=/usr --backend=moar

Though, I had to patch nqp to accept debian's version format debian/2015.03-1

diff --git a/tools/lib/NQP/Configure.pm b/tools/lib/NQP/Configure.pm
index 873a8cd..3fa8d09 100644
--- a/tools/lib/NQP/Configure.pm
+++ b/tools/lib/NQP/Configure.pm
@@ -56,9 +56,9 @@ sub system_or_die {
 sub parse_revision {
     my $rev = shift;
     my $sep = qr/[_.]/;
-    $rev =~ /(\d+)$sep(\d+)(?:$sep(\d+))?(?:-(\d+)-g[a-f0-9]*)?$/
+    $rev =~ /(\d+)$sep(\d+)(?:$sep(\d+))?(?:-(\d+)-g[a-f0-9]*)?(?:-(\d+))?$/
         or die "Unrecognized revision specifier '$rev'\n";
-    return ($1, $2, $3 || 0, $4 || 0);
+    return ($1, $2, $3 || 0, $4 || 0, $5 || 0);
 }

Will this be needed? Or will the version be what you've shown in your paste?

[FROGGS]

@paultcochrane please hook in if needed.

[FROGGS]

Ahh, and as mentioned in IRC the 2015.03's nqp still does create the languages directory while the current developement version does not.

$ /usr/bin/nqp-m -e 'say(42)'
42

[FROGGS]

If someone wants to try the MoarVM debian package, it lives here: git://git.debian.org/git/pkg-rakudo/pkg-moarvm.git

I just had to: make -f debian/rules && make && sudo make install
The installed files of MoarVM are: https://gist.github.com/FROGGS/344e01830f3c06c13e1f
And of NQP are: https://gist.github.com/FROGGS/5a1452da15824392151e

[dod38fr]

Using make install, you skip the Debian instructions for installation and bypass the Debian package manager.

To get what Debian users will get, you should:

• Install dpkg development packages: sudo aptitude install git-buildpackage devscripts
• Run git-buildpackage -us -uc
• Install the built packages: sudo debi

To clean up the source directory you can do make -f debian/rules clean or git reset --hard && git clean -dxf

To view the content of an installed package, do dpkg --listfiles <pkg> e.g. dpkg --listfiles moarvm.
To get the content of a deb package, do dpkg --contents <pkg> e.g. dpkg --contents ../rakudo_2014.07-4_amd64.deb

Hope this helps

[dod38fr]

I'm surprised by the issue with debian/2015.03-1 version. This is a git tag used only in our package repo. This may be due to the way you installed moar (i.e. with sudo make install).

With a moarvm package install with Debian package manager, I have a normal behavior:

$ perl Configure.pl --prefix=/usr --backend=moar
Found /usr/bin/moar version 2015.03, which is new enough.
Cleaning up ...
You can now use 'make' to build NQP.
After that, 'make test' will run some tests and
'make install' will install NQP.

This should solve the package build issue. I'll try that later and keep you posted.

Thanks for the help

[paultcochrane]

This seems to do: perl Configure.pl --prefix=/usr --backend=moar

That command was what worked for me in order to build Debian packages
generated from the files in the perl6/os-build repository[1], given the
assumption that MoarVM, nqp and Rakudo were all installed under /usr. To be
explicit, the build and installation steps were as follows:

• moar build and install

  $ perl Configure.pl --prefix=/usr
  $ make install

• nqp build and install

  $ perl Configure.pl --prefix=/usr --backends=moar
  $ make install

• Rakudo build and install

  $ perl Configure.pl --prefix=/usr --backends=moar
  $ make install

This was a rough first cut intended to develop the Debian control files to
the point that they build with the MoarVM, nqp, Rakudo toolchain (and with a
current version of Rakduo) as opposed to the older Parrot toolchain (and the
2014.07 Rakudo version). The hope was then to pass this information on to
the Debian Perl maintainers so that they had something to work with which
had most of the major bugs ironed out and thus hopefully get an up to date
version of Rakudo in Debian testing.

In the comment I made about --with-moar on IRC yesterday, my use case was
a bit different (nevertheless related). I wanted to check something in
MoarVM, and since MoarVM doesn't have tests itself (it is tested by running
the nqp test suite, at least that's what make test in MoarVM tells me) I
wanted my development nqp to point to my development MoarVM. I thus
tried --with-moar without success as it didn't seem to point nqp to the
correct directory. As it turned out, the moar configuration setting was
being set correctly inside Configure.pl, however the configuration value
wasn't being used when generating the Makefile (i.e. there was no @moar@
in Makefile-Moar.in to be expanded). I hacked something together such
that nqp could run its test suite using my development version of MoarVM
like so:

• insert a @moar@ variable to be expanded in Makefile-Moar.in:

  diff --git a/tools/build/Makefile-Moar.in b/tools/build/Makefile-Moar.in
  index 8d4535e..6c8f809 100644
  --- a/tools/build/Makefile-Moar.in
  +++ b/tools/build/Makefile-Moar.in
  @@ -1,5 +1,5 @@
  BAT = @bat@
  -MOAR = $(BIN_DIR)/moar
  +MOAR = @moar@
  M_RUNNER = nqp-m$(BAT)

  M_STAGE0 = src/vm/moar/stage0

• append the development MoarVM dir to the LD_LIBRARY_PATH when
  configuring nqp

  $ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:../MoarVM/ perl Configure.pl --with-moar=../MoarVM/moar

• correct the paths in the generated Makefile which point to
  MAST/Ops.nqp and MAST/Nodes.nqp (the value share/nqp is hard-coded
  into the path), they should point to the absolute path versions of
  ../MoarVM/lib/MAST/Ops.nqp and ../MoarVM/lib/MAST/Nodes.nqp
  respectively

• build nqp and run the tests also with an updated LD_LIBRARY_PATH

  $ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:../MoarVM/ make
  $ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:../MoarVM/ make test

This works (for some definition of "work"), however isn't very easy to use.

Thus I was wondering if an option could be added to the nqp configuration
phase which merely specifies the path to the required MoarVM directory and
then the moar binary path and the required lib paths are set correctly
from there. Perhaps something along the lines of --moar-path=...?

Sorry this ended up a bit long, however I wanted to have sufficient detail
so that people could reproduce the steps I took if necessary.

Cheers,

Paul

[1] whether or not this is the right place to develop the Debian control
files is another matter...

[paultcochrane]

I'm surprised by the issue with debian/2015.03-1 version. This is a git tag used only in our package repo. This may be due to the way you installed moar (i.e. with sudo make install).

With a moarvm package install with Debian package manager, I have a normal behavior:

$ perl Configure.pl --prefix=/usr --backend=moar
Found /usr/bin/moar version 2015.03, which is new enough.
Cleaning up ...
You can now use 'make' to build NQP.
After that, 'make test' will run some tests and
'make install' will install NQP.

agreed. It shouldn't be necessary to patch nqp's version handling to get
nqp to build with a pre-installed (via package manager) MoarVM.

[dod38fr]

For the record, debian has a patch to avoid using git to get nqp version. This patch was not applied when FROGGS built the package, hence the weirdness...

[FROGGS]

But to be clear: It looks like we can build/package nqp-maor now too without chaning moar at all, right?

[dod38fr]

nqp builds fine now with a pre-installed moar:

$ ./nqp -v
This is nqp version 2015.03 built on MoarVM version 2015.03

But I have one failure during tests:

$ ./nqp t/hll/06-sprintf.t  | grep -A 3 'not ok'
not ok 91 - simple %b with zero value
#   Actual value: 
# Expected value: 0
ok 92 - 5.2 %e

Is this fixable ?

[dod38fr]

@FROGGS : yes, there's no need to change moar or nqp.

Well, may be update Configure.pl doc: using --prefix to specify where to find moar is not really obvious...