"No subtests run" CPAN Testers failure

[calid]

http://www.cpantesters.org/cpan/report/82a6d9fa-c691-11e4-81ee-a225e0bfc7aa
http://www.cpantesters.org/cpan/report/81c2a190-c691-11e4-81ee-a225e0bfc7aa
http://www.cpantesters.org/cpan/report/dc0274fa-c632-11e4-a2b5-449ce0bfc7aa
http://www.cpantesters.org/cpan/report/dfe98c8a-c62c-11e4-8bfc-ee8be0bfc7aa
http://www.cpantesters.org/cpan/report/6bda7bfc-c699-11e4-adfb-e2d3ebda7229
http://www.cpantesters.org/cpan/report/6bda7bfc-c699-11e4-adfb-e2d3ebda7229

Seems to only occur on BSD. One of the CPAN testers provided this info:

I reproduced the failure on the same box using cpanminus.

Wstat: 134 means ( after 134 & 127 ) signal 6 was generated which is abort.

The box does have both libffi and libzmq installed.

libffi-3.0.13 Foreign function interface
zeromq-4.0.4 The ZeroMQ messaging library

[plicease]

So far I haven't been able to reproduce this on my FreeBSD boxes. I have one 9.x and one 10.x 64 bit (both installed ZMQ::FFI cleanly) and one 10.x 32 bit (though the 32bit one is in the process of rebuilding so I haven't gotten there yet). The libffi on those machines is more recent that 3.0.13 though, I will try the older version next. I can also try on my netbsd VM if I can remember how to install packages in that environment :)

I noticed the kFreeBSD failures were similar to the Linux failures (and should be fixed with the Math::BigInt upgrade), so that points to the userland rather than the kernel.

[calid]

@plicease thanks so much for your help, let me know what you find. I've released 1.01 with the newer Math::BigInt requirement.

[calid]

Since adding the newer Math::BigInt requirement test failures only appear to be on BSD now (interestingly only FreeBSD and NetBSD flavors):
http://matrix.cpantesters.org/?dist=ZMQ-FFI+1.02

And they all exhibit the same symptom ("No subtests run"), so at least it's consistent...

[plicease]

Note that the the failures for both Net and FreeBSD are on unthreaded Perls for 1.02.

I haven't been able to do any testing on this yet, but I hope to soon.

[plicease]

Also the Perl that I built for FreeBSD was threaded too, which may explain why I wasn't able to reproduce it.

[plicease]

I think I have this figured out. I was able to reproduce this on NetBSD once I installed zmq since the Perl on my NetBSD VM is unthreaded. It is, as I suspected dumping core:

netbsd64% perl -Mblib t/fd.t
Abort (core dumped)

The problem is that zmq uses pthreads:

netbsd64% ldd /usr/pkg/lib/libzmq.so.4.0.0
/usr/pkg/lib/libzmq.so.4.0.0:
        -lsodium.13 => /usr/pkg/lib/libsodium.so.13
        -lc.12 => /usr/lib/libc.so.12
        -lrt.1 => /usr/lib/librt.so.1
        -lgcc_s.1 => /usr/lib/libgcc_s.so.1
        -lpthread.1 => /usr/lib/libpthread.so.1
        -lstdc++.7 => /usr/lib/libstdc++.so.7
        -lm.0 => /usr/lib/libm.so.0

I was able to get the test suite to pass by setting LD_PRELOAD:

netbsd64% env LD_PRELOAD=/usr/lib/libpthread.so.1 prove -b
t/close.t ....... ok
t/device.t ...... ok
t/errors.t ...... ok
t/fd.t .......... ok
t/fork.t ........ ok
t/gc.t .......... ok
t/multipart.t ... ok
t/options.t ..... ok
t/proxy.t ....... ok
t/pubsub.t ...... ok
t/router-req.t .. ok
t/send_recv.t ... ok
t/unbind.t ...... ok
t/unicode.t ..... ok
All tests successful.
Files=14, Tests=37,  2 wallclock secs ( 0.03 usr  0.03 sys +  0.79 cusr  0.22 csys =  1.07 CPU)
Result: PASS

Solutions as I see it include:

• do not support unthreaded perls on Free/NetBSD
• document the LD_PRELOAD hack and modify the test suite to set it on Free/NetBSD (I am not sure how to do that exactly with MM, but presumably it is possible).

Here are some references:

https://rt.perl.org/Public/Bug/Display.html?id=122906
http://stackoverflow.com/questions/13587325/threads-vs-pthread-in-perl

This does seem to be a general Perl issue and not a FFI / Platypus issue, so that at least is good :)

[plicease]

To clarify, an unthreaded Perl without the LD_PRELOAD hack WOULD work, IF the Perl had been linked with pthreads (which is possible with manual intervention by the person building Perl itself). This is probably half baked, and may not be portable, but I was able to detect if pthreads had been initalized with this snipit:

use strict;
use warnings;
use 5.010;
use FFI::Platypus;

my $ffi = FFI::Platypus->new;
$ffi->find_lib( lib => 'pthread' );

$ffi->attach( pthread_self => [] => 'opaque' );

if(pthread_self() == -1)
{
  warn 'pthreads were not initalized';
}
else
{
  say pthread_self();
}

Let me know if you want me to test whatever solution you come up with.

[plicease]

Not sure why I didn't think of it last night but I think the portable way to determine if pthreads is already baked into the Perl you are using is use find_symbol to search for pthread_self in the current process.

It breaks down if someone else has already loaded a library linked to pthreads, but them having done that is probably broken anyway.

use strict;
use warnings;
use 5.010;
use FFI::Platypus;

my $ffi = FFI::Platypus->new;
$ffi->lib( undef );

if($ffi->find_symbol('pthread_self'))
{
  say "have pthreads";
}
else
{
  say "no pthreads";
}

[calid]

Awesome @plicease, thanks for the update.

Just so I'm clear, the reason this isn't an issue on Linux is because the libc/pthread/dlopen semantics are different? i.e. on Linux pthread init can happen lazily on a subsequent dlopen, but on BSD it only happens on initial loading of the executable?

That is, they're both honoring the NEEDED entry in the ELF dynamic section, it's just a matter of whether pthreads is being initialized on the lazy dlopen or not?

$ readelf -d /usr/lib/libzmq.so

Dynamic section at offset 0xa2d28 contains 31 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libsodium.so.13]
 0x0000000000000001 (NEEDED)             Shared library: [librt.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]  <===
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]

Regarding how to address, yeah I'm thinking go with the find_symbol check you mentioned. Put that in Makefile.PL and bail out with the "unthreaded Perls not supported" warning on BSD... will generate NA on CPAN Testers, and immediately alert people when they try to install the module.

[calid]

Let me know if you want me to test whatever solution you come up with.

@plicease I added the find_symbol check and pushed it to the pthread-self-check branch. If you have time mind giving it a quick whirl? It should just bail installing on an unthreaded BSD perl.

[plicease]

diff --git a/dist.ini b/dist.ini
index 07fd08e..94058a4 100644
--- a/dist.ini
+++ b/dist.ini
@@ -12,11 +12,11 @@ lib = zmq

 [MakeMaker::Awesome]
 delimiter = |
+header = |use FFI::Platypus;
 header = |# Can't currently support unthreaded BSD perls
 header = |# See GH #13
 header = |if ($^O =~ m/bsd/i) {
-header = |   !FFI::Platypus->new
-header = |                 ->lib(undef)
+header = |   !FFI::Platypus->new(lib => [undef])
 header = |                 ->find_symbol('pthread_self')
 header = |                 && exit;
 header = |}

Almost, need to use FFI::Platypus first and lib is a attribute rather than a method so it returns the value instead of the $ffi instance. Once I made that change it worked. Or rather didn't work in the right way on NetBSD.

Fails with exit 0 on NetBSD without pthreads and installs on FreeBSD64 with a threaded Perl. Also installed on NetBSD with the LD_PRELOAD hack.

[calid]

fixed, thanks. All of the outstanding ZMQ::FFI CPAN tester failures were on unthreaded BSD perls, so here's hoping I have a 100% pass rate after this :)

[calid]

well, no luck... still failing on bsd. Checked the Makefile.PL in 1.04 and it has the correct blurb from above, so I'm stumped.

http://www.cpantesters.org/distro/Z/ZMQ-FFI.html?oncpan=1&distmat=1&version=1.04&grade=3

@plicease you think just check $Config{usethreads} instead? Yeah it breaks for the custom Perl/LD_PRELOAD hack use cases, but those are probably pretty out there use cases anyways.

grr this action-at-a-distance stuff is hard to test. I'll reach out to the CPAN Testers and see if they can tell me if a) their Perl is definitely sans threads, and b) why the Makefile.PL didn't bail like it was supposed to.

[calid]

@eserte are these your CPAN Tester boxes? If so mind helping me debug this?

http://www.cpantesters.org/cpan/report/d586ea36-d25b-11e4-99a7-833ee0bfc7aa
http://www.cpantesters.org/cpan/report/d493c1a8-d25b-11e4-99a7-833ee0bfc7aa
http://www.cpantesters.org/cpan/report/d3afd4c0-d25b-11e4-99a7-833ee0bfc7aa
http://www.cpantesters.org/cpan/report/d2c7fb32-d25b-11e4-99a7-833ee0bfc7aa

[plicease]

@calid It's up to you, and I agree that breaking LD_PRELOAD is not a big deal, but the problem with $Config{usethreads} is that it also breaks people without a threaded Perl, but specified -lpthreads when they build Perl, which is definitely something that people do.

It does seem to be working for NetBSD at least. Admittedly I wasn't able to test FreeBSD because my Perl on my FreeBSD system is threaded. For FreeBSD you could check $Config{ldflags} for -pthread:

freebsd% perl -V:ldflags
ldflags='-pthread -Wl,-E  -fstack-protector -L/usr/local/lib';

At some point I do want to build an unthreaded Perl on FreeBSD that I can do some testing on.

[calid]

oh yeah, $Config{ldflags}... yeah that's probably a better option as it wouldn't kill the manually linked pthread case.

I'll probably hold off on further changes until more CPAN Tester reports come in... who knows, maybe those 4 are just a one off and it's working as expected everywhere else.

[plicease]

Also it wouldn't hurt to let me get an unthreaded FreeBSD perl. It would be good to verify that -pthreads is missing from $Config{ldflags} there.

[calid]

Definitely, thanks again for your help

[eserte]

Yes, the test reports mentioned above come from my boxes. The long runtime (wallclock secs > 3200) indicates that the tests were killed because they were hanging (inactivity timeout is 200s times 16 tests = 3200s). And yes, plicease's observation is correct that it seems to fail only for unthreaded FreeBSD perls without -pthread compiled in. Because this is a frequent problem (also for other CPAN distribution) I tend to manually hack -pthread into newer perls, but unfortunately this is not officially supported by perl's configure mechanism.

Right now I am running your module with such perls, so expect a couple of PASSes for the combination FreeBSD + unthreaded perl.

[calid]

@eserte thanks. One question: the newest ZMQ::FFI version has a check in Makefile.PL to exit if it doesn't find pthread_self in the running perl, any idea why the tests started at all? If perl was built without -pthread that symbol should be missing and the Makefile should have exited 0?

From the 1.04 Makefile.PL on CPAN:

use FFI::Platypus;
# Can't currently support unthreaded BSD perls
# See GH #13
if ($^O =~ m/bsd/i) {
   !FFI::Platypus->new(lib => [undef])
                 ->find_symbol('pthread_self')
                 && exit;
}

[eserte]

pthread_self seems to be part of libc:

$ strings /lib/libc.so.7 | grep pthread_self
_pthread_self

[calid]

@plicease thanks. @eserte I've published a developer release 1.05_01 with @plicease's special freebsd check. Mind trying it and letting me know if it works as intended? If so I'll go ahead and publish 1.06 and I think we can finally consider this issue put to rest :)

The direct link is at: http://www.cpan.org/authors/id/C/CA/CALID/ZMQ-FFI-1.05_01.tar.gz

[calid]

I published 1.05_02 and added an error message for unthreaded bsd perls, hopefully I didn't botch the check.

http://www.cpan.org/authors/id/C/CA/CALID/ZMQ-FFI-1.05_02.tar.gz

[eserte]

1.05_02 looks good, it seems that the problematic perl versions are properly skipped on my systems.

[calid]

I've published 1.06 with the fix, thanks guys for the help.