Net::DNS::Native and Mojolicious on Linux

[olegwtf]

As we noticed yesterday Net::DNS::Native with perl not linked with pthreads still may crash on Linux: https://rt.cpan.org/Public/Bug/Display.html?id=100834

Today I found a way to reproduce this problem with the following server

use Mojolicious::Lite;
use Mojo::IOLoop;

get '/' => sub {
        my $self = shift;

        Mojo::IOLoop->timer(rand(1) => sub {
                $self->render(json => [qw/a b c d e d/]);
        });
};

app->start;

and client

use strict;
use Mojo::UserAgent;
use Mojo::IOLoop;

my $ua = Mojo::UserAgent->new;
my @links = ("http://data-flow.ru:3000") x 5000;

while (@links) {
        Mojo::IOLoop->delay->new->steps(
                sub {
                        my @l = splice @links, 0, 50;
                        my $delay = shift;
                        $ua->get($_ => $delay->begin) for @l;
                },
                sub {
                        warn "+50";
                }
        )->wait
}

where you need to change data-flow.ru to your own domain (but not localhost).
This client segfaults 10 times out of 10

And the fix is as simple as

 use strict;
+use Net::DNS::Native;
 use Mojo::UserAgent;
 use Mojo::IOLoop;

And all the problem caused by ... my favorite IO::Socket::IP. Look. Mojolicious loads IO::Socket::IP here and IO::Socket::IP has this construction which executes getprotobyname() at compile time. Looks ok, but this loads non thread safe DNS part of libc (because our perl not linked with pthreads). And only after that Mojolicious loads Net::DNS::Native, which can't replace non-thread safe DNS part of libc with thread safe even with RTLD_GLOBAL flag.
So, you can comment mentioned line of IO::Socket::IP and all will work good
Or you can load IO::Socket::IP after Net::DNS::Native inside Mojolicious and all will work good
But you can't fix this inside Net::DNS::Native

The sad part is that if we'll fix this inside Mojolicious where is the warranty that somebody will not write smth like

use IO::Socket::IP
use Mojo::IOLoop::Client

I can disable non pthreaded perl support on Linux inside Net::DNS::Native again, which will fix this problem at all. But this is a big amount of users.

Btw [Tux] from #p5p promised to add "linking with pthreads by default" feature few months ago. Looks like I need to remind him about this :)

Thoughts?

[kraih]

Thanks for the detailed explanation, i'm not quite sure yet what to do about this.

[kraih]

It looks like we have consensus (on IRC) for removing Net::DNS::Native support from Mojolicious for now. And possibly adding a pluggable resolver API instead, which could be used by 3rd party CPAN modules, to make Net::DNS::Native available again. But the second part would depend entirely on a good enough proposal being submitted first.

[kraih]

And done, wish there was something else we could do. But as long as Perl is not linked with pthreads by default, our options are very limited.

[olegwtf]

Don't know will this change something about Net::DNS::Native + Mojolicious, but I found a way to check is non thread safe symbols loaded which will cause crash. I already added this check in master.
So, now it works like this

$ perl -MIO::Socket::IP -MNet::DNS::Native -E 'say 1'
***********************************************************************
Some package defined non thread safe symbols which Net::DNS::Native uses internally
Please make sure you are not placed loading of modules like IO::Socket::IP
before this one
And not called functions like getaddrinfo(), gethostbyname(), inet_aton()
before loading of Net::DNS::Native
************************************************************************ at /home/oleg/p5-Net-DNS-Native/blib/lib/Net/DNS/Native.pm line 32.
Compilation failed in require.
BEGIN failed--compilation aborted.

$ perl -MNet::DNS::Native -MIO::Socket::IP -E 'say 1'
1

$ perl -Mblib -MSocket -E 'BEGIN{inet_aton("google.com")}; use Net::DNS::Native; say 1'
***********************************************************************
Some package defined non thread safe symbols which Net::DNS::Native uses internally
Please make sure you are not placed loading of modules like IO::Socket::IP
before this one
And not called functions like getaddrinfo(), gethostbyname(), inet_aton()
before loading of Net::DNS::Native
************************************************************************ at /home/oleg/p5-Net-DNS-Native/blib/lib/Net/DNS/Native.pm line 32.
Compilation failed in require.
BEGIN failed--compilation aborted.

Just for the record: next perl release will introduce linking with pthreads by default (http://perl5.git.perl.org/perl.git/commit/237050634b19905cdd9e8e25c2a43d0b24eb59f2) 👏

[jberger]

@olegwtf ++

[leonerd]

I wonder if at least some of this could be fixed by adjusting the code in IO::Socket::IP to defer that compiletime call to getprotobyname() somehow? Would that help?