`make test` fails on OS X El Capitan

[p5pRT]

Migrated from rt.perl.org#126706 (status was 'resolved')

Searchable as RT126706$

[p5pRT]

From dominyktiller@gmail.com

As per the subject line​:

```
DYLD_LIBRARY_PATH=/private/tmp/perl20151121\-6532\-elg51d/perl\-5.22.0 ./runtests choose
dyld​: Library not loaded​: /usr/local/Cellar/perl/5.22.0/lib/5.22.0/darwin-thread-multi-2level/CORE/libperl.dylib
  Referenced from​: /private/tmp/perl20151121-6532-elg51d/perl-5.22.0/t/./perl
  Reason​: image not found
./runtests​: line 70​: 26701 Trace/BPT trap​: 5 ./perl $TESTFILE $TEST_ARGS $TEST_FILES < /dev/tty
make​: *** [test] Error 133
```

OS X El Capitan has a new security feature where the DYLD_LIBRARY_PATH isn't passed down to a child process, which is probably the root cause of the failure here.

If you run​:

```
DYLD_LIBRARY_PATH=/private/tmp/perl20151121\-6532\-elg51d/perl\-5.22.0 env | grep DYLD
```

As a test for example, on OS X Mavericks you see​:

```
DYLD_LIBRARY_PATH=/private/tmp/perl20151121-6532-elg51d/perl-5.22.0
```

On OS X El Capitan you get nothing.

This has been reported in various other projects as well​:
* http​://postgresql.nabble.com/OS-X-El-Capitan-and-DYLD-LIBRARY-PATH-td5869963.html
* https://www.mail-archive.com/pgsql-hackers@postgresql.org/msg272269.html
* https://forum.qt.io/topic/59439/dyld-library-not-loaded-when-debugging-but-ok-when-running-on-os-x-el-capitan-solved/2

[p5pRT]

From @craigberry

On Sat, Nov 21, 2015 at 3​:31 PM, Dominyk Tiller
<perlbug-followup@​perl.org> wrote​:

# New Ticket Created by Dominyk Tiller
# Please include the string​: [perl #126706]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=126706 >

As per the subject line​:

```
DYLD_LIBRARY_PATH=/private/tmp/perl20151121\-6532\-elg51d/perl\-5.22.0 ./runtests choose
dyld​: Library not loaded​: /usr/local/Cellar/perl/5.22.0/lib/5.22.0/darwin-thread-multi-2level/CORE/libperl.dylib
Referenced from​: /private/tmp/perl20151121-6532-elg51d/perl-5.22.0/t/./perl
Reason​: image not found
./runtests​: line 70​: 26701 Trace/BPT trap​: 5 ./perl $TESTFILE $TEST_ARGS $TEST_FILES < /dev/tty
make​: *** [test] Error 133
```

OS X El Capitan has a new security feature where the DYLD_LIBRARY_PATH isn't passed down to a child process, which is probably the root cause of the failure here.

I've done numerous builds of blead and maint-5.22 on El Capitan
without any problems, but then I've never had DYLD_LIBRARY_PATH in my
environment. Can you explain why you need it in yours?

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @tonycoz

On Mon Nov 23 10​:18​:47 2015, craig.a.berry@​gmail.com wrote​:

I've done numerous builds of blead and maint-5.22 on El Capitan
without any problems,

I see the error with a -Duseshrplib build​:

TESTFILE=harness DYLD_LIBRARY_PATH=/Users/tony/dev/perl/git/perl ./runtests choose
dyld​: Library not loaded​: /usr/local/lib/perl5/5.23.6/darwin-thread-multi-2level/CORE/libperl.dylib
  Referenced from​: /Users/tony/dev/perl/git/perl/t/./perl
  Reason​: image not found
./runtests​: line 70​: 34725 Trace/BPT trap​: 5 ./perl $TESTFILE $TEST_ARGS $TEST_FILES < /dev/tty
make​: *** [test_harness] Error 133

but then I've never had DYLD_LIBRARY_PATH in my
environment. Can you explain why you need it in yours?

Configure, hints/darwin.sh and Makefile.SH conspire together to set it on darwin.

It's needed for -Duseshrplib builds, which isn't the default on Darwin, but it useful for mod_perl and other embedded perls.

The attached fixes the most immediate errors, but many more tests continue to fail due to this and they'll need to be tracked down individually if we want to support this.

Tony

[p5pRT]

From @tonycoz

0001-perl-126706-make-DYLD_LIBRARY_PATH-more-visible-on-D.patch

From 3964d88ca2b4c06f3034add5c21a7e7b53c8ea38 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Wed, 16 Dec 2015 16:28:12 +1100
Subject: [perl #126706] make DYLD_LIBRARY_PATH more visible on Darwin

On OS X 10.10, by default, system binaries, including the shell, do
not inherit DYLD_* enviroment variables.

The shell does however pass that variable onto its children, so any
place we need that visible during testing will need to set it
explicitly.
---
 runtests.SH | 8 ++++++++
 t/test.pl   | 8 ++++++++
 2 files changed, 16 insertions(+)

diff --git a/runtests.SH b/runtests.SH
index 71f41d1..feee19c 100755
--- a/runtests.SH
+++ b/runtests.SH
@@ -87,6 +87,14 @@ esac
 
 !NO!SUBS!
 
+case "$ldlibpthname" in
+    '') ;;
+    *) $spitshell >>runtests <<!GROK!THIS ;;
+$ldlibpthname=$PWD
+export $ldlibpthname
+!GROK!THIS
+esac
+
 ## In the following, dollars and backticks do need the extra backslash.
 $spitshell >>runtests <<!GROK!THIS!
 # The second branch is for testing without a tty or controlling terminal,
diff --git a/t/test.pl b/t/test.pl
index cda3840..42d96ea 100644
--- a/t/test.pl
+++ b/t/test.pl
@@ -640,6 +640,14 @@ sub _create_runperl { # Create the string to qx in runperl().
     if ($ENV{PERL_RUNPERL_DEBUG}) {
 	$runperl = "$ENV{PERL_RUNPERL_DEBUG} $runperl";
     }
+    if ($^O eq 'darwin') {
+	# OS X 10.10 drops DYLD_LIBRARY_PATH when inherited by system
+	# processes, including the shell, make sure the child perl
+	# still sees it
+	if (eval { require Config; 1 } && (my $ldlibpthname = $Config::Config{ldlibpthname})) {
+	    $runperl="$ldlibpthname=$ENV{PWD}/.. $runperl";
+	}
+    }
     unless ($args{nolib}) {
 	$runperl = $runperl . ' "-I../lib"'; # doublequotes because of VMS
     }
-- 
2.5.4 (Apple Git-61)

[p5pRT]

From @tonycoz

On Tue, 15 Dec 2015 21​:37​:28 -0800, tonyc wrote​:

The attached fixes the most immediate errors, but many more tests
continue to fail due to this and they'll need to be tracked down
individually if we want to support this.

The attached fixes tests in t/

For cpan/ and dist/ (and probably ext/) it might be useful to make something like which_perl() and which_perl_shell() visible.

Tony

[p5pRT]

From @tonycoz

0001-perl-126706-ensure-DYLD_LIBRARY_PATH-is-set-for-t-te.patch

From 379dbef097618c73e0c1e8528cb8623c9b7d8373 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Wed, 22 Feb 2017 16:03:45 +1100
Subject: (perl #126706) ensure DYLD_LIBRARY_PATH is set for t/ tests on OS X

On OS X 10.10, by default, system binaries, including the shell, do
not inherit DYLD_* enviroment variables.

So when starting a new child perl via the shell on darwin, ensure it's
set.

This fixes only core tests in t/
---
 t/base/term.t       | 12 +++++++++---
 t/io/dup.t          |  3 ++-
 t/io/fflush.t       |  2 +-
 t/io/open.t         |  2 +-
 t/io/openpid.t      |  2 +-
 t/io/through.t      |  2 +-
 t/op/alarm.t        |  2 +-
 t/op/die_exit.t     |  6 ++++--
 t/op/exec.t         |  7 ++++---
 t/op/fork.t         | 15 ++++++++++-----
 t/op/magic.t        |  7 +------
 t/op/srand.t        |  5 +++--
 t/op/taint.t        |  7 +++++++
 t/porting/authors.t |  4 +++-
 t/run/cloexec.t     |  2 +-
 t/run/fresh_perl.t  |  6 +++++-
 t/run/script.t      |  2 +-
 t/run/switcht.t     |  2 +-
 t/test.pl           | 28 ++++++++++++++++++++++++++--
 19 files changed, 82 insertions(+), 34 deletions(-)

diff --git a/t/base/term.t b/t/base/term.t
index 7a16ffd..af35b8f 100644
--- a/t/base/term.t
+++ b/t/base/term.t
@@ -16,9 +16,15 @@ elsif ($x eq chr(21)) { print "ok 1 # EBCDIC\n"; }
 else {print "not ok 1\n";}
 
 # check `` processing
-
-$x = `$^X -le "print 'hi there'"`;
-if ($x eq "hi there\n") {print "ok 2\n";} else {print "not ok 2\n";}
+if ($^O ne 'darwin') {
+    $x = `$^X -le "print 'hi there'"`;
+    if ($x eq "hi there\n") {print "ok 2\n";} else {print "not ok 2\n";}
+}
+else {
+    # the shell doesn't inherit DLYD_LIBRARY_PATH on modern Darwin
+    # and this is too early to mess with %ENV
+    print "ok 2 # SKIP need Config to reliably start a child perl here\n";
+}
 
 # check $#array
 
diff --git a/t/io/dup.t b/t/io/dup.t
index 8a8b27e..c7631d6 100644
--- a/t/io/dup.t
+++ b/t/io/dup.t
@@ -29,7 +29,8 @@ print STDOUT "ok 2\n";
 print STDERR "ok 3\n";
 
 # Since some systems don't have echo, we use Perl.
-$echo = qq{$^X -le "print q(ok %d)"};
+my $Perl = which_perl_shell();
+$echo = qq{$Perl -le "print q(ok %d)"};
 
 $cmd = sprintf $echo, 4;
 print `$cmd`;
diff --git a/t/io/fflush.t b/t/io/fflush.t
index 8e89ebb..053b743 100644
--- a/t/io/fflush.t
+++ b/t/io/fflush.t
@@ -27,7 +27,7 @@ skip_all('fflush(NULL) or equivalent not available')
 
 plan(tests => 7);
 
-my $runperl = $^X =~ m/\s/ ? qq{"$^X"} : $^X;
+my $runperl = which_perl_shell();
 $runperl .= qq{ "-I../lib"};
 
 sub file_eq {
diff --git a/t/io/open.t b/t/io/open.t
index 6be9f0e..59c4556 100644
--- a/t/io/open.t
+++ b/t/io/open.t
@@ -12,7 +12,7 @@ use Config;
 
 plan tests => 156;
 
-my $Perl = which_perl();
+my $Perl = which_perl_shell();
 
 my $afile = tempfile();
 {
diff --git a/t/io/openpid.t b/t/io/openpid.t
index d3fcf78..ccc9860 100644
--- a/t/io/openpid.t
+++ b/t/io/openpid.t
@@ -25,7 +25,7 @@ $| = 1;
 $SIG{PIPE} = 'IGNORE';
 $SIG{HUP} = 'IGNORE' if $^O eq 'interix';
 
-my $perl = which_perl();
+my $perl = which_perl_shell();
 $perl .= qq[ "-I../lib"];
 
 #
diff --git a/t/io/through.t b/t/io/through.t
index 65a64bb..6f24b01 100644
--- a/t/io/through.t
+++ b/t/io/through.t
@@ -10,7 +10,7 @@ BEGIN {
 
 use strict;
 
-my $Perl = which_perl();
+my $Perl = which_perl_shell();
 
 my $data = <<'EOD';
 x
diff --git a/t/op/alarm.t b/t/op/alarm.t
index 749482c..c2d0594 100644
--- a/t/op/alarm.t
+++ b/t/op/alarm.t
@@ -13,7 +13,7 @@ if ( !$Config{d_alarm} ) {
 }
 
 plan tests => 5;
-my $Perl = which_perl();
+my $Perl = which_perl_shell();
 
 my ($start_time, $end_time);
 
diff --git a/t/op/die_exit.t b/t/op/die_exit.t
index e074913..d9c76ad 100644
--- a/t/op/die_exit.t
+++ b/t/op/die_exit.t
@@ -60,14 +60,16 @@ if ($^O eq 'VMS') {
 my $tempfile = tempfile();
 open STDERR, '>', $tempfile or die "Can't open temp error file $tempfile:  $!";
 
+my $ShPerl = which_perl_shell();
+
 foreach my $test (@tests) {
     my($bang, $query, $code) = @$test;
     $code ||= 'die;';
     if ($^O eq 'MSWin32' || $^O eq 'NetWare' || $^O eq 'VMS') {
-        system(qq{$^X -e "\$! = $bang; \$? = $query; $code"});
+        system(qq{$ShPerl -e "\$! = $bang; \$? = $query; $code"});
     }
     else {
-        system(qq{$^X -e '\$! = $bang; \$? = $query; $code'});
+        system(qq{$ShPerl -e '\$! = $bang; \$? = $query; $code'});
     }
     my $exit = $?;
 
diff --git a/t/op/exec.t b/t/op/exec.t
index 1155439..5e3b959 100644
--- a/t/op/exec.t
+++ b/t/op/exec.t
@@ -38,7 +38,8 @@ my $Is_Win32 = $^O eq 'MSWin32';
 
 plan(tests => 25);
 
-my $Perl = which_perl();
+my $ListPerl = which_perl();
+my $Perl = which_perl_shell();
 
 my $exit;
 SKIP: {
@@ -59,7 +60,7 @@ is( $exit, 0, '  exited 0' );
 # On Unix its the opposite.
 my $quote = $Is_VMS || $Is_Win32 ? '"' : '';
 $tnum = curr_test();
-$exit = system $Perl, '-le', 
+$exit = system $ListPerl, '-le', 
                "${quote}print q{ok $tnum - system(PROG, LIST)}${quote}";
 next_test();
 is( $exit, 0, '  exited 0' );
@@ -157,5 +158,5 @@ TODO: {
 }
 
 my $test = curr_test();
-exec $Perl, '-le', qq{${quote}print 'ok $test - exec PROG, LIST'${quote}};
+exec $ListPerl, '-le', qq{${quote}print 'ok $test - exec PROG, LIST'${quote}};
 fail("This should never be reached if the exec() worked");
diff --git a/t/op/fork.t b/t/op/fork.t
index b69a929..444392b 100644
--- a/t/op/fork.t
+++ b/t/op/fork.t
@@ -28,8 +28,13 @@ SKIP: {
     skip "Can't set ulimit -u on this system: $probe"
 	unless $probe eq 'good';
 
+    my $perl = which_perl();
+    my $env = "";
+    if ($^O eq "darwin") {
+        $env = "DYLD_LIBRARY_PATH=$ENV{DYLD_LIBRARY_PATH} ";
+    }
     my $out = qx{
-        $shell -c 'ulimit -u 1; exec $^X -e "
+        $shell -c 'ulimit -u 1; ${env}exec $perl -e "
             print((() = fork) == 1 ? q[ok] : q[not ok])
         "'
     };
@@ -253,10 +258,10 @@ $| = 1;
 $\ = "\n";
 my $getenv;
 if ($^O eq 'MSWin32' || $^O eq 'NetWare') {
-    $getenv = qq[$^X -e "print \$ENV{TST}"];
+    $getenv = qq[$ShPerl -e "print \$ENV{TST}"];
 }
 else {
-    $getenv = qq[$^X -e 'print \$ENV{TST}'];
+    $getenv = qq[$ShPerl -e 'print \$ENV{TST}'];
 }
 $ENV{TST} = 'foo';
 if (fork) {
@@ -477,13 +482,13 @@ child: called as [main::f(foo,bar)]
 waitpid() returned ok
 ########
 # Windows 2000: https://rt.cpan.org/Ticket/Display.html?id=66016#txn-908976
-system $^X,  "-e", "if (\$pid=fork){sleep 1;kill(9, \$pid)} else {sleep 5}";
+system $Perl,  "-e", "if (\$pid=fork){sleep 1;kill(9, \$pid)} else {sleep 5}";
 print $?>>8, "\n";
 EXPECT
 0
 ########
 # Windows 7: https://rt.cpan.org/Ticket/Display.html?id=66016#txn-908976
-system $^X,  "-e", "if (\$pid=fork){kill(9, \$pid)} else {sleep 5}";
+system $Perl,  "-e", "if (\$pid=fork){kill(9, \$pid)} else {sleep 5}";
 print $?>>8, "\n";
 EXPECT
 0
diff --git a/t/op/magic.t b/t/op/magic.t
index 3f71f8e..60bb842 100644
--- a/t/op/magic.t
+++ b/t/op/magic.t
@@ -56,12 +56,7 @@ $Is_Dos      = $^O eq 'dos';
 $Is_os2      = $^O eq 'os2';
 $Is_Cygwin   = $^O eq 'cygwin';
 
-$PERL =
-   ($Is_NetWare ? 'perl'   :
-    $Is_VMS     ? $^X      :
-    $Is_MSWin32 ? '.\perl' :
-                  './perl');
-
+$PERL = which_perl_shell();
 
 sub env_is {
     my ($key, $val, $desc) = @_;
diff --git a/t/op/srand.t b/t/op/srand.t
index 09de60a..cfac4d5 100644
--- a/t/op/srand.t
+++ b/t/op/srand.t
@@ -52,9 +52,10 @@ ok( !eq_array(\@first_run, \@second_run),
 }
 
 # This test checks whether Perl called srand for you.
-@first_run  = `$^X -le "print int rand 100 for 1..100"`;
+my $perl = which_perl_shell();
+@first_run  = `$perl -le "print int rand 100 for 1..100"`;
 sleep(1); # in case our srand() is too time-dependent
-@second_run = `$^X -le "print int rand 100 for 1..100"`;
+@second_run = `$perl -le "print int rand 100 for 1..100"`;
 
 ok( !eq_array(\@first_run, \@second_run), 'srand() called automatically');
 
diff --git a/t/op/taint.t b/t/op/taint.t
index c13eaf6..08cd5af 100644
--- a/t/op/taint.t
+++ b/t/op/taint.t
@@ -55,6 +55,13 @@ my $Invoke_Perl = $Is_VMS      ? 'MCR Sys$Disk:[]Perl.exe' :
                                  './perl'               ;
 my @MoreEnv = qw/IFS CDPATH ENV BASH_ENV/;
 
+if ($^O eq "darwin") {
+    # all the tests here invoke perl through a shell, and the system shell
+    # on darwin doesn't inherit DYLD_LIBRARY_PATH...
+    my ($lib_path) = ($ENV{DYLD_LIBRARY_PATH} =~ /(.*)/);
+    $Invoke_Perl = "DYLD_LIBRARY_PATH=$lib_path $Invoke_Perl";
+}
+
 if ($Is_VMS) {
     my (%old, $x);
     for $x ('DCL$PATH', @MoreEnv) {
diff --git a/t/porting/authors.t b/t/porting/authors.t
index 563b92a..45c4e72 100644
--- a/t/porting/authors.t
+++ b/t/porting/authors.t
@@ -10,8 +10,10 @@ use strict;
 require './t/test.pl';
 find_git_or_skip('all');
 
+my $ShPerl = which_perl_shell();
+
 # This is the subset of "pretty=fuller" that checkAUTHORS.pl actually needs:
 my $quote = $^O =~ /^mswin/i ? q(") : q(');
-system("git log --pretty=format:${quote}Author: %an <%ae>%n${quote} | $^X Porting/checkAUTHORS.pl --tap -");
+system("git log --pretty=format:${quote}Author: %an <%ae>%n${quote} | $ShPerl Porting/checkAUTHORS.pl --tap -");
 
 # EOF
diff --git a/t/run/cloexec.t b/t/run/cloexec.t
index f767267..9d27ed9 100644
--- a/t/run/cloexec.t
+++ b/t/run/cloexec.t
@@ -55,7 +55,7 @@ sub make_tmp_file {
     close  FHTMP             or die "close '$fname': $!";
 }
 
-my $Perl = which_perl();
+my $Perl = which_perl_shell();
 my $quote = "'";
 
 my $tmperr             = tempfile();
diff --git a/t/run/fresh_perl.t b/t/run/fresh_perl.t
index 411ff04..ef05391 100644
--- a/t/run/fresh_perl.t
+++ b/t/run/fresh_perl.t
@@ -16,7 +16,7 @@ BEGIN {
 
 use strict;
 
-my $Perl = which_perl();
+my $Perl = which_perl_shell();
 
 $|=1;
 
@@ -41,6 +41,10 @@ foreach my $prog (@prgs) {
 
     my($prog,$expected) = split(/\nEXPECT\n/, $raw_prog);
     $prog .= "\n";
+
+    # this might break new tests, but there shouldn't be any new tests...
+    $prog =~ s(\./perl\b)($Perl)g;
+
     $expected = '' unless defined $expected;
 
     if ($prog =~ /^\# SKIP: (.+)/m) {
diff --git a/t/run/script.t b/t/run/script.t
index fa61a2c..066deb0 100644
--- a/t/run/script.t
+++ b/t/run/script.t
@@ -7,7 +7,7 @@ BEGIN {
     plan(3);
 }
 
-my $Perl = which_perl();
+my $Perl = which_perl_shell();
 
 my $filename = tempfile();
 
diff --git a/t/run/switcht.t b/t/run/switcht.t
index 01b9f2f..53e8591 100644
--- a/t/run/switcht.t
+++ b/t/run/switcht.t
@@ -8,7 +8,7 @@ BEGIN {
 
 plan tests => 13;
 
-my $Perl = which_perl();
+my $Perl = which_perl_shell();
 
 my $warning;
 local $SIG{__WARN__} = sub { $warning = join "\n", @_; };
diff --git a/t/test.pl b/t/test.pl
index b236103..a4b35fc 100644
--- a/t/test.pl
+++ b/t/test.pl
@@ -710,12 +710,13 @@ sub _create_runperl { # Create the string to qx in runperl().
 	$args{stdin} =~ s/\n/\\n/g;
 	$args{stdin} =~ s/\r/\\r/g;
 
+	my $shell_perl = which_perl_shell();
 	if ($is_mswin || $is_netware || $is_vms) {
-	    $runperl = qq{$Perl -e "print qq(} .
+	    $runperl = qq{$shell_perl -e "print qq(} .
 		$args{stdin} . q{)" | } . $runperl;
 	}
 	else {
-	    $runperl = qq{$Perl -e 'print qq(} .
+	    $runperl = qq{$shell_perl -e 'print qq(} .
 		$args{stdin} . q{)' | } . $runperl;
 	}
     } elsif (exists $args{stdin}) {
@@ -861,6 +862,25 @@ sub which_perl {
     return $Perl;
 }
 
+# A perl executable name suitable for use when run through the shell.
+# Needed because modern darwin system shells drop DYLD_LIBRARY_PATH
+sub which_perl_shell {
+    my $perl = which_perl();
+    if ($perl =~ m/\s/) {
+        $perl = qq{"$perl"};
+    }
+    if ($^O eq 'darwin') {
+	# OS X 10.10 drops DYLD_LIBRARY_PATH when inherited by system
+	# processes, including the shell, make sure the child perl
+	# still sees it
+	if (eval { require Config; 1 } && (my $ldlibpthname = $Config::Config{ldlibpthname})) {
+	    $perl="$ldlibpthname=$ENV{PWD}/.. $perl";
+	}
+    }
+
+    return $perl;
+}
+
 sub unlink_all {
     my $count = 0;
     foreach my $file (@_) {
@@ -1276,6 +1296,10 @@ sub run_multiple_progs {
               or die "Can't dup STDOUT->STDERR: $!;";
         }
 	};
+	if ($prog =~ m/\$(?:Sh)?Perl\b/) {
+	    print $fh "my \$Perl = q\0", which_perl(), "\0;\n";
+	    print $fh "my \$ShPerl = q\0", which_perl_shell(), "\0;\n";
+	}
 	print $fh "\n#line 1\n";  # So the line numbers don't get messed up.
 	print $fh $prog,"\n";
 	close $fh or die "Cannot close $tmpfile: $!";
-- 
2.1.4

[p5pRT]

From @tonycoz

On Sat, 21 Nov 2015 13​:31​:31 -0800, dominyktiller@​gmail.com wrote​:

As per the subject line​:

```
DYLD_LIBRARY_PATH=/private/tmp/perl20151121\-6532\-elg51d/perl\-5.22.0
./runtests choose
dyld​: Library not loaded​:
/usr/local/Cellar/perl/5.22.0/lib/5.22.0/darwin-thread-multi-
2level/CORE/libperl.dylib
Referenced from​: /private/tmp/perl20151121-6532-elg51d/perl-
5.22.0/t/./perl
Reason​: image not found
./runtests​: line 70​: 26701 Trace/BPT trap​: 5 ./perl $TESTFILE
$TEST_ARGS $TEST_FILES < /dev/tty
make​: *** [test] Error 133
```

OS X El Capitan has a new security feature where the DYLD_LIBRARY_PATH
isn't passed down to a child process, which is probably the root cause
of the failure here.

If you run​:

```
DYLD_LIBRARY_PATH=/private/tmp/perl20151121\-6532\-elg51d/perl\-5.22.0
env | grep DYLD
```

As a test for example, on OS X Mavericks you see​:

```
DYLD_LIBRARY_PATH=/private/tmp/perl20151121-6532-elg51d/perl-5.22.0
```

On OS X El Capitan you get nothing.

This has been reported in various other projects as well​:
* http​://postgresql.nabble.com/OS-X-El-Capitan-and-DYLD-LIBRARY-PATH-
td5869963.html
* https://www.mail-archive.com/pgsql-
hackers@​postgresql.org/msg272269.html
* https://forum.qt.io/topic/59439/dyld-library-not-loaded-when-
debugging-but-ok-when-running-on-os-x-el-capitan-solved/2

Please try the attached.

One test currently fails.

Tony

[p5pRT]

From @tonycoz

0001-perl-127606-use-rpath-to-locate-libperl.dylib-on-OS-.patch

From 1d36fa20acb0be4d1f90e8d42c29745afdcb3168 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Thu, 4 Oct 2018 14:41:03 +1000
Subject: (perl #127606) use rpath to locate libperl.dylib on OS X with SIP

SIP (System Integrity Protection) on OS X prevents the DYLD_LIBRARY_PATH
environment variable from being propagated through /bin/sh

This causes many tests to fail (and some more recent build issues.)

To avoid that, we change the way libperl.dylib is linked to perl, so
it searches an RPATH stored in the shared object, and that @rpath
name is then propagated to the executable.

During installation we modify the executable and library to use the
installed library path for the library so it can be found in the installed
directory.
---
 Configure   |  3 +++
 Makefile.SH | 38 ++++++++++++++++++++++++++++++++++++++
 installperl | 20 ++++++++++++++++++++
 3 files changed, 61 insertions(+)

diff --git a/Configure b/Configure
index 127583c..cb21bfb 100755
--- a/Configure
+++ b/Configure
@@ -8816,6 +8816,9 @@ if "$useshrplib"; then
 	cygwin)
 		# cygwin needs only ldlibpth
 		;;
+	darwin)
+		# darwin we set it at installation time
+		;;
 	*)
 		tmp_shrpenv="env LD_RUN_PATH=$shrpdir"
 		;;
diff --git a/Makefile.SH b/Makefile.SH
index 18ebd4e..8ccd178 100755
--- a/Makefile.SH
+++ b/Makefile.SH
@@ -62,6 +62,21 @@ true)
 		-compatibility_version 1 -current_version $patchlevel \
 		-prebind -seg1addr 0x27000000 -install_name \$(shrpdir)/\$@"
 		;;
+	darwin1[5-9]*|darwin[2-9]*)
+	        # from OS X 10.11 (darwin 15+), System Integrity Protection
+	        # prevents DYLD_LIBRARY_PATH from being passed to child processes
+	        # through the shell, so we build to find libperl in the build
+	        # directory, then fix the path during installation
+		shrpldflags="${ldflags} -dynamiclib \
+                            -compatibility_version \
+				${api_revision}.${api_version}.${api_subversion} \
+			     -current_version \
+				${revision}.${patchlevel}.${subversion} \
+			     -install_name @rpath/\$@ -Xlinker -headerpad_max_install_names \
+                             -Xlinker -rpath -Xlinker ` quote "$pwd" `"
+		exeldflags="-Xlinker -rpath -Xlinker ` quote "$pwd" `  -Xlinker -headerpad_max_install_names"
+		#linklibperl="-Xlinker @rpath/libperl.dylib"
+		;;
 	darwin*)
 		shrpldflags="${ldflags} -dynamiclib \
                             -compatibility_version \
@@ -69,6 +84,7 @@ true)
 			     -current_version \
 				${revision}.${patchlevel}.${subversion} \
 			     -install_name \$(shrpdir)/\$@"
+		exeldflags=""
 		;;
 	cygwin*)
 		shrpldflags="$shrpldflags -Wl,--out-implib=libperl.dll.a -Wl,--image-base,0x52000000"
@@ -339,6 +355,14 @@ MANIFEST_SRT = MANIFEST.srt
 
 !GROK!THIS!
 
+case "$useshrplib$osname" in
+truedarwin)
+	$spitshell >>$Makefile <<!GROK!THIS!
+PERL_EXE_LDFLAGS=$exeldflags
+!GROK!THIS!
+	;;
+esac
+
 case "$usecrosscompile$perl" in
 define?*)
 	$spitshell >>$Makefile <<!GROK!THIS!
@@ -1050,6 +1074,20 @@ $(PERL_EXE): $& $(perlmain_dep) $(LIBPERL) $(static_ext) ext.libs $(PERLEXPORT)
 	$(SHRPENV) $(CC) -o perl $(CLDFLAGS) $(CCDLFLAGS) $(perlmain_objs) $(LLIBPERL) $(static_ext) `cat ext.libs` $(libs)
 !NO!SUBS!
         ;;
+
+	darwin)
+	    case "$useshrplib$osvers" in
+	    true1[5-9]*|true[2-9]*) $spitshell >>$Makefile <<'!NO!SUBS!'
+	$(SHRPENV) $(CC) -o perl $(PERL_EXE_LDFLAGS) $(CLDFLAGS) $(CCDLFLAGS) $(perlmain_objs) $(static_ext) $(LLIBPERL) `cat ext.libs` $(libs)
+!NO!SUBS!
+	       ;;
+	    *) $spitshell >>$Makefile <<'!NO!SUBS!'
+	$(SHRPENV) $(CC) -o perl $(CLDFLAGS) $(CCDLFLAGS) $(perlmain_objs) $(static_ext) $(LLIBPERL) `cat ext.libs` $(libs)
+!NO!SUBS!
+	       ;;
+	    esac
+        ;;
+
         *) $spitshell >>$Makefile <<'!NO!SUBS!'
 	$(SHRPENV) $(CC) -o perl $(CLDFLAGS) $(CCDLFLAGS) $(perlmain_objs) $(static_ext) $(LLIBPERL) `cat ext.libs` $(libs)
 !NO!SUBS!
diff --git a/installperl b/installperl
index 3bf79d2..1223d89 100755
--- a/installperl
+++ b/installperl
@@ -304,6 +304,7 @@ elsif ($^O ne 'dos') {
 	safe_unlink("$installbin/$perl_verbase$ver$exe_ext");
 	copy("perl$exe_ext", "$installbin/$perl_verbase$ver$exe_ext");
 	strip("$installbin/$perl_verbase$ver$exe_ext");
+	fix_dep_names("$installbin/$perl_verbase$ver$exe_ext");
 	chmod(0755, "$installbin/$perl_verbase$ver$exe_ext");
     }
     else {
@@ -388,6 +389,7 @@ foreach my $file (@corefiles) {
     if (copy_if_diff($file,"$installarchlib/CORE/$file")) {
 	if ($file =~ /\.(\Q$so\E|\Q$dlext\E)$/) {
 	    strip("-S", "$installarchlib/CORE/$file") if $^O eq 'darwin';
+	    fix_dep_names("$installarchlib/CORE/$file");
 	    chmod($SO_MODE, "$installarchlib/CORE/$file");
 	} else {
 	    chmod($NON_SO_MODE, "$installarchlib/CORE/$file");
@@ -791,4 +793,22 @@ sub strip
     }
 }
 
+sub fix_dep_names {
+    my $file = shift;
+
+    $^O eq "darwin" && $Config{osvers} =~ /^(1[5-9]|[2-9])/
+      && $Config{useshrplib}
+      or return;
+
+    my @opts =
+	  (
+	   "-rpath", getcwd, "$Config{archlibexp}/CORE/",
+	   $file,
+	  );
+
+    $opts{verbose} and print "  install_name_tool @opts\n";
+    system "install_name_tool", @opts
+      and die "Cannot update $file dependency paths\n";
+}
+
 # ex: set ts=8 sts=4 sw=4 et:
-- 
2.8.4 (Apple Git-73)

[p5pRT]

From @tonycoz

On Wed, 03 Oct 2018 22​:51​:15 -0700, tonyc wrote​:

On Sat, 21 Nov 2015 13​:31​:31 -0800, dominyktiller@​gmail.com wrote​:

As per the subject line​:

```
DYLD_LIBRARY_PATH=/private/tmp/perl20151121\-6532\-elg51d/perl\-5.22.0
./runtests choose
dyld​: Library not loaded​:
/usr/local/Cellar/perl/5.22.0/lib/5.22.0/darwin-thread-multi-
2level/CORE/libperl.dylib
Referenced from​: /private/tmp/perl20151121-6532-elg51d/perl-
5.22.0/t/./perl
Reason​: image not found
./runtests​: line 70​: 26701 Trace/BPT trap​: 5 ./perl $TESTFILE
$TEST_ARGS $TEST_FILES < /dev/tty
make​: *** [test] Error 133
```

OS X El Capitan has a new security feature where the DYLD_LIBRARY_PATH
isn't passed down to a child process, which is probably the root cause
of the failure here.

If you run​:

```
DYLD_LIBRARY_PATH=/private/tmp/perl20151121\-6532\-elg51d/perl\-5.22.0
env | grep DYLD
```

As a test for example, on OS X Mavericks you see​:

```
DYLD_LIBRARY_PATH=/private/tmp/perl20151121-6532-elg51d/perl-5.22.0
```

On OS X El Capitan you get nothing.

This has been reported in various other projects as well​:
* http​://postgresql.nabble.com/OS-X-El-Capitan-and-DYLD-LIBRARY-PATH-
td5869963.html
* https://www.mail-archive.com/pgsql-
hackers@​postgresql.org/msg272269.html
* https://forum.qt.io/topic/59439/dyld-library-not-loaded-when-
debugging-but-ok-when-running-on-os-x-el-capitan-solved/2

Please try the attached.

One test currently fails.

Tony

This one is better.

The original patch made the libperl path rpath relative, but that confused the embedding test build, and would likely have caused similar problems with production embedding, such as with mod_perl2.

The new patch uses absolute paths, adjusting them on installation, which fixes the embedding case.

Tony

[p5pRT]

From @tonycoz

0001-perl-127606-adjust-dependency-paths-on-installation-.patch

From ff9dd435d676c1f00fd5f2516fbf0f9a13939bb3 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Thu, 4 Oct 2018 14:41:03 +1000
Subject: (perl #127606) adjust dependency paths on installation on darwin

SIP (System Integrity Protection) on OS X prevents the
DYLD_LIBRARY_PATH environment variable from being propagated through
/bin/sh, causes many tests to fail (and some more recent build issues)
for -Duseshrplib builds.

To avoid that, we change the way libperl.dylib is linked to perl, so
for the initial build the library's id is at the build location rather
than the install location, and the generated executable also expects
to find libperl in that location.

This obviously won't work once we copy both to the installation
directory, so we adjust both the id of the library and the dependency
path in the executable to point to the new location of the library.

A previous attempt set -rpath and used @rpath in the id, but this made
the embedding test fail.
---
 Makefile.SH | 34 ++++++++++++++++++++++++++++++++--
 installperl | 25 +++++++++++++++++++++++++
 2 files changed, 57 insertions(+), 2 deletions(-)

diff --git a/Makefile.SH b/Makefile.SH
index 18ebd4e907..417c020142 100755
--- a/Makefile.SH
+++ b/Makefile.SH
@@ -67,8 +67,16 @@ true)
                             -compatibility_version \
 				${api_revision}.${api_version}.${api_subversion} \
 			     -current_version \
-				${revision}.${patchlevel}.${subversion} \
-			     -install_name \$(shrpdir)/\$@"
+				${revision}.${patchlevel}.${subversion}"
+		case "$osvers" in
+	        1[5-9]*|[2-9]*)
+			shrpldflags="$shrpldflags -install_name `pwd`/\$@ -Xlinker -headerpad_max_install_names"
+			exeldflags="-Xlinker -headerpad_max_install_names"
+			;;
+		*)
+			shrpldflags="$shrpldflags -install_name \$(shrpdir)/\$@"
+			;;
+		esac
 		;;
 	cygwin*)
 		shrpldflags="$shrpldflags -Wl,--out-implib=libperl.dll.a -Wl,--image-base,0x52000000"
@@ -339,6 +347,14 @@ MANIFEST_SRT = MANIFEST.srt
 
 !GROK!THIS!
 
+case "$useshrplib$osname" in
+truedarwin)
+	$spitshell >>$Makefile <<!GROK!THIS!
+PERL_EXE_LDFLAGS=$exeldflags
+!GROK!THIS!
+	;;
+esac
+
 case "$usecrosscompile$perl" in
 define?*)
 	$spitshell >>$Makefile <<!GROK!THIS!
@@ -1050,6 +1066,20 @@ $(PERL_EXE): $& $(perlmain_dep) $(LIBPERL) $(static_ext) ext.libs $(PERLEXPORT)
 	$(SHRPENV) $(CC) -o perl $(CLDFLAGS) $(CCDLFLAGS) $(perlmain_objs) $(LLIBPERL) $(static_ext) `cat ext.libs` $(libs)
 !NO!SUBS!
         ;;
+
+	darwin)
+	    case "$useshrplib$osvers" in
+	    true1[5-9]*|true[2-9]*) $spitshell >>$Makefile <<'!NO!SUBS!'
+	$(SHRPENV) $(CC) -o perl $(PERL_EXE_LDFLAGS) $(CLDFLAGS) $(CCDLFLAGS) $(perlmain_objs) $(static_ext) $(LLIBPERL) `cat ext.libs` $(libs)
+!NO!SUBS!
+	       ;;
+	    *) $spitshell >>$Makefile <<'!NO!SUBS!'
+	$(SHRPENV) $(CC) -o perl $(CLDFLAGS) $(CCDLFLAGS) $(perlmain_objs) $(static_ext) $(LLIBPERL) `cat ext.libs` $(libs)
+!NO!SUBS!
+	       ;;
+	    esac
+        ;;
+
         *) $spitshell >>$Makefile <<'!NO!SUBS!'
 	$(SHRPENV) $(CC) -o perl $(CLDFLAGS) $(CCDLFLAGS) $(perlmain_objs) $(static_ext) $(LLIBPERL) `cat ext.libs` $(libs)
 !NO!SUBS!
diff --git a/installperl b/installperl
index 3bf79d2d6f..6cd65a0923 100755
--- a/installperl
+++ b/installperl
@@ -304,6 +304,7 @@ elsif ($^O ne 'dos') {
 	safe_unlink("$installbin/$perl_verbase$ver$exe_ext");
 	copy("perl$exe_ext", "$installbin/$perl_verbase$ver$exe_ext");
 	strip("$installbin/$perl_verbase$ver$exe_ext");
+	fix_dep_names("$installbin/$perl_verbase$ver$exe_ext");
 	chmod(0755, "$installbin/$perl_verbase$ver$exe_ext");
     }
     else {
@@ -388,6 +389,7 @@ foreach my $file (@corefiles) {
     if (copy_if_diff($file,"$installarchlib/CORE/$file")) {
 	if ($file =~ /\.(\Q$so\E|\Q$dlext\E)$/) {
 	    strip("-S", "$installarchlib/CORE/$file") if $^O eq 'darwin';
+	    fix_dep_names("$installarchlib/CORE/$file");
 	    chmod($SO_MODE, "$installarchlib/CORE/$file");
 	} else {
 	    chmod($NON_SO_MODE, "$installarchlib/CORE/$file");
@@ -791,4 +793,27 @@ sub strip
     }
 }
 
+sub fix_dep_names {
+    my $file = shift;
+
+    $^O eq "darwin" && $Config{osvers} =~ /^(1[5-9]|[2-9])/
+      && $Config{useshrplib}
+      or return;
+
+    my @opts;
+    my $so = $Config{so};
+    my $libperl = "$Config{archlibexp}/CORE/libperl.$Config{so}";
+    if ($file =~ /\blibperl.\Q$Config{so}\E$/a) {
+        push @opts, -id => $libperl;
+    }
+    else {
+        push @opts, -change => getcwd . "/libperl.$so", $libperl;
+    }
+    push @opts, $file;
+
+    $opts{verbose} and print "  install_name_tool @opts\n";
+    system "install_name_tool", @opts
+      and die "Cannot update $file dependency paths\n";
+}
+
 # ex: set ts=8 sts=4 sw=4 et:
-- 
2.11.0

[p5pRT]

From @tonycoz

On Mon, 08 Oct 2018 22​:28​:39 -0700, tonyc wrote​:

This one is better.

The original patch made the libperl path rpath relative, but that
confused the embedding test build, and would likely have caused
similar problems with production embedding, such as with mod_perl2.

The new patch uses absolute paths, adjusting them on installation,
which fixes the embedding case.

Applied as 191f890.

Tony

[p5pRT]

@tonycoz - Status changed from 'open' to 'pending release'

[p5pRT]

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release today of Perl 5.30.0, this and 160 other issues have been
resolved.

Perl 5.30.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.30.0

If you find that the problem persists, feel free to reopen this ticket.

[p5pRT]

@khwilliamson - Status changed from 'pending release' to 'resolved'