Undefined dtrace symbols when compiling miniperl on current illumos

[p5pRT]

Migrated from rt.perl.org#122287 (status was 'resolved')

Searchable as RT122287$

[p5pRT]

From kris@shannon.id.au

When configuring with -Dusedtrace on current illumos miniperl fails to build​:

cc -L/usr/lib -L/usr/ccs/lib -L/opt/SunStudioExpress/prod/lib -L/lib -L/usr/local/lib -L/usr/gnu/lib -o miniperl \
  perlmini.o opmini.o miniperlmain.o gv.o toke.o perly.o pad.o regcomp.o dump.o util.o mg.o reentr.o mro.o keywords.o hv.o av.o run.o pp_hot.o sv.o pp.o scope.o pp_ctl.o pp_sys.o doop.o doio.o regexec.o utf8.o taint.o deb.o universal.o globals.o perlio.o perlapi.o numeric.o mathoms.o locale.o pp_pack.o pp_sort.o caretx.o -lsocket -lnsl -ldl -lm -lc -lsunmath
Undefined first referenced
symbol in file
__dtrace_perl___loaded__file pp_ctl.o
__dtraceenabled_perl___op__entry dump.o
__dtrace_perl___sub__entry pp_hot.o
__dtraceenabled_perl___phase__change perlmini.o
__dtraceenabled_perl___loaded__file pp_ctl.o
__dtrace_perl___phase__change perlmini.o
__dtraceenabled_perl___sub__return pp_hot.o
__dtrace_perl___op__entry dump.o
__dtraceenabled_perl___loading__file pp_ctl.o
__dtraceenabled_perl___sub__entry pp_hot.o
__dtrace_perl___sub__return pp_hot.o
__dtrace_perl___loading__file pp_ctl.o
ld​: fatal​: symbol referencing errors. No output written to miniperl

This is due to the call to dtrace -G in Makefile.SH failing​:

http​://perl5.git.perl.org/perl.git/blob/d7d4eceb184d8fc54797d661a6d5ff9a4a7048c3​:/Makefile.SH#l229

Running that command directly (without the /dev/null redirection) gives​:

dtrace​: failed to link script perldtrace.d​: No probe sites found for declared provider

I believe this ruby ticket ( https://bugs.ruby-lang.org/issues/8606 ) is the same problem.

[p5pRT]

From @tonycoz

On Sun Jul 13 19​:00​:06 2014, Kris.Shannon wrote​:

When configuring with -Dusedtrace on current illumos miniperl fails to
build​:

Does it matter which illumos distribution?

I'm downloading openindiana now, if it's likely to be different please let me know.

I suspect the first part of Richard's patch in https://rt-archive.perl.org/perl5/Ticket/Display.html?id=120120 might be a fix for it, but he didn't explain it, so I didn't apply it.

Tony

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @tonycoz

On Sun Jul 13 22​:47​:28 2014, tonyc wrote​:

I suspect the first part of Richard's patch in
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=120120 might be a fix for
it, but he didn't explain it, so I didn't apply it.

Kris had some trouble with perlbug-followup, and mailed me directly​:

Date​: Tue, 15 Jul 2014 14​:07​:26 +1000
From​: Kris Shannon <kris@​shannon.id.au>
To​: tony@​develop-help.com
Subject​: Re​: [perl #122287] Undefined dtrace symbols when compiling miniperl on current illumos

I tried replying to perlbug-followup@​perl.org and even tried using the
web interface but it seems to have gobbled up my responses which I've
copied below​:

On 15 July 2014 09​:45, Kris Shannon <kris@​shannon.id.au> wrote​:

On 14 July 2014 15​:47, Tony Cook via RT <perlbug-followup@​perl.org> wrote​:

On Sun Jul 13 19​:00​:06 2014, Kris.Shannon wrote​:

When configuring with -Dusedtrace on current illumos miniperl fails to
build​:

Does it matter which illumos distribution?

I'm downloading openindiana now, if it's likely to be different please let me know.

I'm using openindiana, but with a newer kernel compiled from the illumos tree.

I believe the openindiana hipster branch should be new enough to
tickle this - but I'm not 100% sure.

I suspect the first part of Richard's patch in https://rt-archive.perl.org/perl5/Ticket/Display.html?id=120120
+might be a fix for it, but he didn't explain it, so I didn't apply it.

That patch, or something like it is already in perl-5.20.0, which is
what I'm trying with.

If I edit the bit in Makefile.SH that I linked to to not run the
dtrace -G test but instead set dtrace_o and minidtrace_o
unconditionally then the build completes all the way to testing.

I do get one test failure from t/run/dtrace.t but I don't think it's
related to the build issue.

I also get a number of other test failures - but I'm going to do a
build without dtrace and see if they still fail.

---

On 15 July 2014 09​:56, Kris Shannon <kris@​shannon.id.au> wrote​:

On 15 July 2014 09​:45, Kris Shannon <kris@​shannon.id.au> wrote​:

On 14 July 2014 15​:47, Tony Cook via RT <perlbug-followup@​perl.org> wrote​:

On Sun Jul 13 19​:00​:06 2014, Kris.Shannon wrote​:
I suspect the first part of Richard's patch in https://rt-archive.perl.org/perl5/Ticket/Display.html?id=120120
+might be a fix for it, but he didn't explain it, so I didn't apply it.

That patch, or something like it is already in perl-5.20.0, which is
what I'm trying with.

Sorry, I didn't read your response carefully enough.

Yes. I've done something very much like Richard did in the first part
of his patch.

I'm not sure that's necessarily the right answer in general though.
I'm not sure whether the current check is supposed to be about systems
without dtrace, or about a difference between dtrace
versions/implementations.

[p5pRT]

From kris@shannon.id.au

On 14 July 2014 15​:47, Tony Cook via RT <perlbug-followup@​perl.org> wrote​:

On Sun Jul 13 19​:00​:06 2014, Kris.Shannon wrote​:

When configuring with -Dusedtrace on current illumos miniperl fails to
build​:

Does it matter which illumos distribution?

I'm downloading openindiana now, if it's likely to be different please let me know.

I'm using openindiana, but with a newer kernel compiled from the illumos tree.

I believe the openindiana hipster branch should be new enough to
tickle this - but I'm not 100% sure.

I suspect the first part of Richard's patch in https://rt-archive.perl.org/perl5/Ticket/Display.html?id=120120 might be a fix for it, but he didn't explain it, so I didn't apply it.

That patch, or something like it is already in perl-5.20.0, which is
what I'm trying with.

If I edit the bit in Makefile.SH that I linked to to not run the
dtrace -G test but instead set dtrace_o and minidtrace_o
unconditionally then the build completes all the way to testing.

I do get one test failure from t/run/dtrace.t but I don't think it's
related to the build issue.

I also get a number of other test failures - but I'm going to do a
build without dtrace and see if they still fail.

[p5pRT]

From kris@shannon.id.au

On 15 July 2014 09​:45, Kris Shannon <kris@​shannon.id.au> wrote​:

On 14 July 2014 15​:47, Tony Cook via RT <perlbug-followup@​perl.org> wrote​:

On Sun Jul 13 19​:00​:06 2014, Kris.Shannon wrote​:
I suspect the first part of Richard's patch in https://rt-archive.perl.org/perl5/Ticket/Display.html?id=120120 might be a fix for it, but he didn't explain it, so I didn't apply it.

That patch, or something like it is already in perl-5.20.0, which is
what I'm trying with.

Sorry, I didn't read your response carefully enough.

Yes. I've done something very much like Richard did in the first part
of his patch.

I'm not sure that's necessarily the right answer in general though.
I'm not sure whether the current check is supposed to be about systems
without dtrace, or about a difference between dtrace
versions/implementations.

[p5pRT]

From @tonycoz

On Mon Jul 14 23​:34​:38 2014, Kris.Shannon wrote​:

On 14 July 2014 15​:47, Tony Cook via RT <perlbug-followup@​perl.org>
wrote​:

On Sun Jul 13 19​:00​:06 2014, Kris.Shannon wrote​:

When configuring with -Dusedtrace on current illumos miniperl fails
to
build​:

Does it matter which illumos distribution?

I'm downloading openindiana now, if it's likely to be different
please let me know.

I'm using openindiana, but with a newer kernel compiled from the
illumos tree.

I believe the openindiana hipster branch should be new enough to
tickle this - but I'm not 100% sure.

It doesn't seem to be new enough.

Tony

[p5pRT]

From kris@shannon.id.au

On Wed Jul 16 20​:35​:48 2014, tonyc wrote​:

On Mon Jul 14 23​:34​:38 2014, Kris.Shannon wrote​:

I believe the openindiana hipster branch should be
new enough to tickle this - but I'm not 100% sure.

It doesn't seem to be new enough.

Tony

The current stable version of OmniOS has the new DTrace;
they have disabled dtrace in their system perl rather than
patch it.

On Mon Jul 14 23​:34​:46 2014, Kris.Shannon wrote​:

Yes. I've done something very much like Richard
did in the first part of his patch.

I'm not sure that's necessarily the right answer
in general though.
I'm not sure whether the current check is supposed
to be about systems without dtrace, or about a
difference between dtrace versions/implementations.

My analysis of the situation is that there are two major implementations of dtrace; one which requires the use of "dtrace -G" to produce an object file implementing the support for the custom dtrace probes (this is the original implementation in Solaris and derived OS's) and one where the dtrace machinery is included directly by the compiler toolchain (this is the case under OSX)

This difference is what the Makefile.SH snippet is trying to detect. In recent versions of illumos, the 'dtrace -G' command has been updated to fail unless it is passed an object file which uses the probes defined in the .d file; this causes the current Makefile.SH to assume it is running under an OSX like implementation causing link failures later on.

The Ruby bug that I originally linked to ( https://bugs.ruby-lang.org/issues/8606 ) ended up fixing this issue by going through all the normal dtrace steps on a dummy .d​:

  echo "provider conftest{ probe fire(); };" > conftest_provider.d
  dtrace -h -o conftest_provider.h -s conftest_provider.d >/dev/null 2>/dev/null
  cat >conftest.c <<_CONF
  #include "conftest_provider.h"
  int main(void){ CONFTEST_FIRE(); return 0; }
  _CONF
  $CC $CFLAGS -c -o conftest.o conftest.c
  $DTRACE -G -s conftest_provider.d conftest.o

This will work to distinguish between the two different implementations on both old and new illumos and on OSX.

[p5pRT]

From @tonycoz

On Mon Dec 15 16​:05​:08 2014, Kris.Shannon wrote​:

On Wed Jul 16 20​:35​:48 2014, tonyc wrote​:

On Mon Jul 14 23​:34​:38 2014, Kris.Shannon wrote​:

I believe the openindiana hipster branch should be
new enough to tickle this - but I'm not 100% sure.

It doesn't seem to be new enough.

Tony

The current stable version of OmniOS has the new DTrace;
they have disabled dtrace in their system perl rather than
patch it.

Sorry, I missed this response until recently.

I've managed to reproduce the failure on OmniOS.

The Ruby bug that I originally linked to ( https://bugs.ruby-
lang.org/issues/8606 ) ended up fixing this issue by going through all
the normal dtrace steps on a dummy .d​:

echo "provider conftest{ probe fire(); };" > conftest_provider.d
dtrace -h -o conftest_provider.h -s conftest_provider.d >/dev/null
2>/dev/null
cat >conftest.c <<_CONF
#include "conftest_provider.h"
int main(void){ CONFTEST_FIRE(); return 0; }
_CONF
$CC $CFLAGS -c -o conftest.o conftest.c
$DTRACE -G -s conftest_provider.d conftest.o

This will work to distinguish between the two different
implementations on both old and new illumos and on OSX.

Please try the attached.

Tony

[p5pRT]

From @tonycoz

0001-perl-122287-probe-in-Configure-whether-dtrace-builds.patch

From a9e9d27da185029932b4912637d4b9771655e73c Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Wed, 2 Mar 2016 13:34:38 +1100
Subject: [perl #122287] probe in Configure whether dtrace builds an object

---
 Configure        | 29 +++++++++++++++++++++++++++++
 Makefile.SH      | 11 +++++++----
 Porting/Glossary |  3 +++
 3 files changed, 39 insertions(+), 4 deletions(-)

diff --git a/Configure b/Configure
index 3d65fd2..f635b69 100755
--- a/Configure
+++ b/Configure
@@ -1317,6 +1317,7 @@ use64bitall=''
 use64bitint=''
 usecbacktrace=''
 dtrace=''
+dtraceobject=''
 usedtrace=''
 usefaststdio=''
 usekernprocpathname=''
@@ -8171,6 +8172,33 @@ EOM
 	esac
 done
 
+case "$usedtrace" in
+$define)
+	case "$dtraceobject" in
+	$define|true|[yY]*)
+		dtraceobject=$define
+		;;
+	' '|'')
+		$dtrace -h -s ../perldtrace.d -o perldtrace.h
+		$cat >try.c <<EOM
+#include "perldtrace.h"
+int main(void) {
+    PERL_LOADED_FILE("dummy");
+    return 0;
+}
+EOM
+		dtraceobject=$undef
+		if $cc -c -o try.o $optimize $ccflags try.c \
+                    && $dtrace -G -s ../perldtrace.d try.o ; then
+		        dtraceobject=$define
+			echo "Your dtrace builds an object file"
+		fi
+		$rm -f try.c try.o perldtrace.o
+		;;
+	*) dtraceobject=$undef ;;
+	esac
+esac
+
 : See if we want extra modules installed
 echo " "
 case "$extras" in
@@ -24711,6 +24739,7 @@ doublesize='$doublesize'
 drand01='$drand01'
 drand48_r_proto='$drand48_r_proto'
 dtrace='$dtrace'
+dtraceobject='$dtraceobject'
 dynamic_ext='$dynamic_ext'
 eagain='$eagain'
 ebcdic='$ebcdic'
diff --git a/Makefile.SH b/Makefile.SH
index a9ece4f..43701ad 100755
--- a/Makefile.SH
+++ b/Makefile.SH
@@ -234,10 +234,13 @@ dtrace_o=''
 minidtrace_o=''
 case "$usedtrace" in
 define|true)
-	dtrace_h='perldtrace.h' 
-	$dtrace -G -s perldtrace.d -o perldtrace.tmp >/dev/null 2>&1 \
-		&& rm -f perldtrace.tmp && dtrace_o='perldtrace$(OBJ_EXT)' \
-		&& minidtrace_o='miniperldtrace$(OBJ_EXT)'
+	dtrace_h='perldtrace.h'
+	case "$dtraceobject" in
+	define)
+		dtrace_o='perldtrace$(OBJ_EXT)'
+		minidtrace_o='miniperldtrace$(OBJ_EXT)'
+		;;
+	esac
 	;;
 esac
 
diff --git a/Porting/Glossary b/Porting/Glossary
index bc879f7..40e9b40 100644
--- a/Porting/Glossary
+++ b/Porting/Glossary
@@ -2850,6 +2850,9 @@ drand48_r_proto (d_drand48_r.U):
 dtrace (usedtrace.U):
 	This variable holds the location of the dtrace executable.
 
+dtraceobject (usedtrace.U):
+	Whether we need to build an object file with the dtrace tool.
+
 dynamic_ext (Extensions.U):
 	This variable holds a list of XS extension files we want to
 	link dynamically into the package.  It is used by Makefile.
-- 
2.1.4

[p5pRT]

From @tonycoz

On Tue Mar 01 19​:25​:06 2016, tonyc wrote​:

On Mon Dec 15 16​:05​:08 2014, Kris.Shannon wrote​:

The Ruby bug that I originally linked to ( https://bugs.ruby-
lang.org/issues/8606 ) ended up fixing this issue by going through all
the normal dtrace steps on a dummy .d​:

echo "provider conftest{ probe fire(); };" > conftest_provider.d
dtrace -h -o conftest_provider.h -s conftest_provider.d >/dev/null
2>/dev/null
cat >conftest.c <<_CONF
#include "conftest_provider.h"
int main(void){ CONFTEST_FIRE(); return 0; }
_CONF
$CC $CFLAGS -c -o conftest.o conftest.c
$DTRACE -G -s conftest_provider.d conftest.o

This will work to distinguish between the two different
implementations on both old and new illumos and on OSX.

Please try the attached.

Applied as c2538af with an expanded comment.

Thanks for reporting this and your help in tracking it down.

Tony

[p5pRT]

@tonycoz - Status changed from 'open' to 'pending release'

[p5pRT]

From @khwilliamson

Thank you for submitting this report. You have helped make Perl better.
 
With the release of Perl 5.24.0 on May 9, 2016, this and 149 other issues have been resolved.

Perl 5.24.0 may be downloaded via https://metacpan.org/release/RJBS/perl-5.24.0

[p5pRT]

@khwilliamson - Status changed from 'pending release' to 'resolved'