Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests fail with GCC 5.0 because Errno cannot obtain errno constants #14491

Closed
p5pRT opened this issue Feb 10, 2015 · 31 comments
Closed

Tests fail with GCC 5.0 because Errno cannot obtain errno constants #14491

p5pRT opened this issue Feb 10, 2015 · 31 comments
Labels

Comments

@p5pRT
Copy link
Collaborator

@p5pRT p5pRT commented Feb 10, 2015

Migrated from rt.perl.org#123784 (status was 'resolved')

Searchable as RT123784$

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 10, 2015

From @ppisar

Hello,

I don't know if this is the best news before releasing perl 5.20.2, but none
of the perl branches (including blead) does not pass tests when built
with GCC 5.0​:

t/run/switches ................................................. "EACCES" is not
exported by the Errno module
"EISDIR" is not exported by the Errno module
Can't continue after import errors at run/switches.t line 126.
# Looks like you planned 115 tests but ran 8.
FAILED--expected 115 tests, saw 8

etc.

This is caused by wrongly generated Errno.h which is missing all the constants
(e.g. Errno​::EISDIR)​:

Inline Patch
--- /usr/lib64/perl5/Errno.pm   2015-01-23 15:15:43.000000000 +0100
+++ ext/Errno/Errno.pm  2015-02-10 16:06:47.831000000 +0100
@@ -9,10 +9,10 @@
 use strict;
 
 "$Config{'archname'}-$Config{'osvers'}" eq
-"x86_64-linux-thread-multi-3.17.8-300.bz1178975.fc21.x86_64" or
-       die "Errno architecture (x86_64-linux-thread-multi-3.17.8-300.bz1178975.fc21.x86_64) does not match executable architecture ($Config{'archname'}-$Config{'osvers'})";
+"x86_64-linux-thread-multi-3.19.0-0.rc7.git2.1.fc22.x86_64" or
+       die "Errno architecture (x86_64-linux-thread-multi-3.19.0-0.rc7.git2.1.fc22.x86_64) does not match executable architecture ($Config{'archname'}-$Config{'osvers'})";
 
-our $VERSION = "1.20_03";
+our $VERSION = "1.22";
 $VERSION = eval $VERSION;
 our @ISA = 'Exporter';
 
@@ -20,140 +20,6 @@
 
 BEGIN {
     %err = (
-       EPERM => 1,
-       ENOENT => 2,
-       ESRCH => 3,
[...]

That's reportedly caused by a change in GCC 5.0's preprocesor as described in​:

<https://lists.fedoraproject.org/pipermail/devel/2015-February/207549.html>
<https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60723#c27>

The preprocessed output contains a new line with current working directory
directive which make troubles to the Errno_pm.PL parser.

Simple work-around is to call the preprocessor with -P option (inhibit
generation of linemarkers). Once I find a proper fix, I will post it here.

-- Petr

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 10, 2015

From @ppisar

On 2015-02-10, Petr Pisar <perlbug-followup@​perl.org> wrote​:

The preprocessed output contains a new line with current working
directory directive which make troubles to the Errno_pm.PL parser.

The format change is from​:

"ENOSTR" [[60]]

to

"ENOSTR" [[
# 14 "<stdin>" 3 4
  60
# 14 "<stdin>"
  ]]

in preprocessed errno.c in write_errno_pm().

Simple work-around is to call the preprocessor with -P option (inhibit
generation of linemarkers). Once I find a proper fix, I will post it
here.

As each line is broken into 5 lines and because the processed file is
not a C code I'm keen to append the -P option to the preprocesor command
in the write_errno_pm() function. The GCC's -P option is documented as​:

  This might be useful when running the preprocessor on something that
  is not C code, and will be sent to a program which might be confused
  by the linemarkers.

But I don't know whether the option is something universally supported
or recognized just by few compilers. In the latter case, the option
should be gaurded by $Config{gccversion} condition.

-- Petr

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 10, 2015

From @rurban

I perlbugged the fix to p5p, but I'm not sure if I'm still blocked, because I don't see my ticket.

basically gcc as cpp requires now the -P besides the -E

See attachment.

--
Reini Urban

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 10, 2015

From @rurban

0001-Fix-Errno.pm-generation-for-gcc-5.0.patch
From 1daec2374859ca47ac1d03e7cf187473390c55fa Mon Sep 17 00:00:00 2001
From: Reini Urban <rurban@cpanel.net>
Date: Tue, 10 Feb 2015 20:03:03 +0100
Subject: [PATCH 01/20] Fix Errno.pm generation for gcc-5.0

gcc-5.0 -E interleaves now line numbers with expended macros, so that
the generated errno.c will be preprocessed to
EBFONT => [[
           59
                ]]
which is hard to parse in in line-based reader.
So probe for cpp with "-E -P" first, which will skip producing the linenumbers.
---
 Configure | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git Configure Configure
index 034104f..244b063 100755
--- Configure
+++ Configure
@@ -4826,6 +4826,12 @@ fi
 
 if $ok; then
 	: nothing
+elif echo 'Maybe "'"$cc"' -E -P" will work...'; \
+	$cc -E <testcpp.c >testcpp.out 2>&1; \
+	$contains 'abc.*xyz' testcpp.out >/dev/null 2>&1 ; then
+	echo "Yup, it does."
+	x_cpp="$cc $cppflags -E -P"
+	x_minus='';
 elif echo 'Maybe "'"$cc"' -E" will work...'; \
 	$cc -E <testcpp.c >testcpp.out 2>&1; \
 	$contains 'abc.*xyz' testcpp.out >/dev/null 2>&1 ; then
-- 
2.1.4

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 10, 2015

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 11, 2015

From @tonycoz

On Tue Feb 10 12​:43​:33 2015, rurban wrote​:

I perlbugged the fix to p5p, but I'm not sure if I'm still blocked,
because I don't see my ticket.

basically gcc as cpp requires now the -P besides the -E

See attachment.

Unfortunately that patch is broken in at least three different ways​:

1) the echo says it's testing -E -P but then only tests -E

2) even if it's modified to use -P, it fails, since we're testing processing stdin

3) if we fix both of those, adding -P to cppstdin breaks makedepend, since it' uses the #line lines to generate the dependencies

The attached fixed all of those problems, but perhaps this belongs in a change to Errno_pm.PL

Tony

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 11, 2015

From @tonycoz

On Tue Feb 10 20​:32​:03 2015, tonyc wrote​:

The attached fixed all of those problems, but perhaps this belongs in
a change to Errno_pm.PL

Now with the attachment.

Tony

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 11, 2015

From @tonycoz

0001-Fix-Errno.pm-generation-for-gcc-5.0.patch
From 180f90af5a74169a7f016fe874f3c32d71eeaabf Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Wed, 11 Feb 2015 15:29:05 +1100
Subject: [PATCH] Fix Errno.pm generation for gcc-5.0

gcc-5.0 -E interleaves now line numbers with expended macros, so that
the generated errno.c will be preprocessed to

EBFONT => [[
           59
                ]]

which is hard to parse in in line-based reader.

So probe for cpp with "-E -ftrack-macro-expansion=0" first, which
will skip splitting the lines.

It's tempting to use -P instead, but that breaks makedepend.

Based on a patch by Reini Urban.
---
 Configure |   12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/Configure b/Configure
index 034104f..6d21e2b 100755
--- a/Configure
+++ b/Configure
@@ -4826,6 +4826,18 @@ fi
 
 if $ok; then
 	: nothing
+elif echo 'Maybe "'"$cc"' -E -ftrack-macro-expansion=0" will work...'; \
+	$cc -E -ftrack-macro-expansion=0 <testcpp.c >testcpp.out 2>&1; \
+	$contains 'abc.*xyz' testcpp.out >/dev/null 2>&1 ; then
+	echo "Yup, it does."
+	x_cpp="$cc $cppflags -E -ftrack-macro-expansion=0"
+	x_minus='';
+elif echo 'Maybe "'"$cc"' -E -ftrack-macro-expansion=0 -" will work...'; \
+	$cc -E -ftrack-macro-expansion=0 - <testcpp.c >testcpp.out 2>&1; \
+	$contains 'abc.*xyz' testcpp.out >/dev/null 2>&1 ; then
+	echo "Yup, it does."
+	x_cpp="$cc $cppflags -E -ftrack-macro-expansion=0"
+	x_minus='-';
 elif echo 'Maybe "'"$cc"' -E" will work...'; \
 	$cc -E <testcpp.c >testcpp.out 2>&1; \
 	$contains 'abc.*xyz' testcpp.out >/dev/null 2>&1 ; then
-- 
1.7.10.4

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 11, 2015

From @ppisar

On 2015-02-11, Tony Cook via RT <perlbug-followup@​perl.org> wrote​:

On Tue Feb 10 20​:32​:03 2015, tonyc wrote​:

The attached fixed all of those problems, but perhaps this belongs in
a change to Errno_pm.PL

I think Errno_pm.PL is better place. Otherwise the you change CPP flags
globaly and nobody can benefit from the new compiler feature.

Now with the attachment.

With the patch the lib/h2ph.t test fails for me​:

lib/h2ph ....................................................... # Failed test 5 - preamble compiles at ../lib/h2ph.t line 49
# got 'Bareword found where operator expected at _h2ph_pre.ph line 222, near "7fff"
# (Missing operator before fff?)
# Bareword found where operator expected at _h2ph_pre.ph line 228, near "7fffffff"
# (Missing operator before fffffff?)
# Bareword found where operator expected at _h2ph_pre.ph line 234, near "7fffffffffffffffL"
# (Missing operator before fffffffffffffffL?)
# Bareword found where operator expected at _h2ph_pre.ph line 240, near "7f"
# (Missing operator before f?)
# Bareword found where operator expected at _h2ph_pre.ph line 246, near "7fffffffffffffffL"
# (Missing operator before fffffffffffffffL?)
# Bareword found where operator expected at _h2ph_pre.ph line 250, near "7fffffffffffffffL"
# (Missing operator before fffffffffffffffL?)
# Bareword found where operator expected at _h2ph_pre.ph line 254, near "7fffffffffffffffL"
# (Missing operator before fffffffffffffffL?)
# Bareword found where operator expected at _h2ph_pre.ph line 258, near "7fffffffffffffffL"
# (Missing operator before fffffffffffffffL?)
# Bareword found where operator expected at _h2ph_pre.ph line 262, near "7fffffffffffffffL"
# (Missing operator before fffffffffffffffL?)
# Bareword found where operator expected at _h2ph_pre.ph line 266, near "7f"
# (Missing operator before f?)
# syntax error at _h2ph_pre.ph line 222, near "7fff "
# syntax error at _h2ph_pre.ph line 228, near "7fffffff "
# syntax error at _h2ph_pre.ph line 234, near "7fffffffffffffffL "
# syntax error at _h2ph_pre.ph line 240, near "7f "
# syntax error at _h2ph_pre.ph line 246, near "7fffffffffffffffL "
# syntax error at _h2ph_pre.ph line 250, near "7fffffffffffffffL "
# syntax error at _h2ph_pre.ph line 254, near "7fffffffffffffffL "
# syntax error at _h2ph_pre.ph line 258, near "7fffffffffffffffL "
# syntax error at _h2ph_pre.ph line 262, near "7fffffffffffffffL "
# syntax error at _h2ph_pre.ph line 266, near "7f "
# _h2ph_pre.ph has too many errors.
# '
# expected /(?^​:syntax OK$)/
# Failed test 6 - output free of warnings at ../lib/h2ph.t line 56
# got "Bareword found where operator expected at _h2ph_pre.ph line 222, near \"7fff\"\nCompilation failed in require at lib/h2ph.pht line 1.\nCompilation failed in require at -e line 1.\n"
# expected ""
FAILED at test 5

-- Petr

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 11, 2015

From @ppisar

On 2015-02-11, Petr Pisar <ppisar@​redhat.com> wrote​:

On 2015-02-11, Tony Cook via RT <perlbug-followup@​perl.org> wrote​:

On Tue Feb 10 20​:32​:03 2015, tonyc wrote​:

The attached fixed all of those problems, but perhaps this belongs in
a change to Errno_pm.PL

I think Errno_pm.PL is better place. Otherwise the you change CPP flags
globaly and nobody can benefit from the new compiler feature.

Now with the attachment.

With the patch the lib/h2ph.t test fails for me​:

lib/h2ph ....................................................... # Failed test 5 - preamble compiles at ../lib/h2ph.t line 49
# got 'Bareword found where operator expected at _h2ph_pre.ph line 222, near "7fff"
# (Missing operator before fff?)

This failure is probably not releated to the patch as it fails with GCC 5.0
even if I patched Errno_pm.PL instead.

-- Petr

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 11, 2015

From @ppisar

This is my patch patching Errno_pm.PL instead of the global configuration.
Please not that it does not address the h2ph failure.

-- Petr

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 11, 2015

From @ppisar

0001-Fix-Errno.pm-generation-for-gcc-5.0.patch
From 9e99628b4e1c05e04f01522854799dd53228ff40 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Petr=20P=C3=ADsa=C5=99?= <ppisar@redhat.com>
Date: Wed, 11 Feb 2015 15:46:37 +0100
Subject: [PATCH] Fix Errno.pm generation for gcc-5.0

gcc-5.0 -E interleaves now line numbers with expended macros, so that
the generated errno.c will be preprocessed to

EBFONT => [[
           59
                ]]

which is hard to parse in in line-based reader.

So use -P option with gcc >= 5.0. Global -P usage would break makedepend,
global -ftrack-macro-expansion=0 would break lib/h2ph.t.

RT#123784
---
 ext/Errno/Errno_pm.PL | 21 ++++++++++++++++-----
 1 file changed, 16 insertions(+), 5 deletions(-)

diff --git a/ext/Errno/Errno_pm.PL b/ext/Errno/Errno_pm.PL
index 3dadfce..92c8666 100644
--- a/ext/Errno/Errno_pm.PL
+++ b/ext/Errno/Errno_pm.PL
@@ -215,20 +215,31 @@ sub write_errno_pm {
     {	# BeOS (support now removed) did not enter this block
     # invoke CPP and read the output
 
+	my $inhibit_linemarkers = '';
+	if ($Config{gccversion} =~ /\A(5)\./ and $1 >= 5) {
+	    # GCC 5.0 interleaves expanded macros with line numbers breaking
+	    # each line into multiple lines. RT#123784
+	    $inhibit_linemarkers = ' -P';
+	}
+
 	if ($^O eq 'VMS') {
-	    my $cpp = "$Config{cppstdin} $Config{cppflags} $Config{cppminus}";
+	    my $cpp = "$Config{cppstdin} $Config{cppflags}" .
+		$inhibit_linemarkers . " $Config{cppminus}";
 	    $cpp =~ s/sys\$input//i;
 	    open(CPPO,"$cpp  errno.c |") or
 		die "Cannot exec $Config{cppstdin}";
 	} elsif ($IsMSWin32 || $^O eq 'NetWare') {
-	    open(CPPO,"$Config{cpprun} $Config{cppflags} errno.c |") or
-		die "Cannot run '$Config{cpprun} $Config{cppflags} errno.c'";
+	    my $cpp = "$Config{cpprun} $Config{cppflags}" .
+		$inhibit_linemarkers;
+	    open(CPPO,"$cpp errno.c |") or
+		die "Cannot run '$cpp errno.c'";
 	} elsif ($IsSymbian) {
-            my $cpp = "gcc -E -I$ENV{SDK}\\epoc32\\include\\libc -";
+            my $cpp = "gcc -E -I$ENV{SDK}\\epoc32\\include\\libc" .
+		$inhibit_linemarkers ." -";
 	    open(CPPO,"$cpp < errno.c |")
 		or die "Cannot exec $cpp";
         } else {
-	    my $cpp = default_cpp();
+	    my $cpp = default_cpp() . $inhibit_linemarkers;
 	    open(CPPO,"$cpp < errno.c |")
 		or die "Cannot exec $cpp";
 	}
-- 
2.3.0

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 11, 2015

From @ppisar

On 2015-02-11, Petr Pisar <ppisar@​redhat.com> wrote​:

On 2015-02-11, Petr Pisar <ppisar@​redhat.com> wrote​:

lib/h2ph ....................................................... # Failed test 5 - preamble compiles at ../lib/h2ph.t line 49
# got 'Bareword found where operator expected at _h2ph_pre.ph line 222, near "7fff"
# (Missing operator before fff?)

This failure is probably not releated to the patch as it fails with GCC 5.0
even if I patched Errno_pm.PL instead.

The t/_h2ph_pre.ph file generated by the ../utils/h2ps differs, for
example, in this​:

-unless (defined &__INT16_MAX__) { sub __INT16_MAX__() { 32767 } }
+unless (defined &__INT16_MAX__) { sub __INT16_MAX__() { &0x7fff } }

All the test failures are about lines with a hexadecimal number prefixed
by an ampersand.

This could be maybe caused my glibc-headers-2.20.90 which was also built
by GCC 5.0.

-- Petr

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 11, 2015

From @ppisar

On 2015-02-11, Petr Pisar <ppisar@​redhat.com> wrote​:

On 2015-02-11, Petr Pisar <ppisar@​redhat.com> wrote​:

On 2015-02-11, Petr Pisar <ppisar@​redhat.com> wrote​:

lib/h2ph ....................................................... # Failed test 5 - preamble compiles at ../lib/h2ph.t line 49
# got 'Bareword found where operator expected at _h2ph_pre.ph line 222, near "7fff"
# (Missing operator before fff?)

This failure is probably not releated to the patch as it fails with GCC 5.0
even if I patched Errno_pm.PL instead.

The t/_h2ph_pre.ph file generated by the ../utils/h2ps differs, for
example, in this​:

-unless (defined &__INT16_MAX__) { sub __INT16_MAX__() { 32767 } }
+unless (defined &__INT16_MAX__) { sub __INT16_MAX__() { &0x7fff } }

All the test failures are about lines with a hexadecimal number prefixed
by an ampersand.

After I implement hexadecimal integers in the h2ph, I got this failure​:

$ ./perl -MTestInit lib/h2ph.t
1..6
ok 1 - output is free of warnings
ok 2 - ../utils/h2ph runs successfully
ok 3 - generated file has expected contents
ok 4 - output compiles
ok 5 - preamble compiles
not ok 6 - output free of warnings
# Failed test 6 - output free of warnings at lib/h2ph.t line 56
# got "Hexadecimal number > 0xffffffff non-portable at _h2ph_pre.ph line 234.\nCompilation failed in require at lib/h2ph.pht line 1.\nCompilation failed in require at -e line 1.\n"
# expected ""

My understanding is this is rather advice for Perl users not to use such
big numbers. So I think this warning could be ingnored by the "output
free of warnings" test. Am I right?

What surprises me is that the failing number is 0x7fffffffffffffff which
is exactly 9223372036854775807 and perl warns only on the hexadecimal
form. I would expect warning in both cases​:

$ ./perl -Ilib -w -e '$a = 9223372036854775807';
$ ./perl -Ilib -w -e '$a = 0x7fffffffffffffff';
Hexadecimal number > 0xffffffff non-portable at -e line 1.

-- Petr

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 12, 2015

From @tonycoz

On Wed Feb 11 08​:09​:15 2015, ppisar wrote​:

On 2015-02-11, Petr Pisar <ppisar@​redhat.com> wrote​:

On 2015-02-11, Petr Pisar <ppisar@​redhat.com> wrote​:

lib/h2ph ....................................................... #
Failed test 5 - preamble compiles at ../lib/h2ph.t line 49
# got 'Bareword found where operator expected at _h2ph_pre.ph
line 222, near "7fff"
# (Missing operator before fff?)

This failure is probably not releated to the patch as it fails with
GCC 5.0
even if I patched Errno_pm.PL instead.

The t/_h2ph_pre.ph file generated by the ../utils/h2ps differs, for
example, in this​:

-unless (defined &__INT16_MAX__) { sub __INT16_MAX__() { 32767 } }
+unless (defined &__INT16_MAX__) { sub __INT16_MAX__() { &0x7fff } }

All the test failures are about lines with a hexadecimal number
prefixed
by an ampersand.

This could be maybe caused my glibc-headers-2.20.90 which was also
built
by GCC 5.0.

The preamble file generation doesn't handle hex constants.

If you look at cppsymbols in config.sh, you'll see​:

  __INT32_MAX__=0x7fffffff

Attached a patch to fix that.

As to the warning, I suspect hex constants warn because people expect them to be treated as integers, while a decimal constant could be treated either way.

Tony

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 12, 2015

From @tonycoz

0001-h2ph-correct-handling-of-hex-constants-for-the-pream.patch
From 9bab817af6c7d7c028d2c790d2ae4eb319e1077f Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Thu, 12 Feb 2015 14:10:36 +1100
Subject: [PATCH] h2ph: correct handling of hex constants for the preamble

---
 utils/h2ph.PL |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/utils/h2ph.PL b/utils/h2ph.PL
index 9a8b14d..c46d423 100644
--- a/utils/h2ph.PL
+++ b/utils/h2ph.PL
@@ -769,7 +769,7 @@ sub inc_dirs
 sub build_preamble_if_necessary
 {
     # Increment $VERSION every time this function is modified:
-    my $VERSION     = 3;
+    my $VERSION     = 4;
     my $preamble    = "$Dest_dir/_h2ph_pre.ph";
 
     # Can we skip building the preamble file?
@@ -788,6 +788,8 @@ sub build_preamble_if_necessary
 
     open  PREAMBLE, ">$preamble" or die "Cannot open $preamble:  $!";
 	print PREAMBLE "# This file was created by h2ph version $VERSION\n";
+        # prevent large hex constants from warning
+        print PREAMBLE "no warnings qw(portable);\n";
 
 	foreach (sort keys %define) {
 	    if ($opt_D) {
@@ -810,7 +812,7 @@ DEFINE
 		# float:
 		print PREAMBLE
 		    "unless (defined &$_) { sub $_() { $1 } }\n\n";
-	    } elsif ($define{$_} =~ /^([+-]?\d+)U?L{0,2}$/i) {
+	    } elsif ($define{$_} =~ /^([+-]?\d+|0x[\da-f]+)U?L{0,2}$/i) {
 		# integer:
 		print PREAMBLE
 		    "unless (defined &$_) { sub $_() { $1 } }\n\n";
-- 
1.7.10.4

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 12, 2015

From @ppisar

On Wed, Feb 11, 2015 at 07​:21​:19PM -0800, Tony Cook via RT wrote​:

On Wed Feb 11 08​:09​:15 2015, ppisar wrote​:

On 2015-02-11, Petr Pisar <ppisar@​redhat.com> wrote​:

On 2015-02-11, Petr Pisar <ppisar@​redhat.com> wrote​:

lib/h2ph ....................................................... #
Failed test 5 - preamble compiles at ../lib/h2ph.t line 49
# got 'Bareword found where operator expected at _h2ph_pre.ph
line 222, near "7fff"
# (Missing operator before fff?)

This failure is probably not releated to the patch as it fails with
GCC 5.0
even if I patched Errno_pm.PL instead.

The t/_h2ph_pre.ph file generated by the ../utils/h2ps differs, for
example, in this​:

-unless (defined &__INT16_MAX__) { sub __INT16_MAX__() { 32767 } }
+unless (defined &__INT16_MAX__) { sub __INT16_MAX__() { &0x7fff } }

All the test failures are about lines with a hexadecimal number
prefixed
by an ampersand.

This could be maybe caused my glibc-headers-2.20.90 which was also
built
by GCC 5.0.

The preamble file generation doesn't handle hex constants.

If you look at cppsymbols in config.sh, you'll see​:

__INT32_MAX__=0x7fffffff

Attached a patch to fix that.

Yes. It works for me. Thanks.

Just a nit-pick. The regular expression​:

  /^([+-]?\d+|0x[\da-f]+)U?L{0,2}$/i

does not cover negative hexadecimal integers. If I understand the ISO-C11
grammar correctly, one can have -0xf or +0xf literals.

As to the warning, I suspect hex constants warn because people expect them
to be treated as integers, while a decimal constant could be treated either
way.

I don't understand. Hexadecimal constants are not integers?

-- Petr

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 12, 2015

From @ppisar

On Wed, Feb 11, 2015 at 04​:48​:24PM +0100, Petr Pisar wrote​:

This is my patch patching Errno_pm.PL instead of the global configuration.
Please not that it does not address the h2ph failure.

The patch did not expected GCC with major version number greater than 5 and it
failed porting/cmp_version.t because it did not increase Errno's version
value. Attached patch replaces the previous one.

Together with Tony's h2ph​: correct handling of hex constants for the preamble
patch, all tests pass for me now.

-- Petr

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 12, 2015

From @ppisar

0001-Fix-Errno.pm-generation-for-gcc-5.0.patch
From 96bcd6ed97ff05f5b421005f23973279dbfcafbf Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Petr=20P=C3=ADsa=C5=99?= <ppisar@redhat.com>
Date: Wed, 11 Feb 2015 15:46:37 +0100
Subject: [PATCH 1/2] Fix Errno.pm generation for gcc-5.0

gcc-5.0 -E interleaves now line numbers with expended macros, so that
the generated errno.c will be preprocessed to

EBFONT => [[
           59
                ]]

which is hard to parse in in line-based reader.

So use -P option with gcc >= 5.0. Global -P usage would break makedepend,
global -ftrack-macro-expansion=0 would break lib/h2ph.t.

RT#123784
---
 ext/Errno/Errno_pm.PL | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)

diff --git a/ext/Errno/Errno_pm.PL b/ext/Errno/Errno_pm.PL
index 3dadfce..c6bfa06 100644
--- a/ext/Errno/Errno_pm.PL
+++ b/ext/Errno/Errno_pm.PL
@@ -2,7 +2,7 @@ use ExtUtils::MakeMaker;
 use Config;
 use strict;
 
-our $VERSION = "1.22";
+our $VERSION = "1.23";
 
 my %err = ();
 
@@ -215,20 +215,31 @@ sub write_errno_pm {
     {	# BeOS (support now removed) did not enter this block
     # invoke CPP and read the output
 
+	my $inhibit_linemarkers = '';
+	if ($Config{gccversion} =~ /\A(\d+)\./ and $1 >= 5) {
+	    # GCC 5.0 interleaves expanded macros with line numbers breaking
+	    # each line into multiple lines. RT#123784
+	    $inhibit_linemarkers = ' -P';
+	}
+
 	if ($^O eq 'VMS') {
-	    my $cpp = "$Config{cppstdin} $Config{cppflags} $Config{cppminus}";
+	    my $cpp = "$Config{cppstdin} $Config{cppflags}" .
+		$inhibit_linemarkers . " $Config{cppminus}";
 	    $cpp =~ s/sys\$input//i;
 	    open(CPPO,"$cpp  errno.c |") or
 		die "Cannot exec $Config{cppstdin}";
 	} elsif ($IsMSWin32 || $^O eq 'NetWare') {
-	    open(CPPO,"$Config{cpprun} $Config{cppflags} errno.c |") or
-		die "Cannot run '$Config{cpprun} $Config{cppflags} errno.c'";
+	    my $cpp = "$Config{cpprun} $Config{cppflags}" .
+		$inhibit_linemarkers;
+	    open(CPPO,"$cpp errno.c |") or
+		die "Cannot run '$cpp errno.c'";
 	} elsif ($IsSymbian) {
-            my $cpp = "gcc -E -I$ENV{SDK}\\epoc32\\include\\libc -";
+            my $cpp = "gcc -E -I$ENV{SDK}\\epoc32\\include\\libc" .
+		$inhibit_linemarkers ." -";
 	    open(CPPO,"$cpp < errno.c |")
 		or die "Cannot exec $cpp";
         } else {
-	    my $cpp = default_cpp();
+	    my $cpp = default_cpp() . $inhibit_linemarkers;
 	    open(CPPO,"$cpp < errno.c |")
 		or die "Cannot exec $cpp";
 	}
-- 
2.3.0

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 12, 2015

From @tonycoz

On Thu, Feb 12, 2015 at 08​:51​:10AM +0100, Petr Pisar wrote​:

Yes. It works for me. Thanks.

Just a nit-pick. The regular expression​:

/^\(\[\+\-\]?\\d\+|0x\[\\da\-f\]\+\)U?L\{0\,2\}$/i

does not cover negative hexadecimal integers. If I understand the ISO-C11
grammar correctly, one can have -0xf or +0xf literals.

I'll fix that.

As to the warning, I suspect hex constants warn because people expect them
to be treated as integers, while a decimal constant could be treated either
way.

I don't understand. Hexadecimal constants are not integers?

On a 32-bit platform (without -Duse64bitint), a decimal integer over
2**32-1 will be treated as a floating point number, but a hexadecimal
integer over 2**32-1 will be treated as a compile-time error.

This makes me wonder if we have another issue - will _h2ph_pre.ph
compile successfully under the following conditions​:

- 32-bit build of perl
- without -Duse64bitint
- the platform has 64-bit long long,
- and the related constants are defined by the compiler.

Tony

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 13, 2015

From @ppisar

On Thu, Feb 12, 2015 at 10​:10​:39PM +1100, Tony Cook wrote​:

On a 32-bit platform (without -Duse64bitint), a decimal integer over
2**32-1 will be treated as a floating point number, but a hexadecimal
integer over 2**32-1 will be treated as a compile-time error.

This makes me wonder if we have another issue - will _h2ph_pre.ph
compile successfully under the following conditions​:

- 32-bit build of perl
- without -Duse64bitint
- the platform has 64-bit long long,
- and the related constants are defined by the compiler.

You were right. This does not work. I thought it worked, but after installing
patched perl into such system, next rebuild (for not clear reason, the build
used _h2pg_pre.ph from system, so the issue got visible on second build)
revealed​:

t ok 6 - output free of warnings
# Failed test 6 - output free of warnings at lib/h2ph.t line 56
# got "Integer overflow in hexadecimal number at _h2ph_pre.ph line 231.\nCompilation failed in require at lib/h2ph.pht line 1.\nCompilation failed in require at -e line 1.\n"
# expected ""

Supressing "overflow" warnings category makes me nervous.

-- Petr

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 16, 2015

From @tonycoz

On Fri Feb 13 06​:55​:57 2015, ppisar wrote​:

On Thu, Feb 12, 2015 at 10​:10​:39PM +1100, Tony Cook wrote​:

On a 32-bit platform (without -Duse64bitint), a decimal integer over
2**32-1 will be treated as a floating point number, but a hexadecimal
integer over 2**32-1 will be treated as a compile-time error.

This makes me wonder if we have another issue - will _h2ph_pre.ph
compile successfully under the following conditions​:

- 32-bit build of perl
- without -Duse64bitint
- the platform has 64-bit long long,
- and the related constants are defined by the compiler.

You were right. This does not work. I thought it worked, but after
installing
patched perl into such system, next rebuild (for not clear reason, the
build
used _h2pg_pre.ph from system, so the issue got visible on second
build)
revealed​:

t ok 6 - output free of warnings
# Failed test 6 - output free of warnings at lib/h2ph.t line 56
# got "Integer overflow in hexadecimal number at _h2ph_pre.ph
line 231.\nCompilation failed in require at lib/h2ph.pht line
1.\nCompilation failed in require at -e line 1.\n"
# expected ""

Supressing "overflow" warnings category makes me nervous.

Does the attached help? I don't have a 32-bit system with gcc-5 installed.

Replaces my original patch.

Tony

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 16, 2015

From @tonycoz

0001-h2ph-correct-handling-of-hex-constants-for-the-pream.patch
From df7e6433e66989a4cd0d1c5b914dedb065220d30 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Mon, 16 Feb 2015 15:57:00 +1100
Subject: [PATCH] h2ph: correct handling of hex constants for the preamble

Previously they were treated as identifiers resulting in code
generated like C< &0xFFF >.

We also try to prevent compile-time warnings from large hex integers,
the user isn't responsible for the generated code, so we delay those
warnings to run-time.
---
 utils/h2ph.PL |   19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/utils/h2ph.PL b/utils/h2ph.PL
index 9a8b14d..d082f22 100644
--- a/utils/h2ph.PL
+++ b/utils/h2ph.PL
@@ -769,7 +769,7 @@ sub inc_dirs
 sub build_preamble_if_necessary
 {
     # Increment $VERSION every time this function is modified:
-    my $VERSION     = 3;
+    my $VERSION     = 4;
     my $preamble    = "$Dest_dir/_h2ph_pre.ph";
 
     # Can we skip building the preamble file?
@@ -788,6 +788,11 @@ sub build_preamble_if_necessary
 
     open  PREAMBLE, ">$preamble" or die "Cannot open $preamble:  $!";
 	print PREAMBLE "# This file was created by h2ph version $VERSION\n";
+        # Prevent non-portable hex constants from warning.
+        #
+        # We still produce an overflow warning if we can't represent
+        # a hex constant as an integer.
+        print PREAMBLE "no warnings qw(portable);\n";
 
 	foreach (sort keys %define) {
 	    if ($opt_D) {
@@ -814,6 +819,18 @@ DEFINE
 		# integer:
 		print PREAMBLE
 		    "unless (defined &$_) { sub $_() { $1 } }\n\n";
+            } elsif ($define{$_} =~ /^([+-]?0x[\da-f]+)U?L{0,2}$/i) {
+                # hex integer
+                # Special cased, since perl warns on hex integers
+                # that can't be represented in a UV.
+                #
+                # This way we get the warning at time of use, so the user
+                # only gets the warning if they happen to use this
+                # platform-specific definition.
+                my $code = $1;
+                $code = "hex('$code')" if length $code > 10;
+                print PREAMBLE
+                    "unless (defined &$_) { sub $_() { $code } }\n\n";
 	    } elsif ($define{$_} =~ /^\w+$/) {
 		my $def = $define{$_};
 		if ($isatype{$def}) {
-- 
1.7.10.4

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 16, 2015

From @ppisar

On Sun, Feb 15, 2015 at 09​:11​:25PM -0800, Tony Cook via RT wrote​:

On Fri Feb 13 06​:55​:57 2015, ppisar wrote​:

On Thu, Feb 12, 2015 at 10​:10​:39PM +1100, Tony Cook wrote​:

On a 32-bit platform (without -Duse64bitint), a decimal integer over
2**32-1 will be treated as a floating point number, but a hexadecimal
integer over 2**32-1 will be treated as a compile-time error.

This makes me wonder if we have another issue - will _h2ph_pre.ph
compile successfully under the following conditions​:

- 32-bit build of perl
- without -Duse64bitint
- the platform has 64-bit long long,
- and the related constants are defined by the compiler.

You were right. This does not work. I thought it worked, but after
installing
patched perl into such system, next rebuild (for not clear reason, the
build
used _h2pg_pre.ph from system, so the issue got visible on second
build)
revealed​:

t ok 6 - output free of warnings
# Failed test 6 - output free of warnings at lib/h2ph.t line 56
# got "Integer overflow in hexadecimal number at _h2ph_pre.ph
line 231.\nCompilation failed in require at lib/h2ph.pht line
1.\nCompilation failed in require at -e line 1.\n"
# expected ""

Supressing "overflow" warnings category makes me nervous.

Does the attached help? I don't have a 32-bit system with gcc-5 installed.

Yes. It works both on amd64 and on x86 with native integers.

-- Petr

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 17, 2015

From @tonycoz

On Mon Feb 16 04​:57​:45 2015, ppisar wrote​:

On Sun, Feb 15, 2015 at 09​:11​:25PM -0800, Tony Cook via RT wrote​:

Does the attached help? I don't have a 32-bit system with gcc-5
installed.

Yes. It works both on amd64 and on x86 with native integers.

Your patch thanks, applied as 816b056 and mine as 3bea78d.

Tony

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 17, 2015

@tonycoz - Status changed from 'open' to 'pending release'

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 17, 2015

From @ppisar

On Fri, Feb 13, 2015 at 03​:55​:12PM +0100, Petr Pisar wrote​:

(for not clear reason, the build used _h2pg_pre.ph from system, so the issue
got visible on second build)

I tracked the cause why it failed after second rebuild and not on the first
one​:

$ LD_PRELOAD=$PWD/libperl.so strace -fq -s 1024 -e execve,chdir,open -y -v ./perl -MTestInit lib/h2ph.t
[...]
chdir("t") = 0
[...]
[pid 2341] execve("/home/test/perl-git/perl", ["/home/test/perl-git/perl", "-I../lib", "-w", "-e", "$SIG{__WARN__} = sub { die $_[0] }; require q(lib/h2ph.pht);", "-e", ""], ["XDG_SESSION_ID=1", "HOSTNAME=fedora-23", "TERM=screen", "SHELL=/bin/bash", "HISTSIZE=1000", "PERL5LIB=../lib", "LD_PRELOAD=/home/test/perl-git/libperl.so", "USER=test", "LS_COLORS=rs=0​:di=01;34​:ln=01;36​:mh=00​:pi=40;33​:so=01;35​:do=01;35​:bd=40;33;01​:cd=40;33;01​:or=40;31;01​:mi=01;05;37;41​:su=37;41​:sg=30;43​:ca=30;41​:tw=30;42​:ow=34;42​:st=37;44​:ex=01;32​:*.tar=01;31​:*.tgz=01;31​:*.arc=01;31​:*.arj=01;31​:*.taz=01;31​:*.lha=01;31​:*.lz4=01;31​:*.lzh=01;31​:*.lzma=01;31​:*.tlz=01;31​:*.txz=01;31​:*.tzo=01;31​:*.t7z=01;31​:*.zip=01;31​:*.z=01;31​:*.Z=01;31​:*.dz=01;31​:*.gz=01;31​:*.lrz=01;31​:*.lz=01;31​:*.lzo=01;31​:*.xz=01;31​:*.bz2=01;31​:*.bz=01;31​:*.tbz=01;31​:*.tbz2=01;31​:*.tz=01;31​:*.deb=01;31​:*.rpm=01;31​:*.jar=01;31​:*.war=01;31​:*.ear=01;31​:*.sar=01;31​:*.rar=01;31​:*.alz=01;31​:*.ace=01;31​:*.zoo=01;31​:*.cpio=01;31​:*.7z=01;31​:*.rz=01;31​:*.cab=01;31​:*.jpg=01;35​:*.jpeg=01;35​:*.gif=01;35​:*.bmp=01;35​:*.pbm=01;35​:*.pgm=01;35​:*.ppm=01;35​:*.tga=01;35​:*.xbm=01;35​:*.xpm=01;35​:*.tif=01;35​:*.tiff=01;35​:*.png=01;35​:*.svg=01;35​:*.svgz=01;35​:*.mng=01;35​:*.pcx=01;35​:*.mov=01;35​:*.mpg=01;35​:*.mpeg=01;35​:*.m2v=01;35​:*.mkv=01;35​:*.webm=01;35​:*.ogm=01;35​:*.mp4=01;35​:*.m4v=01;35​:*.mp4v=01;35​:*.vob=01;35​:*.qt=01;35​:*.nuv=01"..., "COBBLER_SERVER=lab.rhts.englab.brq.redhat.com", "PATH=/usr/local/bin​:/bin​:/usr/bin​:/usr/local/sbin​:/usr/sbin​:/home/test/bin", "MAIL=/var/spool/mail/test", "PERL_CORE=/home/test/perl-git/perl", "_=/home/test/perl-git/perl", "PWD=/home/test/perl-git/t", "LANG=en_US.UTF-8", "PERLEXE=/home/test/perl-git/perl", "HISTCONTROL=ignoredups", "HOME=/home/test", "SHLVL=2", "LOGNAME=test", "LESSOPEN=||/usr/bin/lesspipe.sh %s"]) = 0
[...]
[pid 2341] open("./lib/h2ph.pht", O_RDONLY) = 3</home/test/perl-git/t/lib/h2ph.pht>
[pid 2341] open("../lib/warnings.pm", O_RDONLY) = 4</home/test/perl-git/lib/warnings.pm>
[pid 2341] open("/usr/lib64/perl5/_h2ph_pre.ph", O_RDONLY) = 3</usr/lib64/perl5/_h2ph_pre.ph>
[...]

It's becuse lib/h2ph.t executes generated code without passing proper -I
argument to point to the place where the generated code resides. Then the
interpreter loads _h2ph_pre.ph from system instead from the build directory.

Attached patch fixes it.

-- Petr

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 17, 2015

From @ppisar

0001-lib-h2ph.t-to-test-generated-t-_h2ph_pre.ph-instead-.patch
From ae54661bfad51c56e0d5c01bace60d44513a77e2 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Petr=20P=C3=ADsa=C5=99?= <ppisar@redhat.com>
Date: Tue, 17 Feb 2015 13:11:00 +0100
Subject: [PATCH] lib/h2ph.t to test generated t/_h2ph_pre.ph instead of the
 system one
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The lib/h2ph.t test executes a t/lib/h2ph.pht which requires
'_h2ph_pre.ph'.  This should find and exercise generated t/_h2ph_pre.ph
file. However, it found a loaded _h2ph_pre.ph from system because the
interpreter has the './' directory after the system paths in the @INC by
default.

This patch adds '-I./' to the runperl() invocation to prefer the
_h2ph_pre.ph generated at build time.

Signed-off-by: Petr Písař <ppisar@redhat.com>
---
 lib/h2ph.t | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/h2ph.t b/lib/h2ph.t
index 2b58f6a..64d9dc0 100644
--- a/lib/h2ph.t
+++ b/lib/h2ph.t
@@ -48,7 +48,7 @@ $result = runperl( progfile => '_h2ph_pre.ph',
                    stderr => 1 );
 like( $result, qr/syntax OK$/, "preamble compiles");
 
-$result = runperl( switches => ["-w"],
+$result = runperl( switches => ['-I.', "-w"],
                    stderr => 1,
                    prog => <<'PROG' );
 $SIG{__WARN__} = sub { die $_[0] }; require q(lib/h2ph.pht);
-- 
2.1.0

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 24, 2015

From @tonycoz

On Tue Feb 17 04​:57​:50 2015, ppisar wrote​:

It's becuse lib/h2ph.t executes generated code without passing proper
-I
argument to point to the place where the generated code resides. Then
the
interpreter loads _h2ph_pre.ph from system instead from the build
directory.

Attached patch fixes it.

Thanks, applied as 3359391.

Tony

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 2, 2015

From @khwilliamson

Thank you for submitting this ticket.

The issue should now be resolved with the release today of Perl v5.22, which is available at http​://www.perl.org/get.html
--
Karl Williamson for the Perl 5 team

@p5pRT p5pRT closed this Jun 2, 2015
@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 2, 2015

@khwilliamson - Status changed from 'pending release' to 'resolved'

@p5pRT p5pRT added the Severity Low label Oct 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant
You can’t perform that action at this time.