Can't build perl with gcc4.9 due to UB in sv.c #13690
Comments
From polacek@redhat.comCreated by polacek@redhat.comPerl Info |
From @tonycozOn Tue, Apr 01, 2014 at 07:24:37AM +0100, Zefram wrote:
Unfortunately there's a lot of code re-ordering differences between Interestingly enough the code that the patch updates has no changes Adding the patch produces a significant change in the code - closer to Without the patch: ; the .loc lines are line number information With the patch: .loc 1 2053 0 .L4724:
I don't think that's practical.
At least back to 4.1. Tony |
|
The RT System itself - Status changed from 'new' to 'open' |
From @tonycozSubject altered so it shows in the ticket too. On Mon, Apr 07, 2014 at 11:25:14AM -0400, Ricardo Signes wrote:
GCC 4.9 hasn't been released yet, but will likely be released by the So I think it's a 5.20 issue. The behaviour we're depending on technically violates the C standard In the short term for 5.20.0 I think we (and I'll make a start on it) In the longer term I think we need to deal with the problem. In some cases, like the handling of RExC_naughty in regcomp.c, I think Other cases, like the overflow at line 4661 in regcomp.c: data->last_start_max += is_inf ? SSize_t_MAX could be real problems leading to bugs down the road. (the above 2 of many detected with -fsantize=undefined) Tony [1] 4.7.0 and 4.8.0 were released in March 2012 and 2013 respectively [2] I don't know if that matters much, I only see one test for |
From @doughera88On Tue, Apr 08, 2014 at 05:00:13PM +1000, Tony Cook wrote:
I think it would have to be Configure. cflags.SH only affects the (I'm afraid I haven't looked at the underlying issue at all, since it -- |
From zefram@fysh.orgAndy Dougherty wrote:
Yes. -zefram |
From @tonycozOn Tue Apr 08 04:46:13 2014, zefram@fysh.org wrote:
Patch to Configure attached for review. Tony |
From @tonycoz0001-perl-121505-add-fwrapv-to-ccflags-for-gcc-4.9-and-la.patchFrom 1d48451145d8351349ecdfa6a13ea422a4480cd3 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Wed, 9 Apr 2014 11:58:33 +1000
Subject: [PATCH] [perl #121505] add -fwrapv to ccflags for gcc 4.9 and later
But skip adding it if -fwrapv is already provided, or if -fno-wrapv or
-fsanitize=undefined is supplied in ccflags.
---
Configure | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/Configure b/Configure
index 15b3da1..56ce12d 100755
--- a/Configure
+++ b/Configure
@@ -4643,6 +4643,22 @@ case "$gccversion" in
$rm -f try try.*
esac
+# gcc 4.9 by default does some optimizations that break perl.
+# see ticket 121505.
+#
+# The -fwrapv disables those optimizations (and probably others,) so
+# for gcc 4.9 (and later, since the optimizations probably won't go
+# away), add -fwrapv unless the user requests -fno-wrapv, which
+# disables -fwrapv, or if the user requests -fsanitize=undefined,
+# which turns the overflows -fwrapv ignores into runtime errors.
+case "$gccversion" in
+4.9.*|4.1[0-9].*|[5-9].*)
+ case "$ccflags" in
+ *-fno-wrapv*|*-fsanitize=undefined*|*-fwrapv*) ;;
+ *) ccflags="$ccflags -fwrapv" ;;
+ esac
+esac
+
: What should the include directory be ?
: Use sysroot if set, so findhdr looks in the right place.
echo " "
--
1.7.10.4
|
From @TuxOn Tue, 8 Apr 2014 18:59:35 -0700, "Tony Cook via RT"
Why make it a new case, and not integrate into the switch just above? And are there any plans at all to "fix" the cause for this patch in -- |
From @tonycozOn Wed, Apr 09, 2014 at 08:09:33AM +0200, H.Merijn Brand wrote:
To make it easier to remove later (or at least make it easier to to
I think it's too much work to fix them all and be sure the fixes I think they should be fixed for 5.22, and then the Configure change Tony |
From zefram@fysh.orgTony Cook via RT wrote:
Should also add it for older gccs, as far back as the flag is supported
Misses a lot of possible version numbers. For 4.9+, you want 4.9.*|4.[1-9][0-9]*|[5-9].*|[1-9][0-9]*) A bit of googling says -fwrapv is broken prior to 4.3, so that would be 4.[3-9].*|4.[1-9][0-9]*|[5-9].*|[1-9][0-9]*) Some explanation: http://www.airs.com/blog/archives/120 That page also offers the options -fno-strict-overflow and -zefram |
From @TuxOn Wed, 9 Apr 2014 10:15:53 +0100, Zefram <zefram@fysh.org> wrote:
I already pushed Tony's patch. Shall I amend it?
-- |
From zefram@fysh.orgH.Merijn Brand wrote:
+1 to amend. -zefram |
From @TuxOn Wed, 9 Apr 2014 10:33:23 +0100, Zefram <zefram@fysh.org> wrote:
For those not following git, both the original patch and the amendment commit 00051dd -fwrapv is broken prior to gcc-4.3 (googled and patched by Zefram) commit 8697475 gcc 4.9 by default does some optimizations that break perl (#121505) Patch by Tony Cook -- |
From @tonycozOn Wed Apr 09 02:16:26 2014, zefram@fysh.org wrote:
The problem with including 4.3 - 4.8 is -fwrapv disables optimizations, and I still need to produce an update for win32/makefile.mk. Tony |
From @tonycozOn Wed Apr 09 14:55:54 2014, tonyc wrote:
Though op/numconvert.t doesn't fail, nor does any other relevant test, with 4.9.0 20140218. I don't really see a way to pick out a version in makefile.mk either, the only real solution I see is to always add it. Tony |
From @steve-m-hayOn 10 April 2014 00:57, Tony Cook via RT <perlbug-followup@perl.org> wrote:
Add a new CCTYPE, as is done for different MSVC++ versions? GCC49 or |
From zefram@fysh.orgTony Cook via RT wrote:
It disables specifically the optimisations that break our code.
The problems wouldn't necessarily have been noticed. -zefram |
From @tonycozOn Thu Apr 10 00:03:05 2014, shay wrote:
Adding extra CCTYPE values means touching every place that looks at CCTYPE. I tried to add GCCVERSION and then parse that using findstr, but it was unwieldly[1] and didn't work, since dmake treats the non-zero exit from findstr when it doesn't find the string as a build-time error. I ended up adding a GCCWRAPV macro that can be overridden from the dmake command-line which enables -fwrapv by default. This builds and tests ok with the gcc 4.7.3 included with the strawberry perl I have installed. It builds perl successfully with a mingw gcc-4.9 snapshot, but I suspect testing would fail due to some strangeness in the built Errno.pm (the errno codes have l (lowercase L) suffixes. For review/comment. Tony [1] findstr supports regexps - but the regexps don't support alternation |
From @steve-m-hayOn Sun Apr 13 20:35:51 2014, tonyc wrote:
I don't see any new attachment or smoke-me branch for review/comment. Am I missing something? |
From @rurbanI'm following this thread with more and more disgust. Seriously, fix the problems as patched by Mark Polacek who knows what |
From @tonycozOn Fri Apr 18 08:25:02 2014, shay wrote:
Oops, here it is. Tony |
From @tonycoz0001-perl-121505-include-fwrapv-by-default-for-GCC.patchFrom 738956387b9513375e745028974e58b211bd4573 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Mon, 14 Apr 2014 13:28:26 +1000
Subject: [perl #121505] include -fwrapv by default for GCC
Unfortunately dmake/the Win32 command-line make it difficult to parse
a GCC version (even if we pre-extract it), set -fwrapv by default and
make it easy to disable, eg.
dmake GCCWRAPV=undef
---
win32/makefile.mk | 15 ++++++++++++++-
1 files changed, 14 insertions(+), 1 deletions(-)
diff --git a/win32/makefile.mk b/win32/makefile.mk
index c11a7a9..ee25f11 100644
--- a/win32/makefile.mk
+++ b/win32/makefile.mk
@@ -141,6 +141,12 @@ USE_LARGE_FILES *= define
CCTYPE *= GCC
#
+# If you are using GCC, by default we add the -fwrapv option.
+# See https://rt.perl.org/Ticket/Display.html?id=121505
+#
+GCCWRAPV *= define
+
+#
# If you are using Intel C++ Compiler uncomment this
#
#__ICC *= define
@@ -408,12 +414,19 @@ INST_HTML = $(INST_TOP)$(INST_VER)\html
.USESHELL :
+MINIBUILDOPT *=
+
.IF "$(CCTYPE)" == "GCC"
.IF "$(GCCCROSS)" == "define"
ARCHPREFIX = x86_64-w64-mingw32-
.ENDIF
+.IF "$(GCCWRAPV)" == "define"
+BUILDOPT += -fwrapv
+MINIBUILDOPT += -fwrapv
+.ENDIF
+
CC = $(ARCHPREFIX)gcc
LINK32 = $(ARCHPREFIX)g++
LIB32 = $(ARCHPREFIX)ar rc
@@ -1127,7 +1140,7 @@ $(MINIDIR) :
if not exist "$(MINIDIR)" mkdir "$(MINIDIR)"
$(MINICORE_OBJ) : $(CORE_NOCFG_H)
- $(CC) -c $(CFLAGS) -DPERL_EXTERNAL_GLOB -DPERL_IS_MINIPERL $(OBJOUT_FLAG)$@ ..\$(*B).c
+ $(CC) -c $(CFLAGS) $(MINIBUILDOPT) -DPERL_EXTERNAL_GLOB -DPERL_IS_MINIPERL $(OBJOUT_FLAG)$@ ..\$(*B).c
$(MINIWIN32_OBJ) : $(CORE_NOCFG_H)
$(CC) -c $(CFLAGS) -DPERL_IS_MINIPERL $(OBJOUT_FLAG)$@ $(*B).c
--
1.7.4.msysgit.0
|
From @tonycozOn Fri Apr 18 12:30:19 2014, rurban wrote:
That patch fixes only a few of several issues detected by -fsanitize=undefined. Fixing all of the integer overflow undefined behaviour in perl would either mean releasing a perl with possibly tricky changes without extensive testing, or a several month delay while we wait for the changes to settle down. Note that most of these issues are in 5.18, 5.16, 5.14 etc, we're not releasing a perl with new bugs. (well, not these new bugs)
The -fwrapv change is only a workaround, which you know if you're read the thread. I don't plan to close this ticket once these workarounds are in, I'll open new tickets that this ticket depends on for each of the issues -fsanitize=undefined. No-one is happy with the -fwrapv "solution". Tony |
From @steve-m-hayOn Mon Apr 21 16:05:34 2014, tonyc wrote:
Testing with a 32-bit snapshot (http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Personal%20Builds/Experimental_Builds/4.9.0/threads-win32/sjlj/) I see no problem to start with, but with a 64-bit snapshot (http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/Experimental_Builds/4.9.0/threads-win32/seh/) I get failures in op/range.t and op/numconvert.t without the patch. Those failures are fixed by the patch. Why is the -fwrapv left out from the MINIWIN32_OBJ compilation? My test results suggest that we don't currently have a problem with those files (win32.c, win32sck.c, win32thread.c, fcrypt.c and win32io.c) but surely it wouldn't hurt to compile them with it anyway to protect against any future trouble? (Likewise the x2p files, although they hardly matter since they're moving out of core in due course anyway.) Also, since the failure only happens on a 64-bit build, is it worth only including -fwrapv when doing a 64-bit build? That way, users of earlier versions of gcc that don't support -fwrapv aren't bothered by this if they're only doing a 32-bit build anyway. (Or have I just been lucky in not seeing any failure in a 32-bit build?) |
From @steve-m-hayOn Tue Apr 22 05:10:18 2014, shay wrote:
The following also initializes GCCWRAPV better, so that it's only set to "define" by default for 4.3.0 or higer: GCCWRAPV *= $(shell for /f "delims=. tokens=1,2,3" %i in ('gcc -dumpversion') do @if "%i"=="4" (if "%j" geq "3" echo define) else if "%i" geq "5" (echo define)) |
From @steve-m-hayOn 22 April 2014 18:12, Steve Hay via RT <perlbug-followup@perl.org> wrote:
(I forgot: that probably needs putting in a GCC-specific section to |
From @tonycozOn Tue Apr 22 12:37:46 2014, shay wrote:
Oops, this ticket slipped my (alleged) mind. I've attached an updated patch: - uses the shell macro you provided to set a default GCCWRAPV, replacing gcc with $(CC) *after* CC has been set based on the GCCCROSS macro - compiles MINIWIN32_OBJ with $(MINIBUILDOPT) Tony |
From @tonycoz0001-perl-121505-include-fwrapv-by-default-for-GCC-4.3-an.patchFrom e0c717880fdaf205d43f66cd54ba672119129d22 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Mon, 5 May 2014 10:22:25 +1000
Subject: [perl #121505] include -fwrapv by default for GCC 4.3 and later
---
win32/makefile.mk | 19 +++++++++++++++++--
1 files changed, 17 insertions(+), 2 deletions(-)
diff --git a/win32/makefile.mk b/win32/makefile.mk
index 99193a2..e005ee5 100644
--- a/win32/makefile.mk
+++ b/win32/makefile.mk
@@ -141,6 +141,12 @@ USE_LARGE_FILES *= define
CCTYPE *= GCC
#
+# If you are using GCC, 4.3 or later by default we add the -fwrapv option.
+# See https://rt.perl.org/Ticket/Display.html?id=121505
+#
+#GCCWRAPV *= define
+
+#
# If you are using Intel C++ Compiler uncomment this
#
#__ICC *= define
@@ -408,6 +414,8 @@ INST_HTML = $(INST_TOP)$(INST_VER)\html
.USESHELL :
+MINIBUILDOPT *=
+
.IF "$(CCTYPE)" == "GCC"
.IF "$(GCCCROSS)" == "define"
@@ -420,6 +428,13 @@ LIB32 = $(ARCHPREFIX)ar rc
IMPLIB = $(ARCHPREFIX)dlltool
RSC = $(ARCHPREFIX)windres
+GCCWRAPV *= $(shell for /f "delims=. tokens=1,2,3" %i in ('$(CC) -dumpversion') do @if "%i"=="4" (if "%j" geq "3" echo define) else if "%i" geq "5" (echo define))
+
+.IF "$(GCCWRAPV)" == "define"
+BUILDOPT += -fwrapv
+MINIBUILDOPT += -fwrapv
+.ENDIF
+
i = .i
o = .o
a = .a
@@ -1139,10 +1154,10 @@ $(MINIDIR) :
if not exist "$(MINIDIR)" mkdir "$(MINIDIR)"
$(MINICORE_OBJ) : $(CORE_NOCFG_H)
- $(CC) -c $(CFLAGS) -DPERL_EXTERNAL_GLOB -DPERL_IS_MINIPERL $(OBJOUT_FLAG)$@ ..\$(*B).c
+ $(CC) -c $(CFLAGS) $(MINIBUILDOPT) -DPERL_EXTERNAL_GLOB -DPERL_IS_MINIPERL $(OBJOUT_FLAG)$@ ..\$(*B).c
$(MINIWIN32_OBJ) : $(CORE_NOCFG_H)
- $(CC) -c $(CFLAGS) -DPERL_IS_MINIPERL $(OBJOUT_FLAG)$@ $(*B).c
+ $(CC) -c $(CFLAGS) $(MINIBUILDOPT) -DPERL_IS_MINIPERL $(OBJOUT_FLAG)$@ $(*B).c
# -DPERL_IMPLICIT_SYS needs C++ for perllib.c
# rules wrapped in .IFs break Win9X build (we end up with unbalanced []s unless
--
1.7.4.msysgit.0
|
From @steve-m-hayOn 5 May 2014 01:27, Tony Cook via RT <perlbug-followup@perl.org> wrote:
I'm not sure I'd have left the commented-out GCCWRAPV option at the Would it also make sense to only do this ".IF "$(WIN64)" == "define""? |
From @tonycozOn Tue May 06 10:14:50 2014, shay wrote:
A developer might want to disable it to track down the problems -fwrapv hides. In the Configure change we look for the -fno-wrapv option and disable -fwrapv if that (or -fsanitize=undefined) is present, so you can investigate undefined value issues that -fwrapv hides with: ./Configure -Accflags=-fsanitize=undefined ... Including the commented out entry documents it as something the developer can change, or supply on the command-line, like: dmake GCCWRAPV=undef BUILDOPTEXTRA=-fsanitize=undefined
No, these optimizations aren't limited to 64-bit builds. Tony |
From @steve-m-hayOn 8 May 2014 05:42, Tony Cook via RT <perlbug-followup@perl.org> wrote:
Ok, fair enough. This patch looks fine to me then. |
From @jkeenanOn Sun May 11 17:07:01 2014, tonyc wrote:
This ticket has been referenced by another ticket (126820), so this seems to be an appropriate point to ask: Have we addressed the longer-term issues mentioned above? Thank you very much. -- |
From @iabynOn Sat, Dec 05, 2015 at 04:27:11PM -0800, James E Keenan via RT wrote:
No, so it needs to be kept open for now. -- |
From @khwilliamsonOn Mon, 28 Mar 2016 07:49:49 -0700, davem wrote:
This appears to have been dropped. Shouldn't we revisit this in early 5.29? |
See also gh Perl#13690 for some discussion about signed integer overflow.
See also gh Perl#13690 for some discussion about signed integer overflow.
|
@tonycoz clang usan no longer shows undefined behavior except for ListUtils, a cpan module. So I think this ticket is closable. |
Was that a build with -fwrapv removed? |
|
It is clang, and there is no -fwrapv set |
|
No response in nearly a year; The only UB I have seen recently is in List::Utils, so closing |
Migrated from rt.perl.org#121505 (status was 'open')
Searchable as RT121505$
The text was updated successfully, but these errors were encountered: