number parsing

[p5pRT]

Migrated from rt.perl.org#131725 (status was 'resolved')

Searchable as RT131725$

[p5pRT]

From ruud.vantol@booking.com

Created by ruud.vantol@booking.com

perl -wE'say 1e--5'
-4

I think it should die if there is no digit directly after the 'e' and sign.

Perl Info

Flags:
    category=core
    severity=low

[p5pRT]

From @tonycoz

On Sun, 09 Jul 2017 05​:11​:37 -0700, ruud.vantol@​booking.com wrote​:

perl -wE'say 1e--5'
-4

I think it should die if there is no digit directly after the 'e' and sign.

The attached prevents consuming the "e-" if there's no digits in the exponent.

Tony

[p5pRT]

From @tonycoz

0001-perl-131725-ignore-the-exponent-on-a-decimal-float-i.patch

From 01378fadacb66c3905dd881fc28f30f573547aee Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Thu, 3 Aug 2017 12:11:56 +1000
Subject: (perl #131725) ignore the exponent on a decimal float if no digits

Previously the "1e-" in "1e--5" would be treated as "1", but consumed
the "e-".

This wasn't an issue for hex floats.

I considered (and implemented) croaking instead, but this was
inconsistent with the behaviour for hex floats, which only reach this
code if a full hex float has been parsed.
---
 t/lib/croak/toke | 10 ++++++++++
 toke.c           | 21 +++++++++++++++++++--
 2 files changed, 29 insertions(+), 2 deletions(-)

diff --git a/t/lib/croak/toke b/t/lib/croak/toke
index 2603224..c477be0 100644
--- a/t/lib/croak/toke
+++ b/t/lib/croak/toke
@@ -394,3 +394,13 @@ $a = <<~ ;
 
 EXPECT
 Use of bare << to mean <<"" is forbidden at - line 1.
+########
+# NAME incomplete floating point decimal exponent (#131725)
+1e--5
+EXPECT
+Bareword found where operator expected at - line 1, near "1e"
+	(Missing operator before e?)
+Number found where operator expected at - line 1, near "--5"
+	(Missing operator before 5?)
+syntax error at - line 1, near "1e"
+Execution of - aborted due to compilation errors.
diff --git a/toke.c b/toke.c
index 6aa5f26..6de7d09 100644
--- a/toke.c
+++ b/toke.c
@@ -11182,9 +11182,11 @@ Perl_scan_num(pTHX_ const char *start, YYSTYPE* lvalp)
               || UNLIKELY(hexfp && isALPHA_FOLD_EQ(*s, 'p')))
             && strchr("+-0123456789_", s[1]))
         {
-            floatit = TRUE;
+            int exp_digits = 0;
+            const char *save_s = s;
+            char * save_d = d;
 
-	    /* regardless of whether user said 3E5 or 3e5, use lower 'e',
+            /* regardless of whether user said 3E5 or 3e5, use lower 'e',
                ditto for p (hexfloats) */
             if ((isALPHA_FOLD_EQ(*s, 'e'))) {
 		/* At least some Mach atof()s don't grok 'E' */
@@ -11216,6 +11218,7 @@ Perl_scan_num(pTHX_ const char *start, YYSTYPE* lvalp)
 	    /* read digits of exponent */
 	    while (isDIGIT(*s) || *s == '_') {
 	        if (isDIGIT(*s)) {
+                    ++exp_digits;
 		    if (d >= e)
 		        Perl_croak(aTHX_ "%s", number_too_long);
 		    *d++ = *s++;
@@ -11227,6 +11230,20 @@ Perl_scan_num(pTHX_ const char *start, YYSTYPE* lvalp)
 		   lastub = s++;
 		}
 	    }
+
+            if (!exp_digits) {
+                /* no exponent digits, the [eEpP] could be for something else,
+                 * though in practice we don't get here for p since that's preparsed
+                 * earlier, and results in only the 0xX being consumed, so behave similarly
+                 * for decimal floats and consume only the D.DD, leaving the [eE] to the
+                 * next token.
+                 */
+                s = save_s;
+                d = save_d;
+            }
+            else {
+                floatit = TRUE;
+            }
 	}
 
 
-- 
2.1.4

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @tonycoz

On Wed, 02 Aug 2017 19​:15​:25 -0700, tonyc wrote​:

On Sun, 09 Jul 2017 05​:11​:37 -0700, ruud.vantol@​booking.com wrote​:

perl -wE'say 1e--5'
-4

I think it should die if there is no digit directly after the 'e' and
sign.

The attached prevents consuming the "e-" if there's no digits in the
exponent.

Applied as adb0f5c.

If it ends up causing lots of CPAN failures it may need to be changed to a deprecation.

Tony

[p5pRT]

From @xsawyerx

On 08/14/2017 07​:59 AM, Tony Cook via RT wrote​:

On Wed, 02 Aug 2017 19​:15​:25 -0700, tonyc wrote​:

On Sun, 09 Jul 2017 05​:11​:37 -0700, ruud.vantol@​booking.com wrote​:

perl -wE'say 1e--5'
-4

I think it should die if there is no digit directly after the 'e' and
sign.
The attached prevents consuming the "e-" if there's no digits in the
exponent.
Applied as adb0f5c.

If it ends up causing lots of CPAN failures it may need to be changed to a deprecation.

I think that's a good idea.

[p5pRT]

From @tonycoz

On Sun, 13 Aug 2017 22​:59​:59 -0700, tonyc wrote​:

On Wed, 02 Aug 2017 19​:15​:25 -0700, tonyc wrote​:

On Sun, 09 Jul 2017 05​:11​:37 -0700, ruud.vantol@​booking.com wrote​:

perl -wE'say 1e--5'
-4

I think it should die if there is no digit directly after the 'e'
and
sign.

The attached prevents consuming the "e-" if there's no digits in the
exponent.

Applied as adb0f5c.

If it ends up causing lots of CPAN failures it may need to be changed
to a deprecation.

I didn't see any reported issues, closing.

Tony

[p5pRT]

@tonycoz - Status changed from 'open' to 'pending release'

[p5pRT]

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release today of Perl 5.30.0, this and 160 other issues have been
resolved.

Perl 5.30.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.30.0

If you find that the problem persists, feel free to reopen this ticket.

[p5pRT]

@khwilliamson - Status changed from 'pending release' to 'resolved'