Some incomplete expressions treated as valid #17010
Comments
From @khwilliamsonThis is a bug report for perl from khw@khw-xps-8930.(none), These cases are extracted from [perl #112084] filed by Tom Christiansen, the rest of which has been fixed or rejected. % blead -wle 'print 0x - 1' Surely that's again an error? Those are the same kind of thing as this: % blead -wle 'print 3.14e' --tom Flags: Site configuration information for perl 5.31.0: Configured by khw at Wed May 22 08:58:17 MDT 2019. Summary of my perl5 (revision 5 version 31 subversion 0) configuration: @INC for perl 5.31.0: Environment for perl 5.31.0: PATH=/usr/lib/ccache:/home/khw/bin:/home/khw/perl5/perlbrew/bin:/home/khw/print/bin:/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/usr/games:/usr/local/games:/snap/bin:/home/khw/iands/www:/home/khw/cxoffice/bin |
From @tonycozOn Wed, 22 May 2019 11:33:05 -0700, public@khwilliamson.com wrote:
Something like the attached? Tony |
From @tonycoz0001-perl-134125-accept-only-complete-hex-and-binary-lite.patchFrom 4d409f4671be0c6b879a163c0a364c9516789117 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Tue, 4 Jun 2019 16:35:35 +1000
Subject: (perl #134125) accept only complete hex and binary literals
---
t/lib/croak/toke | 16 ++++++++++++++++
t/lib/warnings/toke | 3 ---
t/op/hexfp.t | 4 ++--
toke.c | 9 +++++++++
4 files changed, 27 insertions(+), 5 deletions(-)
diff --git a/t/lib/croak/toke b/t/lib/croak/toke
index 21851229fe..70d307b0c4 100644
--- a/t/lib/croak/toke
+++ b/t/lib/croak/toke
@@ -502,3 +502,19 @@ format=
EXPECT
syntax error at - line 4, next token ???
Execution of - aborted due to compilation errors.
+########
+# NAME [perl #134045] incomplete hex number
+0x x 2;
+0xx 2;
+0x_;
+EXPECT
+Number found where operator expected at - line 1, near "x 2"
+ (Do you need to predeclare x?)
+Bareword found where operator expected at - line 2, near "0xx"
+ (Missing operator before xx?)
+Number found where operator expected at - line 2, near "xx 2"
+ (Do you need to predeclare xx?)
+Bareword found where operator expected at - line 3, near "0x_"
+ (Missing operator before x_?)
+syntax error at - line 1, near "x 2"
+Execution of - aborted due to compilation errors.
diff --git a/t/lib/warnings/toke b/t/lib/warnings/toke
index 83641e5106..5343e957ba 100644
--- a/t/lib/warnings/toke
+++ b/t/lib/warnings/toke
@@ -756,7 +756,6 @@ $a = 1_; print "$a\n";
$a = 01_; print "$a\n";
$a = 0_; print "$a\n";
$a = 0x1_; print "$a\n";
-$a = 0x_; print "$a\n";
$a = 1.2_; print "$a\n";
$a = 1._2; print "$a\n";
$a = 1._; print "$a\n";
@@ -768,12 +767,10 @@ Misplaced _ in number at - line 6.
Misplaced _ in number at - line 7.
Misplaced _ in number at - line 8.
Misplaced _ in number at - line 9.
-Misplaced _ in number at - line 10.
1
1
0
1
-0
1.2
1.2
1
diff --git a/t/op/hexfp.t b/t/op/hexfp.t
index 64f81364b7..8904c48d29 100644
--- a/t/op/hexfp.t
+++ b/t/op/hexfp.t
@@ -138,7 +138,7 @@ sub get_warn() {
{ # Test certain things that are not hexfloats and should stay that way.
eval '0xp3';
- like(get_warn(), qr/Missing operator before p3/);
+ like(get_warn(), qr/Missing operator before xp3/);
eval '5p3';
like(get_warn(), qr/Missing operator before p3/);
@@ -149,7 +149,7 @@ sub get_warn() {
undef $a;
eval '$a = eval "0x.3"';
- is($a, '03');
+ is($a, ''); # 0 x .3
undef $a;
eval '$a = eval "0xc.3"';
diff --git a/toke.c b/toke.c
index cecfa82378..67a31a9b93 100644
--- a/toke.c
+++ b/toke.c
@@ -11025,6 +11025,7 @@ Perl_scan_num(pTHX_ const char *start, YYSTYPE* lvalp)
I32 shift;
bool overflowed = FALSE;
bool just_zero = TRUE; /* just plain 0 or binary number? */
+ bool has_digs = FALSE;
static const NV nvshift[5] = { 1.0, 2.0, 4.0, 8.0, 16.0 };
static const char* const bases[5] =
{ "", "binary", "", "octal", "hexadecimal" };
@@ -11116,6 +11117,7 @@ Perl_scan_num(pTHX_ const char *start, YYSTYPE* lvalp)
digit:
just_zero = FALSE;
+ has_digs = TRUE;
if (!overflowed) {
assert(shift >= 0);
x = u << shift; /* make room for the digit */
@@ -11331,6 +11333,13 @@ Perl_scan_num(pTHX_ const char *start, YYSTYPE* lvalp)
}
}
+ if (shift != 3 && !has_digs) {
+ /* 0x or 0b with no digits, treat it as if the x or b is part of the
+ next token
+ */
+ s = start + 1;
+ }
+
if (overflowed) {
if (n > 4294967295.0)
Perl_ck_warner(aTHX_ packWARN(WARN_PORTABLE),
--
2.11.0
|
|
The RT System itself - Status changed from 'new' to 'open' |
From @tonycozOn Mon, 03 Jun 2019 23:36:37 -0700, tonyc wrote:
Applied as 7259f41. Tony |
|
@tonycoz - Status changed from 'open' to 'pending release' |
|
@tonycoz - Status changed from 'pending release' to 'open' |
From @tonycozOn Tue, 11 Jun 2019 17:18:10 -0700, tonyc wrote:
After some thought, I'd prefer to make this fatal instead. In most cases adjusting the parse position will result in a later syntax error, but in a few cases with 0x it can result in a silent change in behaviour instead, which is bad. The attached makes this fatal. Tony |
From @tonycoz0001-perl-134125-make-no-digits-after-0x-0b-fatal.patchFrom 76edcdedd4305d2c6c57d0116dabe9aa2c91bd9e Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Thu, 20 Jun 2019 11:06:13 +1000
Subject: (perl #134125) make no digits after 0x/0b fatal
My original change in 7259f4194 silently adjusted the parse
position to immediately after the 0 in an incomplete hex or binary
literal. In most cases this leads to a syntax error, but in some
cases, especially with 0x, this can lead to a silent change in
behaviour.
So throw an error instead.
---
pod/perldiag.pod | 5 +++++
t/lib/croak/toke | 14 +++++---------
t/op/hexfp.t | 4 ++--
toke.c | 12 +++++++++---
4 files changed, 21 insertions(+), 14 deletions(-)
diff --git a/pod/perldiag.pod b/pod/perldiag.pod
index 166d29b4bb..325bfce50f 100644
--- a/pod/perldiag.pod
+++ b/pod/perldiag.pod
@@ -4100,6 +4100,11 @@ for some reason the current debugger (e.g. F<perl5db.pl> or a C<Devel::>
module) didn't define a C<DB::sub> routine to be called at the beginning
of each ordinary subroutine call.
+=item No digits found for %s literal
+
+(F) No hexadecimal digits were found following C<0x> or no binary digits
+were found following C<0b>.
+
=item No directory specified for -I
(F) The B<-I> command-line switch requires a directory name as part of the
diff --git a/t/lib/croak/toke b/t/lib/croak/toke
index e7bf609637..4ef6f726fc 100644
--- a/t/lib/croak/toke
+++ b/t/lib/croak/toke
@@ -507,16 +507,12 @@ Execution of - aborted due to compilation errors.
0x x 2;
0xx 2;
0x_;
+0b;
EXPECT
-Number found where operator expected at - line 1, near "x 2"
- (Do you need to predeclare x?)
-Bareword found where operator expected at - line 2, near "0xx"
- (Missing operator before xx?)
-Number found where operator expected at - line 2, near "xx 2"
- (Do you need to predeclare xx?)
-Bareword found where operator expected at - line 3, near "0x_"
- (Missing operator before x_?)
-syntax error at - line 1, near "x 2"
+No digits found for hexadecimal literal at - line 1, near "0x "
+No digits found for hexadecimal literal at - line 2, near "0xx"
+No digits found for hexadecimal literal at - line 3, near "0x_;"
+No digits found for binary literal at - line 4, near "0b;"
Execution of - aborted due to compilation errors.
########
# NAME [perl #130585] close paren in subparse
diff --git a/t/op/hexfp.t b/t/op/hexfp.t
index 8904c48d29..eeb2c9d364 100644
--- a/t/op/hexfp.t
+++ b/t/op/hexfp.t
@@ -138,7 +138,7 @@ sub get_warn() {
{ # Test certain things that are not hexfloats and should stay that way.
eval '0xp3';
- like(get_warn(), qr/Missing operator before xp3/);
+ like(get_warn(), qr/Missing operator before p3/);
eval '5p3';
like(get_warn(), qr/Missing operator before p3/);
@@ -149,7 +149,7 @@ sub get_warn() {
undef $a;
eval '$a = eval "0x.3"';
- is($a, ''); # 0 x .3
+ is($a, undef); # throws an error
undef $a;
eval '$a = eval "0xc.3"';
diff --git a/toke.c b/toke.c
index 694b008761..753f3cbeeb 100644
--- a/toke.c
+++ b/toke.c
@@ -11336,10 +11336,16 @@ Perl_scan_num(pTHX_ const char *start, YYSTYPE* lvalp)
}
if (shift != 3 && !has_digs) {
- /* 0x or 0b with no digits, treat it as if the x or b is part of the
- next token
+ /* 0x or 0b with no digits, treat it as an error.
+ Originally this backed up the parse before the b or
+ x, but that has the potential for silent changes in
+ behaviour, like for: "0x.3" and "0x+$foo".
*/
- s = start + 1;
+ const char *d = s;
+ if (*d) ++d; /* so the user sees the bad non-digit */
+ PL_bufptr = (char *)d; /* so yyerror reports the context */
+ yyerror(Perl_form(aTHX_ "No digits found for %s literal",
+ shift == 4 ? "hexadecimal" : "binary"));
}
if (overflowed) {
--
2.11.0
|
From @tonycozOn Wed, 19 Jun 2019 18:08:28 -0700, tonyc wrote:
Applied as 1ef28cc. Tony |
|
@tonycoz - Status changed from 'open' to 'pending release' |
Migrated from rt.perl.org#134125 (status was 'pending release')
Searchable as RT134125$
The text was updated successfully, but these errors were encountered: