Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.
Sign uptoke.c:3813: char *S_scan_const(char *): Assertion Failed. #15790
Comments
This comment has been minimized.
This comment has been minimized.
From @geeknikTriggered with Perl v5.25.8-132-gc10193e while fuzzing with AFL. od -tx1 test097 ./perl test097 |
This comment has been minimized.
This comment has been minimized.
From @geeknik |
This comment has been minimized.
This comment has been minimized.
From @hvdsOn Tue, 03 Jan 2017 10:57:21 -0800, brian.carpenter@gmail.com wrote:
I'm slightly confused here; I think first we see "m\x{0} \x{0}" which triggers UTF-16LE detection, so the input gets converted into "m \x{3030}\x{6b30}#\x{5c30}\x{3030}x". Next we parse that as a pattern, with \x{3030} as the delimiter, and the //x flag. Next we see the '#', and attempt to scan past the //x-enabled comment with this at toke.c:3260: The string it's trying to scan past at that point is "#\x{5c30}" encoded as {23 e5 b0 b0}. Since send points past the end of our pattern, this does one byte too little, leaving s pointing at the trailing \xb0. We then try to process that remaining byte, and think we've seen invalid utf8. I'm confused though that if I try to bypass the UTF-16LE conversion I don't get the same assert: I'll try to dig further, but it may need more eyes. Hugo |
This comment has been minimized.
This comment has been minimized.
The RT System itself - Status changed from 'new' to 'open' |
This comment has been minimized.
This comment has been minimized.
From @hvdsOn Wed, 04 Jan 2017 05:00:20 -0800, hv wrote:
Oh, I think I understand - two wrongs make a right, so failing to skip all the characters it should meant it was ok for there to be no s==send check at the end of the if/else chain. So I think the fix should actually look like the below. I'll do some more testing. Hugo @@ -3298,6 +3298,11 @@ S_scan_const(pTHX_ char *start) /* End of else if chain - OP_TRANS rejoin rest */ + if (UNLIKELY(s >= send)) { |
This comment has been minimized.
This comment has been minimized.
From @hvdsOn Wed, 04 Jan 2017 05:28:35 -0800, hv wrote:
Should be fixed by 8faf4f3: Hugo |
This comment has been minimized.
This comment has been minimized.
@hvds - Status changed from 'open' to 'pending release' |
This comment has been minimized.
This comment has been minimized.
From @craigberryOn Wed, Jan 4, 2017 at 9:03 AM, Hugo van der Sanden via RT
The new test in t/re/pat.t throws a warning: ok 827 - empty pattern in regex codeblock: produced the right exception message That's in fresh_perl() in t/test.pl where we write out the test open TEST, '>', $tmpfile or die "Cannot open $tmpfile: $!"; Surely we have other places where there are UTF-8 characters embedded |
This comment has been minimized.
This comment has been minimized.
From @hvdsOn Fri, 06 Jan 2017 14:41:29 -0800, craig.a.berry@gmail.com wrote:
I propose the patch below; I think it should be safe but would welcome a second opinion. Hugo commit ef6b07d107b3e73a8c13536ba41cc386f2bf075c [perl #130495] add fresh_perl() option for prog with embedded utf8 Inline Patchdiff --git a/t/re/pat.t b/t/re/pat.t
index a72989f..c5de2cd 100644
--- a/t/re/pat.t
+++ b/t/re/pat.t
@@ -1864,7 +1864,7 @@ EOF_CODE
# [perl #130495] /x comment skipping stopped a byte short, leading
# to assertion failure or 'malformed utf-8 character" warning
fresh_perl_is(
- "use utf8; m{a#\x{124}}x", '', {},
+ "use utf8; m{a#\x{124}}x", '', {wide_chars => 1},
'[perl #130495] utf-8 character at end of /x comment should not misparse',
);
}
diff --git a/t/test.pl b/t/test.pl
index 32d0d64..d86f633 100644
--- a/t/test.pl
+++ b/t/test.pl
@@ -979,6 +979,7 @@ sub fresh_perl {
$runperl_args->{stderr} = 1 unless exists $runperl_args->{stderr};
open TEST, '>', $tmpfile or die "Cannot open $tmpfile: $!";
+ binmode TEST, ':utf8' if $runperl_args->{wide_chars};
print TEST $prog;
close TEST or die "Cannot close $tmpfile: $!";
|
This comment has been minimized.
This comment has been minimized.
From @craigberryOn Fri, Jan 6, 2017 at 7:32 PM, Hugo van der Sanden via RT
I like the opt-in idea so only tests that specify this get it. Looks |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
From @khwilliamsonThank you for filing this report. You have helped make Perl better. With the release today of Perl 5.26.0, this and 210 other issues have been Perl 5.26.0 may be downloaded via: If you find that the problem persists, feel free to reopen this ticket. |
This comment has been minimized.
This comment has been minimized.
@khwilliamson - Status changed from 'pending release' to 'resolved' |
Migrated from rt.perl.org#130495 (status was 'resolved')
Searchable as RT130495$