regcomp.c:18417: void S_skip_to_be_ignored_text(RExC_state_t *, char **, const _Bool): Assertion `! UTF || UTF8_IS_INVARIANT(**p) || UTF8_IS_START(**p)' failed. #16035
Comments
From @dur-randirCreated by @dur-randirWhile fuzzing perl v5.27.1-37-g4c95ee9f29 built with afl and run 00000000 30 2c 73 70 6c 69 74 0d 70 61 63 6b 22 55 75 68 |0,split.pack"Uuh| to cause an assertion failure, even when run under -c for a syntax commit 361446f Allow (#...) anywhere white space is under qr//x Wherever you can have white space under /x, you can also have a (#...) This resolves [perl #116639]. GDB info about the crash location is: (gdb) bt Perl Info |
From @dur-randir |
From @khwilliamsonI looked at this and the assertion that's failing is valid. The problem is that pack is returning malformed UTF-8, which you can see if you add -Dr to the command line options. So I'm unsure how to proceed. |
|
The RT System itself - Status changed from 'new' to 'open' |
From @khwilliamsonIt seems to me that the right fix to this is to forbid pack from returning malformed UTF-8. When I change it to do that, various tests in our suite fail. All these look to be deliberate attempts to generate malformed UTF-8, and testing how this is handled, and they use pack to do that generating. So, it's been known that you can use pack for this, and people have taken advantage of it. No /cpan tests rely on this. But I think we've gotten wise over the years about the perils of malformed UTF-8, and I think this is one that should be fixed. I don't know if it is too late in the 5.28 development cycle to do so, however. But another way of looking at it, it is just in time to fix bugs that would otherwise occur in 5.28 |
From @cpansproutOn Sun, 01 Apr 2018 17:13:52 -0700, khw wrote:
I too agree that it should be fixed.
I’m a little wary of doing it this close to a stable release if it’s possible people are relying on it. -- Father Chrysostomos |
From @xsawyerxOn 04/02/2018 04:16 AM, Father Chrysostomos via RT wrote:
Agreed. I would rather we save it for 5.29. |
From @khwilliamsonOn Mon, 02 Apr 2018 12:28:46 -0700, xsawyerx@gmail.com wrote:
I'm not sure what the best approach is. Attached is a patch that just croaks when the returned SV is UTF-8 and malformed. Another option would be to just turn off the UTF-8 flag. Should this use be deprecated instead? |
From @khwilliamson0001-Trial-patch-for-131642.patchFrom d8f351f05faf4badc3b80de38390cfac14e56418 Mon Sep 17 00:00:00 2001
From: Karl Williamson <khw@cpan.org>
Date: Sun, 1 Jul 2018 22:39:47 -0600
Subject: [PATCH] Trial patch for #131642
---
pp_pack.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/pp_pack.c b/pp_pack.c
index 5e9cc64301..1eed6c4d79 100644
--- a/pp_pack.c
+++ b/pp_pack.c
@@ -3147,6 +3147,15 @@ PP(pp_pack)
packlist(cat, pat, patend, MARK, SP + 1);
+ if (SvUTF8(cat)) {
+ STRLEN result_len;
+ const char * result = SvPV_nomg(cat, result_len);
+
+ if (! is_utf8_string((U8 *) result, result_len)) {
+ Perl_croak(aTHX_ "Malformed UTF-8 string returned from pack");
+ }
+ }
+
SvSETMAGIC(cat);
SP = ORIGMARK;
PUSHs(cat);
--
2.17.1
|
From @khwilliamsonFixed by commit fd879d9 PATCH: [perl #131642] pack returning malformed UTF-8 We'll see if this breaks cpan, with people relying on pcak to create malformed UTF-8. But my current inclination is they should be doing something else to generate it. |
|
@khwilliamson - Status changed from 'open' to 'pending release' |
From @khwilliamsonThank you for filing this report. You have helped make Perl better. With the release today of Perl 5.30.0, this and 160 other issues have been Perl 5.30.0 may be downloaded via: If you find that the problem persists, feel free to reopen this ticket. |
|
@khwilliamson - Status changed from 'pending release' to 'resolved' |
Migrated from rt.perl.org#131642 (status was 'resolved')
Searchable as RT131642$
The text was updated successfully, but these errors were encountered: