erroneous regex warning after utf8 conversion #15958
Comments
From saint.snit@gmail.comThis is a bug report for perl from saint.snit@gmail.com, Perl emits an inapplicable warning for some regular expressions. use experimental 'smartmatch'; $_ = "x"; This produces the output: Wide character (U+FFFD) in pattern match (m//) at - line 8. even though no such character is ever specified. The bug only seems to occur with the particular combination of "use" (For reference, discussion about this bug originated at Thank you for your time. Flags: Site configuration information for perl 5.22.2: Configured by Gentoo at Thu Oct 20 22:32:43 CDT 2016. Summary of my perl5 (revision 5 version 22 subversion 2) configuration: Locally applied patches: @INC for perl 5.22.2: Environment for perl 5.22.2: |
From zefram@fysh.orgBisecting shows that the warning started appearing for that test script Attempting to minimise the test script, it turns out that the "use $ perl ../rt131190 Looks like the problem is that the check for wide characters should be -zefram |
|
The RT System itself - Status changed from 'new' to 'open' |
From @khwilliamsonOn 04/23/2017 01:33 PM, Zefram wrote:
Actually, the decode function shouldn't be getting called at all if
|
From @demerphqOn 24 April 2017 at 06:19, Karl Williamson <public@khwilliamson.com> wrote:
I pushed a fix for this yesterday. Dave M changed the var "nextchr" so that it supported a negative Unfortunately this doesn't play entirely nicely with the utf8 code if (utf8_target sees the -10, casts it to a U8, producing 255, and then considers it The fix was to change this to if (utf8_target I was going to push this to a branch etc, but I managed to make a big After running out of time cleaning up the stuff that definitely So this issues is fixed and understood, but the exact status of the Yves -- |
From @khwilliamsonOn 04/24/2017 01:20 AM, demerphq wrote:
Note that no test for this problem has been committed. |
From @demerphqNo. Because we are debating if it should be reverted due to code freeze. I Yves On 26 Apr 2017 6:17 p.m., "Karl Williamson" <public@khwilliamson.com> wrote:
|
From @khwilliamsonOn 04/26/2017 10:52 AM, demerphq wrote:
Right, but if we keep it, shouldn't we also push a test? Note that this is a regression introduced in 5.25.
|
From @demerphqOn 26 Apr 2017 7:21 p.m., "Karl Williamson" <public@khwilliamson.com> wrote: On 04/26/2017 10:52 AM, demerphq wrote:
Right, but if we keep it, shouldn't we also push a test? Note that this is a regression introduced in 5.25 I already messed up the change log with a bunch of reverts. Id rather not But one way or another we will need a test for sure. Yves |
From @xsawyerxJust for the official record (and relating to James' email), I'm still (I'd rather discuss it with Yves instead of making a single-sided decision.) Everyone, feel free to provide your opinion. Just keep it polite. :) On 04/26/2017 06:52 PM, demerphq wrote:
|
From @khwilliamsonOn 04/27/2017 05:13 AM, Sawyer X wrote:
I think the patch should stay. It is extremely low risk, and fixes a But, I want to point out that this highlights a flaw in the perl
|
From @demerphqOn 27 April 2017 at 19:35, Karl Williamson <public@khwilliamson.com> wrote:
Summarizes my view too. We regressed in 5.25.x on this, and despite If there were any doubt that this patch could have a secondary
I think I mispoke when I said that nexchr holds a codepoint, I think I If I am correct in this then I believe the concern you raise here is One of us should double check, or maybe Dave can speakup on this. cheers, |
From saint.snit@gmail.com
Not to advocate for or against the patch's inclusion, but the bug has |
From @khwilliamsonOn 04/27/2017 12:32 PM, demerphq wrote:
You're right. I didn't look at the code here before writing. But there I'm now thinking the patch would be better to be
instead of what you have:
The macro is: #define NEXTCHR_IS_EOS (nextchr < 0) so it evaluates to the exact same thing as your patch, but uses the |
From @khwilliamsonOn 04/27/2017 01:42 PM, saint.snit@gmail.com wrote:
Hmmm. I tried 5.24.1 before I said it was introduced in 5.25. |
From saint.snit@gmail.com
Upthread, Zefram said it was introduced in v5.21.7-165-g613abc6. |
From zefram@fysh.orgsaint.snit@gmail.com wrote:
No. The misinterpretation of EOS exists only in the code that checks -zefram |
From @demerphqIf it has been present prior to 5.25 then I will revert. Yves On 28 Apr 2017 00:34, "Zefram" <zefram@fysh.org> wrote:
|
From @demerphqOn 28 April 2017 at 08:55, demerphq <demerphq@gmail.com> wrote:
Since this was present pre 5.25 I have reverted with It still feels kinda wrong but whatever. Yves |
From @iabynOn Thu, Apr 27, 2017 at 08:32:52PM +0200, demerphq wrote:
Yep, at the top of the main loop is this: SET_nextchr; On Thu, Apr 27, 2017 at 01:45:46PM -0600, Karl Williamson wrote:
+1 -- |
From @demerphqOn 28 April 2017 at 10:44, Dave Mitchell <davem@iabyn.com> wrote:
Thanks, so that is taken care of for now.
For what its worth I dont mind doing this but I don't like it much. I #define NEXTCHR_IS_OCTET (nextchr>=0) and then use that. That way we can distinguish between "not end of The other approach BTW would be to add a new UTF8 define that does not Or do both. :-) Anyway, since this bug was present prior to 5.25 I have reverted my cheers, -- |
From @xsawyerxOn 04/28/2017 10:57 AM, demerphq wrote:
Thanks, everyone! :) |
From @khwilliamsonThis was fixed by 2c2da8e but the ticket didn't get closed then |
|
@khwilliamson - Status changed from 'open' to 'pending release' |
From @khwilliamsonThank you for filing this report. You have helped make Perl better. With the release yesterday of Perl 5.28.0, this and 185 other issues have been Perl 5.28.0 may be downloaded via: If you find that the problem persists, feel free to reopen this ticket. |
|
@khwilliamson - Status changed from 'pending release' to 'resolved' |
Migrated from rt.perl.org#131190 (status was 'resolved')
Searchable as RT131190$
The text was updated successfully, but these errors were encountered: