tr/// with named Unicode chars broken since 5.23.3

[p5pRT]

Migrated from rt.perl.org#128734 (status was 'resolved')

Searchable as RT128734$

[p5pRT]

From @eserte

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.40 running under perl 5.23.3.

---

The following oneliner works for all perls until 5.23.2, and is broken
with 5.23.3 and later​:

$ perl5.22.2 -e 'use Test​::More "no_plan"; my $string = "\x{00e0}"; $string =~ tr/\N{U+00e0}/A/; is $string, "A";'
ok 1
1..1

$ perl5.24.0 -e 'use Test​::More "no_plan"; my $string = "\x{00e0}"; $string =~ tr/\N{U+00e0}/A/; is $string, "A";'
not ok 1
# Failed test at -e line 1.
# got​: '�'
# expected​: 'A'
1..1
# Looks like you failed 1 test of 1.

Running bisect delivers the following result​:

HEAD is now at f424037 Refactor tr/// parsing to work on EBCDIC, fix other bug
bad - non-zero exit from ./perl -Ilib /tmp/a.pl
f424037 is the first bad commit
commit f424037
Author​: Karl Williamson <khw@​cpan.org>
Date​: Fri Sep 4 11​:32​:26 2015 -0600

  Refactor tr/// parsing to work on EBCDIC, fix other bug
 
  This expands the concept introduced for regular expressions in v5.22 of
  a portable range, to the transliteration operators. A portable range
  has at least one endpoint expressed as \N{} that indicates that the
  Unicode definition is desired, or has the endpoints expressed as both
  uppercase ASCII alphabetic letters or both lowercase ASCII alphabetics.
 
  The refactor fixes several EBCDIC problems, and it fixes the problem in
  all platforms wherein the first endpoint of a range was not checked to
  be <= the final endpoint in UTF-8 strings.
 
  There remains a bug in which if any transliterated code point is larger
  than IV_MAX, perl loops.

:040000 040000 b761e0d3d62cc2854153dd0a7560ea8094b5a3ee 1bee1201b49c33914abafaf8b0be4eba4f0e69d4 M pod
:040000 040000 16dc0b2bc9fc101b3bfaea8455b5c87b86408f71 baf1219eac437e350f84f5b9ee61ccd745cb0b0a M t
:100644 100644 70318a7609eefba7d4ad99cfaefeeb6115d8f323 27e6650b515a9ab1545e866c21dca2589aeab5dc M toke.c
bisect run success
That took 566 seconds.

Regards,
  Slaven

---

---

Flags​:
  category=core
  severity=high

---

Site configuration information for perl 5.23.3​:

Configured by eserte at Tue Sep 29 19​:49​:56 CEST 2015.

Summary of my perl5 (revision 5 version 23 subversion 3) configuration​:
 
  Platform​:
  osname=freebsd, osvers=9.2-release, archname=amd64-freebsd
  uname='freebsd cvrsnica.herceg.de 9.2-release freebsd 9.2-release #0 r255898​: thu sep 26 22​:50​:31 utc 2013 root@​bake.isc.freebsd.org​:usrobjusrsrcsysgeneric amd64 '
  config_args='-ds -e -Dprefix=/usr/perl5.23.3sp -Dusedevel -Dusemallocwrap=no -Duseshrplib -Dcf_email=srezic@​cpan.org'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=undef, usemultiplicity=undef
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_FORTIFY_SOURCE=2',
  optimize='-O',
  cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
  ccversion='', gccversion='4.2.1 20070831 patched [FreeBSD]', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags ='-pthread -Wl,-E -fstack-protector -L/usr/local/lib'
  libpth=/usr/lib /usr/local/lib /usr/include/gcc/4.2 /usr/lib
  libs=-lpthread -lgdbm -lm -lcrypt -lutil -lc
  perllibs=-lpthread -lm -lcrypt -lutil -lc
  libc=, so=so, useshrplib=true, libperl=libperl.so
  gnulibc_version=''
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' -Wl,-R/usr/perl5.23.3sp/lib/5.23.3/amd64-freebsd/CORE'
  cccdlflags='-DPIC -fPIC', lddlflags='-shared -L/usr/local/lib -fstack-protector'

---

@​INC for perl 5.23.3​:
  /usr/perl5.23.3sp/lib/site_perl/5.23.3/amd64-freebsd
  /usr/perl5.23.3sp/lib/site_perl/5.23.3
  /usr/perl5.23.3sp/lib/5.23.3/amd64-freebsd
  /usr/perl5.23.3sp/lib/5.23.3
  .

---

Environment for perl 5.23.3​:
  HOME=/home/e/eserte
  LANG (unset)
  LANGUAGE (unset)
  LC_ALL=de_DE.UTF-8
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
  PATH=/usr/local/bin​:/usr/bin​:/bin​:/usr/local/sbin​:/usr/sbin​:/sbin​:/home/e/eserte/bin/freebsd9.1​:/home/e/eserte/bin/sh​:/home/e/eserte/bin​:/home/e/eserte/bin/pistachio-perl/bin​:/usr/games​:/home/e/eserte/devel
  PERLDOC=-MPod​::Perldoc​::ToTextOverstrike
  PERL_BADLANG (unset)
  SHELL=/usr/local/bin/zsh

[p5pRT]

@khwilliamson - Status changed from 'new' to 'open'

[p5pRT]

From @khwilliamson

Thanks for finding this. Fixed by
3a34ca0
--
Karl Williamson

[p5pRT]

@khwilliamson - Status changed from 'open' to 'pending release'

[p5pRT]

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release today of Perl 5.26.0, this and 210 other issues have been
resolved.

Perl 5.26.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.26.0

If you find that the problem persists, feel free to reopen this ticket.

[p5pRT]

@khwilliamson - Status changed from 'pending release' to 'resolved'