Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

s// on utf8 string occasionally crashes with "Malformed UTF-8 character" #16019

Closed
p5pRT opened this issue Jun 14, 2017 · 6 comments
Closed

s// on utf8 string occasionally crashes with "Malformed UTF-8 character" #16019

p5pRT opened this issue Jun 14, 2017 · 6 comments
Labels

Comments

@p5pRT
Copy link
Collaborator

@p5pRT p5pRT commented Jun 14, 2017

Migrated from rt.perl.org#131575 (status was 'resolved')

Searchable as RT131575$

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 14, 2017

From @ntyni

This is a bug report for perl from Niko Tyni <ntyni@​debian.org>,
generated with the help of perlbug 1.40 running under perl 5.27.1.


As reported by Benjamin Bayart in https://bugs.debian.org/864782 a
regression between 5.22 and 5.24 made s// on a valid utf8 string start
to crash occasionally with 'Malformed UTF-8 character' diagnostics.

I was able to get it down to

#!perl
my $text = "%t%\x{6bce}";
$text =~ s{~*%[a-z]%}{}g;
print "ok\n";

which crashes for me on blead ~50% of the time with something like

Malformed UTF-8 character​: \x91 (unexpected continuation byte 0x91, with no preceding start byte) in substitution (s///) at /home/ntyni/864782.pl line 3.
Malformed UTF-8 character (fatal) at /home/ntyni/864782.pl line 3.

Running this with valgrind shows invalid reads, and bisecting
indicates that those errors started with v5.23.3-110-g147f21b5b8 :

  commit 147f21b
  Author​: David Mitchell <davem@​iabyn.com>
  Date​: Sat Sep 26 13​:12​:40 2015 +0100

  make /fixed-substr/ much faster.



Flags​:
  category=core
  severity=medium


Site configuration information for perl 5.27.1​:

Configured by ntyni at Wed Jun 14 20​:16​:03 BST 2017.

Summary of my perl5 (revision 5 version 27 subversion 1) configuration​:
  Commit id​: b50535d
  Platform​:
  osname=linux
  osvers=4.9.0-3-amd64
  archname=x86_64-linux
  uname='linux carme 4.9.0-3-amd64 #1 smp debian 4.9.25-1 (2017-05-02) x86_64 gnulinux '
  config_args='-des -Dusedevel -DDEBUGGING'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=undef
  usemultiplicity=undef
  use64bitint=define
  use64bitall=define
  uselongdouble=undef
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='cc'
  ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
  optimize='-O2 -g'
  cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='6.3.0 20170516'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='off_t'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='cc'
  ldflags =' -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/6/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
  libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.24.so
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version='2.24'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags='-Wl,-E'
  cccdlflags='-fPIC'
  lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector-strong'


@​INC for perl 5.27.1​:
  lib
  /usr/local/lib/perl5/site_perl/5.27.1/x86_64-linux
  /usr/local/lib/perl5/site_perl/5.27.1
  /usr/local/lib/perl5/5.27.1/x86_64-linux
  /usr/local/lib/perl5/5.27.1


Environment for perl 5.27.1​:
  HOME=/home/ntyni
  LANG=en_GB.UTF-8
  LANGUAGE=en_GB​:en
  LC_CTYPE=fi_FI.UTF-8
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
  PATH=/usr/local/bin​:/usr/bin​:/bin​:/usr/games
  PERL_BADLANG (unset)
  SHELL=/bin/zsh

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 16, 2017

From @iabyn

On Wed, Jun 14, 2017 at 01​:00​:12PM -0700, Niko Tyni wrote​:

As reported by Benjamin Bayart in https://bugs.debian.org/864782 a
regression between 5.22 and 5.24 made s// on a valid utf8 string start
to crash occasionally with 'Malformed UTF-8 character' diagnostics.

I was able to get it down to

#!perl
my $text = "%t%\x{6bce}";
$text =~ s{~*%[a-z]%}{}g;
print "ok\n";

which crashes for me on blead ~50% of the time with something like

Malformed UTF-8 character​: \x91 (unexpected continuation byte 0x91, with no preceding start byte) in substitution (s///) at /home/ntyni/864782.pl line 3.
Malformed UTF-8 character (fatal) at /home/ntyni/864782.pl line 3.

Running this with valgrind shows invalid reads, and bisecting
indicates that those errors started with v5.23.3-110-g147f21b5b8 :

commit 147f21b
Author​: David Mitchell <davem@​iabyn.com>
Date​: Sat Sep 26 13​:12​:40 2015 +0100

make /fixed\-substr/ much faster\.

Now fixed by v5.27.0-322-gbb152a4

--
Monto Blanco... scorchio!

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 16, 2017

The RT System itself - Status changed from 'new' to 'open'

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 17, 2017

@iabyn - Status changed from 'open' to 'pending release'

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 23, 2018

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release yesterday of Perl 5.28.0, this and 185 other issues have been
resolved.

Perl 5.28.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.28.0

If you find that the problem persists, feel free to reopen this ticket.

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 23, 2018

@khwilliamson - Status changed from 'pending release' to 'resolved'

@p5pRT p5pRT closed this Jun 23, 2018
@p5pRT p5pRT added the Severity Low label Oct 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.