Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

script runs can panic on some inputs #16546

Closed
p5pRT opened this issue May 4, 2018 · 10 comments
Closed

script runs can panic on some inputs #16546

p5pRT opened this issue May 4, 2018 · 10 comments

Comments

@p5pRT
Copy link

p5pRT commented May 4, 2018

Migrated from rt.perl.org#133175 (status was 'resolved')

Searchable as RT133175$

@p5pRT
Copy link
Author

p5pRT commented May 4, 2018

From @khwilliamson

This is a bug report for perl from khw@​khw.(none),
generated with the help of perlbug 1.41 running under perl 5.28.0.


The script feature in regexes can panic. This happens when some memory
gets freed twice, due to its forgetting that it has already been freed.
The solution is to simply set the variables to NULL after freeing.

I don't know what to do about a test. I can figure out a test that's
valid for the current version of Unicode, but which a future version
could easily change and the .t still passes but won't actually be
testing the current failure. This is because this area of Unicode is
still in a lot of flux. I found this bug by testing with a recent, but
not current, Unicode version.



Flags​:
  category=core
  severity=high


Site configuration information for perl 5.28.0​:

Configured by khw at Tue May 1 10​:52​:11 MDT 2018.

Summary of my perl5 (revision 5 version 28 subversion 0) configuration​:
  Commit id​: 38c84d6
  Platform​:
  osname=linux
  osvers=4.10.0-42-generic
  archname=x86_64-linux-thread-multi-ld
  uname='linux khw 4.10.0-42-generic #46-ubuntu smp mon dec 4
14​:38​:01 utc 2017 x86_64 x86_64 x86_64 gnulinux '
  config_args='-des -Uversiononly -Dprefix=/home/khw/blead -Dusedevel
-A'optimize=-ggdb3' -A'optimize=-O0' -Accflags='-DPERL_BOOL_AS_CHAR'
-Accflags='-Wno-deprecated' -Accflags='-DPERL_EXTERNAL_GLOB'
-Dman1dir=none -Dman3dir=none -Dcc=g++ -DDEBUGGING -Dusemorebits
-Dusecbacktrace -Dusethreads'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=define
  usemultiplicity=define
  use64bitint=define
  use64bitall=define
  uselongdouble=define
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='g++'
  ccflags ='-D_REENTRANT -D_GNU_SOURCE -DPERL_BOOL_AS_CHAR
-Wno-deprecated -DPERL_EXTERNAL_GLOB -fwrapv -DDEBUGGING
-fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include
-DUSE_C_BACKTRACE -g -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
-D_FORTIFY_SOURCE=2'
  optimize='-O2 -ggdb3 -O0'
  cppflags='-D_REENTRANT -D_GNU_SOURCE -DPERL_BOOL_AS_CHAR
-Wno-deprecated -DPERL_EXTERNAL_GLOB -fwrapv -DDEBUGGING
-fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='6.3.0 20170406'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='long double'
  nvsize=16
  Off_t='off_t'
  lseeksize=8
  alignbytes=16
  prototype=define
  Linker and Libraries​:
  ld='g++'
  ldflags =' -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/include/c++/6 /usr/include/x86_64-linux-gnu/c++/6
/usr/include/c++/6/backward /usr/local/lib
/usr/lib/gcc/x86_64-linux-gnu/6/include-fixed
/usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib
/usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
  libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.24.so
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version='2.24'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags='-Wl,-E'
  cccdlflags='-fPIC'
  lddlflags='-shared -O2 -ggdb3 -O0 -L/usr/local/lib
-fstack-protector-strong'


@​INC for perl 5.28.0​:
  /home/khw/blead/lib/perl5/site_perl/5.28.0/x86_64-linux-thread-multi-ld
  /home/khw/blead/lib/perl5/site_perl/5.28.0
  /home/khw/blead/lib/perl5/5.28.0/x86_64-linux-thread-multi-ld
  /home/khw/blead/lib/perl5/5.28.0
  /home/khw/blead/lib/perl5/site_perl/5.27.11
  /home/khw/blead/lib/perl5/site_perl/5.27.10
  /home/khw/blead/lib/perl5/site_perl/5.27.9
  /home/khw/blead/lib/perl5/site_perl/5.27.8
  /home/khw/blead/lib/perl5/site_perl/5.27.7
  /home/khw/blead/lib/perl5/site_perl/5.27.6
  /home/khw/blead/lib/perl5/site_perl/5.27.5
  /home/khw/blead/lib/perl5/site_perl/5.27.4
  /home/khw/blead/lib/perl5/site_perl/5.27.3
  /home/khw/blead/lib/perl5/site_perl/5.27.2
  /home/khw/blead/lib/perl5/site_perl/5.27.1
  /home/khw/blead/lib/perl5/site_perl/5.27.0
  /home/khw/blead/lib/perl5/site_perl/5.26.0
  /home/khw/blead/lib/perl5/site_perl/5.25.12
  /home/khw/blead/lib/perl5/site_perl/5.25.11
  /home/khw/blead/lib/perl5/site_perl/5.25.10
  /home/khw/blead/lib/perl5/site_perl/5.25.9
  /home/khw/blead/lib/perl5/site_perl/5.25.8
  /home/khw/blead/lib/perl5/site_perl/5.25.7
  /home/khw/blead/lib/perl5/site_perl/5.25.6
  /home/khw/blead/lib/perl5/site_perl/5.25.5
  /home/khw/blead/lib/perl5/site_perl/5.25.4
  /home/khw/blead/lib/perl5/site_perl/5.25.3
  /home/khw/blead/lib/perl5/site_perl/5.25.2
  /home/khw/blead/lib/perl5/site_perl/5.25.1
  /home/khw/blead/lib/perl5/site_perl/5.24.0
  /home/khw/blead/lib/perl5/site_perl/5.23.10
  /home/khw/blead/lib/perl5/site_perl/5.23.9
  /home/khw/blead/lib/perl5/site_perl/5.23.8
  /home/khw/blead/lib/perl5/site_perl/5.23.7
  /home/khw/blead/lib/perl5/site_perl/5.23.6
  /home/khw/blead/lib/perl5/site_perl/5.23.5
  /home/khw/blead/lib/perl5/site_perl/5.23.4
  /home/khw/blead/lib/perl5/site_perl/5.23.3
  /home/khw/blead/lib/perl5/site_perl/5.23.2
  /home/khw/blead/lib/perl5/site_perl/5.23.1
  /home/khw/blead/lib/perl5/site_perl/5.23.0
  /home/khw/blead/lib/perl5/site_perl/5.22.0
  /home/khw/blead/lib/perl5/site_perl/5.21.12
  /home/khw/blead/lib/perl5/site_perl/5.21.11
  /home/khw/blead/lib/perl5/site_perl/5.21.10
  /home/khw/blead/lib/perl5/site_perl/5.21.9
  /home/khw/blead/lib/perl5/site_perl/5.21.8
  /home/khw/blead/lib/perl5/site_perl/5.21.7
  /home/khw/blead/lib/perl5/site_perl/5.21.6
  /home/khw/blead/lib/perl5/site_perl/5.21.5
  /home/khw/blead/lib/perl5/site_perl/5.21.4
  /home/khw/blead/lib/perl5/site_perl/5.21.3
  /home/khw/blead/lib/perl5/site_perl/5.21.2
  /home/khw/blead/lib/perl5/site_perl/5.21.1
  /home/khw/blead/lib/perl5/site_perl/5.20.0
  /home/khw/blead/lib/perl5/site_perl/5.19.12
  /home/khw/blead/lib/perl5/site_perl/5.19.11
  /home/khw/blead/lib/perl5/site_perl/5.19.10
  /home/khw/blead/lib/perl5/site_perl


Environment for perl 5.28.0​:
  HOME=/home/khw
  LANG=en_US.UTF-8
  LANGUAGE=en_US
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)

PATH=/usr/lib/ccache​:/home/khw/bin​:/home/khw/perl5/perlbrew/bin​:/home/khw/print/bin​:/bin​:/usr/local/sbin​:/usr/local/bin​:/usr/sbin​:/usr/bin​:/sbin​:/usr/games​:/usr/local/games​:/home/khw/iands/www​:/home/khw/cxoffice/bin
  PERL5OPT=-w
  PERL_BADLANG (unset)
  PERL_DIFF_TOOL=wgdiff
  PERL_POD_PEDANTIC=1
  SHELL=/bin/ksh

@p5pRT
Copy link
Author

p5pRT commented May 5, 2018

From @jkeenan

On Fri, 04 May 2018 21​:34​:24 GMT, public@​khwilliamson.com wrote​:

This is a bug report for perl from khw@​khw.(none),
generated with the help of perlbug 1.41 running under perl 5.28.0.

-----------------------------------------------------------------
The script feature in regexes can panic. This happens when some
memory
gets freed twice, due to its forgetting that it has already been
freed.
The solution is to simply set the variables to NULL after freeing.

I don't know what to do about a test. I can figure out a test that's
valid for the current version of Unicode, but which a future version
could easily change and the .t still passes but won't actually be
testing the current failure. This is because this area of Unicode is
still in a lot of flux. I found this bug by testing with a recent,
but
not current, Unicode version.

Given the murkiness of the situation, I would be surprised if we can find a solution in time for perl-5.28.0. We should therefore consider yanking it from blead.

Thank you very much.
Jim Keenan

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented May 5, 2018

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented May 5, 2018

From @khwilliamson

On 05/04/2018 08​:09 PM, James E Keenan via RT wrote​:

On Fri, 04 May 2018 21​:34​:24 GMT, public@​khwilliamson.com wrote​:

This is a bug report for perl from khw@​khw.(none),
generated with the help of perlbug 1.41 running under perl 5.28.0.

-----------------------------------------------------------------
The script feature in regexes can panic. This happens when some
memory
gets freed twice, due to its forgetting that it has already been
freed.
The solution is to simply set the variables to NULL after freeing.

I don't know what to do about a test. I can figure out a test that's
valid for the current version of Unicode, but which a future version
could easily change and the .t still passes but won't actually be
testing the current failure. This is because this area of Unicode is
still in a lot of flux. I found this bug by testing with a recent,
but
not current, Unicode version.

Given the murkiness of the situation, I would be surprised if we can find a solution in time for perl-5.28.0. We should therefore consider yanking it from blead.

Thank you very much.
Jim Keenan

I don't see that the situation is murky at all. Attached is a patch
that fixes the problem.

Note that this feature is marked experimental.

@p5pRT
Copy link
Author

p5pRT commented May 5, 2018

From @khwilliamson

0039-PATCH-perl-133175-script-run-free-from-wrong-pool-pa.patch
From 9ef19eee5bfe6edfc9688b38e6a131175f006f39 Mon Sep 17 00:00:00 2001
From: Karl Williamson <khw@cpan.org>
Date: Fri, 4 May 2018 21:26:31 -0600
Subject: [PATCH 39/39] PATCH: [perl #133175] script run free from wrong pool
 panic

Setting the pointer to NULL after freeing signals the code in later
interations that it has been freed already
---
 regexec.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/regexec.c b/regexec.c
index 9a5e87e9e5..56d5b10fd8 100644
--- a/regexec.c
+++ b/regexec.c
@@ -10583,9 +10583,11 @@ Perl_isSCRIPT_RUN(pTHX_ const U8 * s, const U8 * send, const bool utf8_target)
             /* If there is only a single script in common, set to that.
              * Otherwise, use the intersection going forward */
             Safefree(intersection);
+            intersection = NULL;
             if (intersection_len == 1) {
                 script_of_run = script_of_char = new_overlap[0];
                 Safefree(new_overlap);
+                new_overlap = NULL;
             }
             else {
                 intersection = new_overlap;
-- 
2.11.0

@p5pRT
Copy link
Author

p5pRT commented May 9, 2018

From @xsawyerx

On 05/05/2018 06​:36 AM, Karl Williamson wrote​:

On 05/04/2018 08​:09 PM, James E Keenan via RT wrote​:

On Fri, 04 May 2018 21​:34​:24 GMT, public@​khwilliamson.com wrote​:

This is a bug report for perl from khw@​khw.(none),
generated with the help of perlbug 1.41 running under perl 5.28.0.

-----------------------------------------------------------------
The script feature in regexes can panic.  This happens when some
memory
gets freed twice, due to its forgetting that it has already been
freed.
The solution is to simply set the variables to NULL after freeing.

I don't know what to do about a test.  I can figure out a test that's
valid for the current version of Unicode, but which a future version
could easily change and the .t still passes but won't actually be
testing the current failure.  This is because this area of Unicode is
still in a lot of flux.  I found this bug by testing with a recent,
but
not current,  Unicode version.

Given the murkiness of the situation, I would be surprised if we can
find a solution in time for perl-5.28.0.  We should therefore
consider yanking it from blead.

Thank you very much.
Jim Keenan

I don't see that the situation is murky at all.

Agreed.

  Attached is a patch that fixes the problem.

Do you think it should be merged now?

@p5pRT
Copy link
Author

p5pRT commented May 9, 2018

From @khwilliamson

On 05/09/2018 01​:09 PM, Sawyer X via RT wrote​:

On 05/05/2018 06​:36 AM, Karl Williamson wrote​:

On 05/04/2018 08​:09 PM, James E Keenan via RT wrote​:

On Fri, 04 May 2018 21​:34​:24 GMT, public@​khwilliamson.com wrote​:

This is a bug report for perl from khw@​khw.(none),
generated with the help of perlbug 1.41 running under perl 5.28.0.

-----------------------------------------------------------------
The script feature in regexes can panic.  This happens when some
memory
gets freed twice, due to its forgetting that it has already been
freed.
The solution is to simply set the variables to NULL after freeing.

I don't know what to do about a test.  I can figure out a test that's
valid for the current version of Unicode, but which a future version
could easily change and the .t still passes but won't actually be
testing the current failure.  This is because this area of Unicode is
still in a lot of flux.  I found this bug by testing with a recent,
but
not current,  Unicode version.

Given the murkiness of the situation, I would be surprised if we can
find a solution in time for perl-5.28.0.  We should therefore
consider yanking it from blead.

Thank you very much.
Jim Keenan

I don't see that the situation is murky at all.

Agreed.

  Attached is a patch that fixes the problem.

Do you think it should be merged now?

Yes.

@p5pRT
Copy link
Author

p5pRT commented May 9, 2018

From @xsawyerx

On 05/09/2018 10​:22 PM, Karl Williamson wrote​:

On 05/09/2018 01​:09 PM, Sawyer X via RT wrote​:

On 05/05/2018 06​:36 AM, Karl Williamson wrote​:

On 05/04/2018 08​:09 PM, James E Keenan via RT wrote​:

On Fri, 04 May 2018 21​:34​:24 GMT, public@​khwilliamson.com wrote​:

This is a bug report for perl from khw@​khw.(none),
generated with the help of perlbug 1.41 running under perl 5.28.0.

-----------------------------------------------------------------
The script feature in regexes can panic.  This happens when some
memory
gets freed twice, due to its forgetting that it has already been
freed.
The solution is to simply set the variables to NULL after freeing.

I don't know what to do about a test.  I can figure out a test that's
valid for the current version of Unicode, but which a future version
could easily change and the .t still passes but won't actually be
testing the current failure.  This is because this area of Unicode is
still in a lot of flux.  I found this bug by testing with a recent,
but
not current,  Unicode version.

Given the murkiness of the situation, I would be surprised if we can
find a solution in time for perl-5.28.0.  We should therefore
consider yanking it from blead.

Thank you very much.
Jim Keenan

I don't see that the situation is murky at all.

Agreed.

   Attached is a patch that fixes the problem.

Do you think it should be merged now?

Yes.

Go for it.

@p5pRT
Copy link
Author

p5pRT commented May 9, 2018

From @khwilliamson

This was fixed by
77dddf9
--
Karl Williamson

@p5pRT p5pRT closed this as completed May 9, 2018
@p5pRT
Copy link
Author

p5pRT commented May 9, 2018

@khwilliamson - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant