"Attempt to free unreferenced scalar..." with failed regexp compilation including heredoc #13660

p5pRT · 2014-03-14T15:40:51Z

Migrated from rt.perl.org#121437 (status was 'resolved')

Searchable as RT121437$

p5pRT · 2014-03-14T15:40:51Z

From @wolfsage

Created by wolfsage@gmail.com

With perl-5.14.2:

$ perl -e '/(?{print <<END\nok 64 - here-doc in re-eval\nEND\n})/;'
Can't find string terminator "
END" anywhere before EOF at (re_eval 1) line 1.
Compilation failed in regexp at -e line 1.

With blead (and later Perls):

$ perl -e '/(?{print <<END\nok 64 - here-doc in re-eval\nEND\n})/;'
Can't find string terminator "END" anywhere before EOF at -e line 1.
Attempt to free unreferenced scalar: SV 0x23d9080 at -e line 1.

Bisected with attached bisecter.pl:

../perl-1/Porting/bisect.pl -j 8 --start=v5.14.2 --target=miniperl
-- ./miniperl -Ilib /home/mhorsfall/bisecter.pl 2>&1 | tee ~/out.txt

To:

3328ab5 is the first bad commit
commit 3328ab5
Author: Father Chrysostomos <sprout@cpan.org>
Date: Wed Aug 29 12:35:49 2012 -0700

Finish fixing here-docs in re-evals

This commit fixes here-docs in single-line re-evals in files (as
opposed to evals) and here-docs in single-line quote-like operators
inside re-evals.

In both cases, the here-doc parser has to look into an outer
lexing scope to find the here-doc body. And in both cases it
was stomping on PL_linestr (the current line buffer) while
PL_sublex_info.re_eval_start was pointing to an offset in that buffer.
(re_eval_start is used to construct the string to include in the
regexp's stringification once the lexer reaches the end of the
re-eval.)

Fixing this entails moving re_eval_start and re_eval_str to
PL_parser->lex_shared, making the pre-localised values visible.
This is so that the code that peeks into an outer linestr buffer to
steal the here-doc body can set up re_eval_str in the right scope.
(re_eval_str is used to store the re-eval text when the here-
oc parser has no choice but to modify linestr; see also commit
db44426.)

It also entails making the stream-based parser (i.e., that reads from
an input stream) leave PL_linestr alone, instead of clobbering it and
then reconstructing part of it afterwards.

Perl Info


Flags:
    category=core
    severity=low

Site configuration information for perl 5.14.2:

Configured by Debian Project at Mon Mar 18 19:16:26 UTC 2013.

Summary of my perl5 (revision 5 version 14 subversion 2) configuration:

  Platform:
    osname=linux, osvers=2.6.42-37-generic,
archname=x86_64-linux-gnu-thread-multi
    uname='linux batsu 2.6.42-37-generic #58-ubuntu smp thu jan 24
15:28:10 utc 2013 x86_64 x86_64 x86_64 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN
-Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr
-Dprivlib=/usr/share/perl/5.14 -Darchlib=/usr/lib/perl/5.14
-Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5
-Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local
-Dsitelib=/usr/local/share/perl/5.14.2
-Dsitearch=/usr/local/lib/perl/5.14.2 -Dman1dir=/usr/share/man/man1
-Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1
-Dsiteman3dir=/usr/local/man/man3 -Duse64bitint -Dman1ext=1
-Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh
-Ud_ualarm -Uusesfio -Uusenm -Ui_libutil -DDEBUGGING=-g -Doptimize=-O2
-Duseshrplib -Dlibperl=libperl.so.5.14.2 -des'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN
-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing
-pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.6.3', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib
/usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=, so=so, useshrplib=true, libperl=libperl.so.5.14.2
    gnulibc_version='2.15'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib
-fstack-protector'

Locally applied patches:



@INC for perl 5.14.2:
    /etc/perl
    /usr/local/lib/perl/5.14.2
    /usr/local/share/perl/5.14.2
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.14
    /usr/share/perl/5.14
    /usr/local/lib/site_perl
    .


Environment for perl 5.14.2:
    HOME=/home/mhorsfall
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/mhorsfall/perl5/perlbrew/bin:/home/mhorsfall/bin:/home/mhorsfall/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
    PERLBREW_BASHRC_VERSION=0.66
    PERLBREW_HOME=/home/mhorsfall/.perlbrew
    PERLBREW_ROOT=/home/mhorsfall/perl5/perlbrew
    PERL_BADLANG (unset)
    SHELL=/bin/bash

p5pRT · 2014-03-14T15:40:51Z

From @wolfsage

bisecter.pl

p5pRT · 2014-05-18T22:33:47Z

From @jkeenan

On Fri Mar 14 08:40:51 2014, alh wrote:

This is a bug report for perl from wolfsage@gmail.com,
generated with the help of perlbug 1.39 running under perl 5.14.2.

-----------------------------------------------------------------
[Please describe your issue here]

With perl-5.14.2:

$ perl -e '/(?{print <<END\nok 64 - here-doc in re-eval\nEND\n})/;'
Can't find string terminator "
END" anywhere before EOF at (re_eval 1) line 1.
Compilation failed in regexp at -e line 1.

With blead (and later Perls):

$ perl -e '/(?{print <<END\nok 64 - here-doc in re-eval\nEND\n})/;'
Can't find string terminator "END" anywhere before EOF at -e line 1.
Attempt to free unreferenced scalar: SV 0x23d9080 at -e line 1.

I can confirm that I got these results in blead, which means we will get them in 5.20 as well.

Thank you very much.
Jim Keenan

p5pRT · 2014-05-18T22:33:47Z

The RT System itself - Status changed from 'new' to 'open'

p5pRT · 2014-05-18T22:58:44Z

From @jkeenan

On Fri Mar 14 08:40:51 2014, alh wrote:

This is a bug report for perl from wolfsage@gmail.com,
generated with the help of perlbug 1.39 running under perl 5.14.2.

-----------------------------------------------------------------
[Please describe your issue here]

With perl-5.14.2:

$ perl -e '/(?{print <<END\nok 64 - here-doc in re-eval\nEND\n})/;'
Can't find string terminator "
END" anywhere before EOF at (re_eval 1) line 1.
Compilation failed in regexp at -e line 1.

With blead (and later Perls):

$ perl -e '/(?{print <<END\nok 64 - here-doc in re-eval\nEND\n})/;'
Can't find string terminator "END" anywhere before EOF at -e line 1.
Attempt to free unreferenced scalar: SV 0x23d9080 at -e line 1.

Is this really the same issue as https://rt-archive.perl.org/perl5/Ticket/Display.html?id=121438?

Thank you very much.
Jim Keenan

p5pRT · 2014-05-28T01:09:41Z

From @wolfsage

On Sun, May 18, 2014 at 6:58 PM, James E Keenan via RT
<perlbug-followup@perl.org> wrote:

Is this really the same issue as https://rt-archive.perl.org/perl5/Ticket/Display.html?id=121438?

I don't believe so.

This issue is that Perl mishandles the heredoc, #121438 is that
B::Deparse doesn't properly decode such constructs back to their
original form.

-- Matthew Horsfall (alh)

p5pRT · 2015-03-26T13:40:59Z

From @iabyn

On Fri, Mar 14, 2014 at 08:40:52AM -0700, Matthew Horsfall wrote:

With perl-5.14.2:

$ perl \-e '/\(?\{print \<\<END\\nok 64 \- here\-doc in re\-eval\\nEND\\n\}\)/;'
Can't find string terminator "
END" anywhere before EOF at \(re\_eval 1\) line 1\.
Compilation failed in regexp at \-e line 1\.

With blead (and later Perls):

$ perl \-e '/\(?\{print \<\<END\\nok 64 \- here\-doc in re\-eval\\nEND\\n\}\)/;'
Can't find string terminator "END" anywhere before EOF at \-e line 1\.
Attempt to free unreferenced scalar&#8203;: SV 0x23d9080 at \-e line 1\.

This appears to have been fixed sometime between 5.21.5 and 5.21.6

--
The optimist believes that he lives in the best of all possible worlds.
As does the pessimist.

p5pRT · 2015-03-26T14:44:53Z

From @wolfsage

On Thu, Mar 26, 2015 at 9:30 AM, Dave Mitchell <davem@iabyn.com> wrote:

On Fri, Mar 14, 2014 at 08:40:52AM -0700, Matthew Horsfall wrote:
With perl-5.14.2:
$ perl \-e '/$?\{print \<\<END\\nok 64 \- here\-doc in re\-eval\\nEND\\n\}$/;'
Can't find string terminator "
END" anywhere before EOF at $re\_eval 1$ line 1\.
Compilation failed in regexp at \-e line 1\.
With blead (and later Perls):
$ perl \-e '/$?\{print \<\<END\\nok 64 \- here\-doc in re\-eval\\nEND\\n\}$/;'
Can't find string terminator "END" anywhere before EOF at \-e line 1\.
Attempt to free unreferenced scalar&#8203;: SV 0x23d9080 at \-e line 1\.
This appears to have been fixed sometime between 5.21.5 and 5.21.6

According to a bisect, fixed with:

commit fd2709d
Author: Father Chrysostomos <sprout@cpan.org>
Date: Sat Nov 15 13:31:40 2014 -0800

Fix double free with unterminated /(?{ <<END })/

If we are parsing from a stream (file), and we are inside a quote-like
operator, and we find a here-doc marker that tries to extract the
here-doc body from the stream (this is the last line of the quote-like
operator, or it only has one line), and the the here-doc terminator
cannot be found, then we end up trying to free a scalar twice:

$ ./miniperl -e '"${ print <<END"'
Can't find string terminator "END" anywhere before EOF at -e line 1.
Attempt to free temp prematurely: SV 0x7fcafb82fd98, Perl
interpreter: 0x7fcafb803200 at -e line 1.
Attempt to free unreferenced scalar: SV 0x7fcafb82fd98, Perl
interpreter: 0x7fcafb803200 at -e line 1.

I caused that in v5.17.3-187-g3328ab5.

The current line of code in the parser is usually stored in
PL_parser->linestr (aka PL_linestr) and gets freed when the parser
itself is freed.

The heredoc parser, when extracting the body from a stream, tempor-
arily sets aside PL_linestr, replacing it with another SV. If it
doesn’t find the terminator, it frees the PL_linestr value that has
been set aside, under the assumption that parser_free will take care
of freeing the new value.

Inside quote-like operators that does not work, because PL_linestr has
been localised and set to a new value, already prospectively freed via
SAVEFREESV, in sublex_push. So we can’t free that value again.

-- Matthew Horsfall (alh)

p5pRT · 2015-03-26T17:10:51Z

@iabyn - Status changed from 'open' to 'pending release'

p5pRT · 2015-06-02T03:40:59Z

From @khwilliamson

Thanks for submitting this ticket

The issue should be resolved with the release today of Perl v5.22. If you find that the problem persists, feel free to reopen this ticket

--
Karl Williamson for the Perl 5 porters team

p5pRT · 2015-06-02T03:40:59Z

@khwilliamson - Status changed from 'pending release' to 'resolved'

p5pRT closed this as completed Jun 2, 2015

p5pRT added Severity Low distro-Linux type-core labels Oct 19, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

"Attempt to free unreferenced scalar..." with failed regexp compilation including heredoc #13660

"Attempt to free unreferenced scalar..." with failed regexp compilation including heredoc #13660

p5pRT commented Mar 14, 2014

p5pRT commented Mar 14, 2014

p5pRT commented Mar 14, 2014

p5pRT commented May 18, 2014

p5pRT commented May 18, 2014

p5pRT commented May 18, 2014

p5pRT commented May 28, 2014

p5pRT commented Mar 26, 2015

p5pRT commented Mar 26, 2015

p5pRT commented Mar 26, 2015

p5pRT commented Jun 2, 2015

p5pRT commented Jun 2, 2015

"Attempt to free unreferenced scalar..." with failed regexp compilation including heredoc #13660

"Attempt to free unreferenced scalar..." with failed regexp compilation including heredoc #13660

Comments

p5pRT commented Mar 14, 2014

p5pRT commented Mar 14, 2014

From @wolfsage

Created by wolfsage@gmail.com

p5pRT commented Mar 14, 2014

From @wolfsage

p5pRT commented May 18, 2014

From @jkeenan

p5pRT commented May 18, 2014

p5pRT commented May 18, 2014

From @jkeenan

p5pRT commented May 28, 2014

From @wolfsage

p5pRT commented Mar 26, 2015

From @iabyn

p5pRT commented Mar 26, 2015

From @wolfsage

p5pRT commented Mar 26, 2015

p5pRT commented Jun 2, 2015

From @khwilliamson

p5pRT commented Jun 2, 2015