SEGV caused by isLEXWARN_off while PL_curcop is NULL(gp_free vs dounwind).

[p5pRT]

Migrated from rt.perl.org#128597 (status was 'resolved')

Searchable as RT128597$

[p5pRT]

From @hkoba

Created by @hkoba

I found following script causes SEGV, in perl5.22 ~ blead 59a08c7

  perl -e 'open my $fh, ">", \ (my $buf = ""); my $sub = eval q|sub {die}|; $sub->()'
 
Below is a backtrace from the blead​:

  (gdb) run -Ilib -e 'open my $fh, ">", \ (my $buf = ""); my $sub = eval q|sub {die}|; $sub->()'
  Starting program​: /home/hkoba/blob/src/perl5/perl -Ilib -e 'open my $fh, ">", \ (my $buf = ""); my $sub = eval q|sub {die}|; $sub->()'
  [Thread debugging using libthread_db enabled]
  Using host libthread_db library "/lib64/libthread_db.so.1".
  Died at (eval 1) line 1.
 
  Program received signal SIGSEGV, Segmentation fault.
  0x0000000000562460 in Perl_ckwarn_d (w=5) at util.c​:2072
  2072 if (isLEXWARN_off)
  (gdb) bt
  #0 0x0000000000562460 in Perl_ckwarn_d (w=5) at util.c​:2072
  #1 0x0000000000483f8e in Perl_gp_free (gv=0xaa7178) at gv.c​:2559
  #2 0x00000000005f89cc in Perl_sv_clear (orig_sv=0xaa7178) at sv.c​:6593
  #3 0x00000000005fb108 in Perl_sv_free2 (sv=0xaa7178, rc=1) at sv.c​:6962
  #4 0x00000000005c5049 in S_SvREFCNT_dec_NN (sv=0xaa7178) at inline.h​:177
  #5 0x000000000060f0e0 in Perl_sv_unref_flags (ref=0xac5830, flags=5)
  at sv.c​:10488
  #6 0x00000000005edf0b in Perl_sv_force_normal_flags (sv=0xac5830, flags=5)
  at sv.c​:5222
  #7 0x000000000065a2a3 in Perl_leave_scope (base=0) at scope.c​:1100
  #8 0x0000000000668096 in Perl_dounwind (cxix=-1) at pp_ctl.c​:1527
  #9 0x000000000046ec90 in S_my_exit_jump () at perl.c​:5191
  #10 0x000000000046eb00 in Perl_my_failure_exit () at perl.c​:5178
  #11 0x0000000000669257 in Perl_die_unwind (msv=0xabc2d0) at pp_ctl.c​:1744
  #12 0x0000000000561550 in Perl_croak_sv (baseex=0xabc360) at util.c​:1758
  #13 0x0000000000561439 in Perl_die_sv (baseex=0xabc360) at util.c​:1668
  #14 0x000000000068851e in Perl_pp_die () at pp_sys.c​:523
  #15 0x000000000055b0a5 in Perl_runops_debug () at dump.c​:2232
  #16 0x0000000000462ae0 in S_run_body (oldscope=1) at perl.c​:2521
  #17 0x000000000046210b in perl_run (my_perl=0xaa5010) at perl.c​:2444
  #18 0x000000000041efce in main (argc=4, argv=0x7fffffffda28,
  env=0x7fffffffda50) at perlmain.c​:123
  (gdb)
 

Perl Info

Flags:
    category=core
    severity=high

Site configuration information for perl 5.22.2:

Configured by hkoba at Tue Jun 14 14:28:25 JST 2016.

Summary of my perl5 (revision 5 version 22 subversion 2) configuration:
   
  Platform:
    osname=linux, osvers=4.4.11-200.fc22.x86_64, archname=x86_64-linux
    uname='linux chaika.localdomain 4.4.11-200.fc22.x86_64 #1 smp tue may 24 00:20:46 utc 2016 x86_64 x86_64 x86_64 gnulinux '
    config_args='-Dprefix=/home/hkoba/db/perl5/plenv/versions/5.22.2 -de -Dusedevel -A'eval:scriptdir=/home/hkoba/db/perl5/plenv/versions/5.22.2/bin''
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2',
    optimize='-O2',
    cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion='', gccversion='5.3.1 20160406 (Red Hat 5.3.1-6)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib /lib/../lib64 /usr/lib/../lib64 /lib /lib64 /usr/lib64 /usr/local/lib64
    libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
    perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.21.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.21'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'

Locally applied patches:
    Devel::PatchPerl 1.40


@INC for perl 5.22.2:
    /home/hkoba/db/opam/dot.opam/4.02.0/lib/perl5
    /home/hkoba/.opam/4.02.0/lib/perl5
    /home/hkoba/db/perl5/plenv/versions/5.22.2/lib/site_perl/5.22.2/x86_64-linux
    /home/hkoba/db/perl5/plenv/versions/5.22.2/lib/site_perl/5.22.2
    /home/hkoba/db/perl5/plenv/versions/5.22.2/lib/5.22.2/x86_64-linux
    /home/hkoba/db/perl5/plenv/versions/5.22.2/lib/5.22.2
    .


Environment for perl 5.22.2:
    HOME=/home/hkoba
    LANG=ja_JP.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/hkoba/db/perl5/plenv/versions/5.22.2/bin:/home/hkoba/db/perl5/.plenv/libexec:/home/hkoba/db/perl5/plenv/plugins/perl-build/bin:/home/hkoba/db/opam/dot.opam/4.02.0/bin:/home/hkoba/db/perl5/plenv/shims:/home/hkoba/db/perl5/.plenv/bin:/home/hkoba/.opam/4.02.0/bin:/home/hkoba/db/tools/nodebrew/current/bin:/home/ssri/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/home/hkoba/bin:/home/hkoba/Local/bin:/home/hkoba/Configs/bin:/home/hkoba/Configs/common/bin:/home/hkoba/mytools:/home/hkoba/db/bin:/home/hkoba/db/mytools:/home/hkoba/db/mykit/bin:/home/hkoba/db/opam:/home/hkoba/db/tools/android-ndk:/home/hkoba/db/src/perl6/rakudobrew/bin/
    PERL5LIB=/home/hkoba/db/opam/dot.opam/4.02.0/lib/perl5:/home/hkoba/.opam/4.02.0/lib/perl5
    PERL_BADLANG (unset)
    SHELL=/bin/zsh

[p5pRT]

From @cpansprout

On Sun Jul 10 20​:00​:12 2016, hkoba@​cpan.org wrote​:

This is a bug report for perl from hkoba@​cpan.org,
generated with the help of perlbug 1.40 running under perl 5.22.2.

-----------------------------------------------------------------
[Please describe your issue here]

I found following script causes SEGV, in perl5.22 ~ blead 59a08c7

perl -e 'open my $fh, ">", \ (my $buf = ""); my $sub = eval q|sub
{die}|; $sub->()'

Bisect​:

96d7c88 is the first bad commit
commit 96d7c88
Author​: Father Chrysostomos <sprout@​cpan.org>
Date​: Wed Sep 17 21​:16​:58 2014 -0800

  [perl #57512] Warnings for implicitly closed handles

--

Father Chrysostomos

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From zefram@fysh.org

Father Chrysostomos via RT wrote​:

On Sun Jul 10 20​:00​:12 2016, hkoba@​cpan.org wrote​:

perl -e 'open my $fh, ">", \ (my $buf = ""); my $sub = eval q|sub
{die}|; $sub->()'

Bisect​:
commit 96d7c88
[perl #57512] Warnings for implicitly closed handles

The crash is dependent on both $fh and $sub being lexical variables,
and on them being declared in that order. What's interesting about this
requirement is that it means that $fh gets implicitly closed when the
pad is freed, and the crash only happens if $sub has already been freed
at that moment. Indeed, another way to avert the crash is to stick a
reference to $sub into another lexical variable declared before $fh,
so it does seem to be order of freeing that matters.

Is there perhaps a reference to $sub on the context stack, which
gets followed into freed memory while processing the closure of $fh?
No warning is actually generated, but maybe the context is examined
while checking the warning flags?

-zefram

[p5pRT]

From @cpansprout

On Mon Jul 11 10​:53​:15 2016, zefram@​fysh.org wrote​:

Father Chrysostomos via RT wrote​:

On Sun Jul 10 20​:00​:12 2016, hkoba@​cpan.org wrote​:

perl -e 'open my $fh, ">", \ (my $buf = ""); my $sub = eval q|sub
{die}|; $sub->()'

Bisect​:
commit 96d7c88
[perl #57512] Warnings for implicitly closed handles

The crash is dependent on both $fh and $sub being lexical variables,
and on them being declared in that order. What's interesting about this
requirement is that it means that $fh gets implicitly closed when the
pad is freed, and the crash only happens if $sub has already been freed
at that moment. Indeed, another way to avert the crash is to stick a
reference to $sub into another lexical variable declared before $fh,
so it does seem to be order of freeing that matters.

Is there perhaps a reference to $sub on the context stack, which
gets followed into freed memory while processing the closure of $fh?
No warning is actually generated, but maybe the context is examined
while checking the warning flags?

I don’t think the context stack is related per se. It probably has to do with the current cop being freed. op.c​:S_cop_free sets PL_curcop to NULL if it points to the cop being freed.

If the $sub is freed first, then the statement containing the ‘die’, which PL_curcop still points to, gets freed, and PL_curcop is set to NULL. Then $fh is freed, which tries to check the warning flags, but I don’t know why.

I have not confirmed any of this. It also does not explain why the eval is necessary.

I suspect any warning that can happen during freeing could result in a similar crash. (Are there any other such warnings?)

--

Father Chrysostomos

[p5pRT]

From zefram@fysh.org

Father Chrysostomos via RT wrote​:

Then $fh is freed, which tries to check the warning flags, but I don't
know why.

The added code in gv.c checks whether I/O warnings are enabled as part
of deciding whether to try closing a writable I/O handle. The test case
also depends on the handle being writable; if opened in "<" mode then it
doesn't crash. The check of the warning is done via ckWARN_d(WARN_IO),
which is implemented by ckwarn_d(), which tests isLEXWARN_off, which
dereferences PL_curcop without checking for nullness.

It also does not explain why the eval is necessary.

Without it, the sub is additionally referenced by the anoncode op in the
main program, so the cop isn't freed that early. With string eval the
equivalent anoncode op is in the temporary code generated by the eval,
which will be freed on unwind.

I suspect any warning that can happen during freeing could result in
a similar crash. (Are there any other such warnings?)

Good question. I'd also like to question why the I/O closure warning
is being checked against the lexical warning flags of a place that is no
longer executing and doesn't have any real connection to the I/O handle.

-zefram

[p5pRT]

From @cpansprout

On Mon Jul 11 13​:15​:13 2016, zefram@​fysh.org wrote​:

Father Chrysostomos via RT wrote​:

I suspect any warning that can happen during freeing could result in
a similar crash. (Are there any other such warnings?)

Good question.

I do not see any other warnings (in [shag].c and pad.c) except things that indicate internal inconsistency, such as ‘Attempt to free unreferenced glob pointers’.

I'd also like to question why the I/O closure warning
is being checked against the lexical warning flags of a place that is no
longer executing and doesn't have any real connection to the I/O handle.

This is one of the inherent problems with trying to warn during scope exit. Which scope is active? In this particular case, no scope is active, so $^W applies.

This is one reason why

  use warnings FATAL => 'io​::whatever';

is not a reliable way to fatalise this warning, though people have requested such a feature.

I don’t know the solution to that particular problem.

--

Father Chrysostomos

[p5pRT]

From zefram@fysh.org

Father Chrysostomos via RT wrote​:

I don't know the solution to that particular problem.

Whether a particular I/O handle should be warned for, upon implicit
close, seems like a feature of the handle, which should be stored as
a flag in the handle structure. Obviously the flag state has to be
initialised upon opening, and by default the flag could reasonably be
set according to the warning flag prevailing *at the point of the open*.
(It should also be possible to control the flag explicitly per handle.
This could be done as a self-popping I/O layer, which would allow both
setting the flag during open and changing it later.)

-zefram

[p5pRT]

From @cpansprout

On Sun Jul 10 20​:00​:12 2016, hkoba@​cpan.org wrote​:

I found following script causes SEGV, in perl5.22 ~ blead 59a08c7

perl -e 'open my $fh, ">", \ (my $buf = ""); my $sub = eval q|sub
{die}|; $sub->()'

Thank you.

I have fixed the crash in commit a2637ca.

--

Father Chrysostomos

[p5pRT]

From @cpansprout

On Mon Jul 11 15​:35​:54 2016, zefram@​fysh.org wrote​:

Father Chrysostomos via RT wrote​:

I don't know the solution to that particular problem.

Whether a particular I/O handle should be warned for, upon implicit
close, seems like a feature of the handle, which should be stored as
a flag in the handle structure. Obviously the flag state has to be
initialised upon opening, and by default the flag could reasonably be
set according to the warning flag prevailing *at the point of the open*.
(It should also be possible to control the flag explicitly per handle.
This could be done as a self-popping I/O layer, which would allow both
setting the flag during open and changing it later.)

I have opened ticket #128597 for that.

--

Father Chrysostomos

[p5pRT]

@cpansprout - Status changed from 'open' to 'pending release'

[p5pRT]

From @hkoba

Hi!

Thank you all for discussion and quickly solving this issue.

Would this patch be merged to maintenance release of 5.22 and 5.24 too?
If so, do you know something about when they are?

(If it takes some months, I need to file this issue&patch to
Fedora's bugzilla too. They use 5.22 for their latest F24.)

Thank you.
--
hkoba

On 2016-7月-11 月 17​:16​:18, sprout wrote​:

On Sun Jul 10 20​:00​:12 2016, hkoba@​cpan.org wrote​:

I found following script causes SEGV, in perl5.22 ~ blead 59a08c7

perl -e 'open my $fh, ">", \ (my $buf = ""); my $sub = eval q|sub
{die}|; $sub->()'

Thank you.

I have fixed the crash in commit a2637ca.

[p5pRT]

From @cpansprout

On Tue Jul 12 21​:09​:07 2016, hkoba@​cpan.org wrote​:

Hi!

Thank you all for discussion and quickly solving this issue.

Would this patch be merged to maintenance release of 5.22 and 5.24 too?

I have proposed it. It requires the votes of two other committers.

If so, do you know something about when they are?

It looks as though it will be soon​: http​://perl5.git.perl.org/perl.git/commit/e94880707b4983968cb376474e3570790e71c109

(If it takes some months, I need to file this issue&patch to
Fedora's bugzilla too. They use 5.22 for their latest F24.)

Thank you.
--
hkoba

On 2016-7月-11 月 17​:16​:18, sprout wrote​:

On Sun Jul 10 20​:00​:12 2016, hkoba@​cpan.org wrote​:

I found following script causes SEGV, in perl5.22 ~ blead 59a08c7

perl -e 'open my $fh, ">", \ (my $buf = ""); my $sub = eval q|sub
{die}|; $sub->()'

Thank you.

I have fixed the crash in commit a2637ca.

--

Father Chrysostomos

[p5pRT]

From @hkoba

Thank you for your proposal for merging
and also thank you for telling me about the release schedule! :+1​: :bow​:

I love to hear such efforts made by you perl5-porters
to maintain perl5 as a robust workhorse.

Thank you!
--
hkoba

On 2016-7月-15 金 13​:11​:22, sprout wrote​:

On Tue Jul 12 21​:09​:07 2016, hkoba@​cpan.org wrote​:

Hi!

Thank you all for discussion and quickly solving this issue.

Would this patch be merged to maintenance release of 5.22 and 5.24
too?

I have proposed it. It requires the votes of two other committers.

If so, do you know something about when they are?

It looks as though it will be soon​:
http​://perl5.git.perl.org/perl.git/commit/e94880707b4983968cb376474e3570790e71c109

(If it takes some months, I need to file this issue&patch to
Fedora's bugzilla too. They use 5.22 for their latest F24.)

Thank you.
--
hkoba

On 2016-7月-11 月 17​:16​:18, sprout wrote​:

On Sun Jul 10 20​:00​:12 2016, hkoba@​cpan.org wrote​:

I found following script causes SEGV, in perl5.22 ~ blead 59a08c7

perl -e 'open my $fh, ">", \ (my $buf = ""); my $sub = eval q|sub
{die}|; $sub->()'

Thank you.

I have fixed the crash in commit a2637ca.

[p5pRT]

From @xsawyerx

On Sat Jul 16 23​:01​:11 2016, hkoba@​cpan.org wrote​:

Thank you for your proposal for merging
and also thank you for telling me about the release schedule! :+1​:
:bow​:

I love to hear such efforts made by you perl5-porters
to maintain perl5 as a robust workhorse.

Thank you!

Thank you for your valuable contribution in improving perl! :)