List::Util tests give ‘Attempt to free unreferenced scalar’ warnings

[p5pRT]

Migrated from rt.perl.org#78070 (status was 'resolved')

Searchable as RT78070$

[p5pRT]

From @cpansprout

I�ve been seeing these warnings lately when running perl�s tests​:

cpan/List-Util/t/first.........................................Attempt to free unreferenced scalar​: SV 0x81f460, Perl interpreter​: 0x800000 during global destruction.
ok

cpan/List-Util/t/reduce........................................Attempt to free unreferenced scalar​: SV 0x84fd50, Perl interpreter​: 0x800000 during global destruction.
ok

Is this a new bug in perl (maybe from something I committed) or a List​::Util bug?

---

Flags​:
  category=core
  severity=medium

---

Site configuration information for perl 5.13.5​:

Configured by sprout at Sun Sep 19 17​:34​:01 PDT 2010.

Summary of my perl5 (revision 5 version 13 subversion 5 patch v5.13.5-8-g8e58c70) configuration​:
  Snapshot of​: 8e58c70
  Platform​:
  osname=darwin, osvers=10.4.0, archname=darwin-thread-multi-2level
  uname='darwin pint.local 10.4.0 darwin kernel version 10.4.0​: fri apr 23 18​:28​:53 pdt 2010; root​:xnu-1504.7.4~1release_i386 i386 '
  config_args='-de -Dusedevel -DDEBUGGING -Duseithreads'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=define, usemultiplicity=define
  useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
  use64bitint=undef, use64bitall=undef, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-fno-common -DPERL_DARWIN -no-cpp-precomp -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include',
  optimize='-O3 -g',
  cppflags='-no-cpp-precomp -fno-common -DPERL_DARWIN -no-cpp-precomp -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
  ccversion='', gccversion='4.2.1 (Apple Inc. build 5664)', gccosandvers=''
  intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
  ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc', ldflags =' -fstack-protector -L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib
  libs=-ldbm -ldl -lm -lutil -lc
  perllibs=-ldl -lm -lutil -lc
  libc=/usr/lib/libc.dylib, so=dylib, useshrplib=false, libperl=libperl.a
  gnulibc_version=''
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
  cccdlflags=' ', lddlflags=' -bundle -undefined dynamic_lookup -L/usr/local/lib -fstack-protector'

Locally applied patches​:
 

---

@​INC for perl 5.13.5​:
  /usr/local/lib/perl5/site_perl/5.13.5/darwin-thread-multi-2level
  /usr/local/lib/perl5/site_perl/5.13.5
  /usr/local/lib/perl5/5.13.5/darwin-thread-multi-2level
  /usr/local/lib/perl5/5.13.5
  /usr/local/lib/perl5/site_perl
  .

---

Environment for perl 5.13.5​:
  DYLD_LIBRARY_PATH (unset)
  HOME=/Users/sprout
  LANG=en_US.UTF-8
  LANGUAGE (unset)
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
  PATH=/usr/bin​:/bin​:/usr/sbin​:/sbin​:/usr/local/bin​:/usr/X11/bin​:/usr/local/bin
  PERL_BADLANG (unset)
  SHELL=/bin/bash

[p5pRT]

From @iabyn

On Sun, Sep 26, 2010 at 01​:50​:31PM -0700, Father Chrysostomos wrote​:

Iâ��ve been seeing these warnings lately when running perlâ��s tests​:

cpan/List-Util/t/first.........................................Attempt to free unreferenced scalar​: SV 0x81f460, Perl interpreter​: 0x800000 during global destruction.
ok

I'm not seeing them, nor are the smokes. Can you bisect?

--
Wesley Crusher gets beaten up by his classmates for being a smarmy git,
and consequently has a go at making some friends of his own age for a
change.
  -- Things That Never Happen in "Star Trek" #18

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @cpansprout

On Mon Sep 27 04​:30​:17 2010, davem wrote​:

On Sun, Sep 26, 2010 at 01​:50​:31PM -0700, Father Chrysostomos wrote​:

Iâ��ve been seeing these warnings lately when running perlâ��s tests​:

cpan/List-
Util/t/first.........................................Attempt to free
unreferenced scalar​: SV 0x81f460, Perl interpreter​: 0x800000 during
global destruction.
ok

I'm not seeing them, nor are the smokes. Can you bisect?

__BEGIN__
From ca556bc Mon Sep 17 00​:00​:00 2001
From​: David Mitchell <davem@​iabyn.com>
Date​: Sun, 19 Sep 2010 12​:33​:04 +0100
Subject​: [PATCH] [perl #40389] perl_destruct() leaks PL_defstash

With PERL_DESTRUCT_LEVEL >= 1, PL_defstash is explicitly freed,
but doesn't actually get freed at that point due to a reference loop
between %​:: and *​::. Break that loop to ensure that PL_defstash gets freed
at that point. Actually, its not as serious as it sounds, as it would get
freed a bit later anyway by sv_clean_all(), but this new way has these
benefits​:

* it gets freed where you expect it to be
* it gets freed cleanly, rather than by the more brutal sv_clean_all()
(which can leave dangling pointers to freed SVs)
* since its freed while *not* under the influence of
PL_in_clean_all = TRUE, it's more likely to flag up bugs related to
double-freeing etc. Indeed, the two previous commits to this are a
result of that.

---

perl.c | 2 ++
1 files changed, 2 insertions(+), 0 deletions(-)

Inline Patch

diff --git a/perl.c b/perl.c
index a04cfd6..860cdfc 100644
--- a/perl.c
+++ b/perl.c
@@ -1042,6 +1042,8 @@ perl_destruct(pTHXx)
    /* Prepare to destruct main symbol table.  */

    hv = PL_defstash;
+    /* break ref loop  *:: <=> %:: */
+    (void)hv_delete(hv, "main::", 6, G_DISCARD);
    PL_defstash = 0;
    SvREFCNT_dec(hv);
    SvREFCNT_dec(PL_curstname);

__END__

This seems to be a case of the �it's more likely to flag up bugs related
to double-freeing etc.� above.

It only gives this message under make test, not under harness, because
of PERL_DESTRUCT_LEVEL, which must be 2.

To reproduce​:

make
cd cpan/List-Util/
PERL_DESTRUCT_LEVEL=2 ../../perl -I../.. -MTestInit=U2T t/first.t

[p5pRT]

From @iabyn

On Sun, Oct 03, 2010 at 01​:06​:30PM -0700, Father Chrysostomos wrote​:

cpan/List-Util/t/first.........................................Attempt to free unreferenced scalar​: SV 0x81f460, Perl interpreter​: 0x800000 during global destruction.
ok

[snip]

Can you bisect?

From ca556bc Mon Sep 17 00​:00​:00 2001
From​: David Mitchell <davem@​iabyn.com>
Date​: Sun, 19 Sep 2010 12​:33​:04 +0100
Subject​: [PATCH] [perl #40389] perl_destruct() leaks PL_defstash

[snip]

This seems to be a case of the �it's more likely to flag up bugs related
to double-freeing etc.� above.

Indeed, that turns out to be the case. Fixed by the commit below.
Thanks for the investigation.

commit f837477
Author​: David Mitchell <davem@​iabyn.com>
AuthorDate​: Tue Oct 19 23​:13​:07 2010 +0100
Commit​: David Mitchell <davem@​iabyn.com>
CommitDate​: Tue Oct 19 23​:29​:25 2010 +0100

  Recursive MULTICALL prematurely freed CV
 
  See [perl #78070].
 
  Basically, POPSUB/LEAVESUB had a mechanism to decrement the reference
  count of the CV only at CvDEPTH==1; POP_MULTICALL was decrementing it at
  all depths.

M cop.h
M ext/XS-APItest/t/multicall.t

--
I don't want to achieve immortality through my work...
I want to achieve it through not dying.
  -- Woody Allen

[p5pRT]

@iabyn - Status changed from 'open' to 'resolved'

[p5pRT]

From @cpansprout

On Tue Oct 19 15​:32​:48 2010, davem wrote​:

Basically\, POPSUB/LEAVESUB had a mechanism to decrement the

reference
count of the CV only at CvDEPTH==1; POP_MULTICALL was decrementing
it at
all depths.

This sounds strikingly similar to #71076. Letâ��s see​:

use List​::Util 'first';
for(1,2){
sub foo {
  if($comparison) {
  return 1
  }
  else {
  local $comparison = 1;
  first \&foo, 1,2,3;
  }
}
foo()
}

$ pbpaste|perl # 5.10.1
Segmentation fault

Is this something worth working around in List​::Util? It would mean
incrementing the refcount before calling POP_MULTICALL, depending on the
perl version and the value of CvDEPTH.