Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

panic: magic_killbackrefs during global destruction. #7174

Closed
p5pRT opened this issue Mar 14, 2004 · 8 comments
Closed

panic: magic_killbackrefs during global destruction. #7174

p5pRT opened this issue Mar 14, 2004 · 8 comments

Comments

@p5pRT
Copy link

@p5pRT p5pRT commented Mar 14, 2004

Migrated from rt.perl.org#27630 (status was 'resolved')

Searchable as RT27630$

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Mar 14, 2004

From @jlokier

Created by @jlokier

The following test case causes Perl to write "panic​:
magic_killbackrefs during global destruction." and return a non-zero
exit status. It is clearly not an operator error​: slight changes to
the program such as adding more code make the error go away.

The bug is sensitive to the number of variables in the environment,
and the content of the LANG environment variable (a UTF-8 locale is
required to trigger the bug).

As this bug is so sensitive to the exact number of entries in the
environment, the locale, and exact numbers of various kinds of
statements, it's possible this won't trigger with current versions of
Perl, or with another OS environment or another architecture.

If the bug doesn't show with the current development version, the only
way to be sure it is actually fixed now is if you recognise it
already, or to run the test case with the Perl version that does
trigger it, perhaps requiring the same OS environment, and figure out
the cause from there. The version of Perl I am using is Red Hat 9's
perl-5.8.0-88.3, in a fairly standard Red Hat 9 installation with all
updates currently applied, i686 architecture. The detailed config is
below as usual.

Unfortunately I was unable to reduce it smaller than 6 small files.
This shell script will create them. Then do "sh environ" to run Perl
appropriately to trigger the bug​:

--- START OF SHELL SCRIPT WHICH UNPACKS FILES ---

cat <<'END' >environ
#!/bin/sh
env -i LANG=en_GB.UTF-8 \
ENV2= ENV3= ENV4= ENV5= ENV6= ENV7= ENV8= ENV9= ENV10= ENV11= ENV12= \
ENV13= ENV14= ENV15= ENV16= ENV17= ENV18= ENV19= ENV20= ENV21= ENV22= \
ENV23= ENV24= ENV25= ENV26= ENV27= ENV28= ENV29= ENV30= ENV31= ENV32= \
ENV33= ENV34= ENV35= ENV36= ENV37= ENV38= ENV39= \
perl bug.pl
END

cat <<'END' >bug.pl
# This triggers "panic​: magic_killbackrefs during global destruction".
BEGIN {
  require 'bug1.pl';
  for my $name (grep { 1 } keys %{'CORE​::'}) {
  *{$name} = ${'foo​::'}{$name};
  # Writing like this doesn't cause the panic​:
  #${__PACKAGE__."​::"}{$name} = ${'foo​::'}{$name};
  }
}
END

cat <<'END' >bug1.pl
use bug2;
use bug3;
use threads;
1;
END
cat <<'END' >bug2.pm
sub bug0 {}
eval "";
1;
END

cat <<'END' >bug3.pm
package bug3;
use bug4;
$bug4 = bug4 (0,1);
eval "";
1;
END

cat <<'END' >bug4.pm
package bug4;
@​EXPORT = qw(bug4);
@​ISA = 'Exporter';
require Exporter;
use Scalar​::Util 'weaken';
sub bug4($;$) {
  my $pool = [undef, undef, undef];
  weaken ($pool->[0] = $pool);
  weaken ($pool->[0] = $pool);
  return $pool;
}
1;
END

--- END OF SHELL SCRIPT WHICH UNPACKS FILES ---

Perl Info

Flags:
    category=core
    severity=high

Site configuration information for perl v5.8.0:

Configured by bhcompile'
cf_email='bhcompile at Wed Aug 13 11:45:59 EDT 2003.

Summary of my rderl (revision 5.0 version 8 subversion 0) configuration:
  Platform:
    osname=linux, osvers=2.4.21-1.1931.2.382.entsmp, archname=i386-linux-thread-multi
    uname='linux str'
    config_args='-des -Doptimize=-O2 -g -pipe -march=i386 -mcpu=i686 -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dinstallprefix=/usr -Dprefix=/usr -Darchname=i386-linux -Dvendorprefix=/usr -Dsiteprefix=/usr -Dotherlibdirs=/usr/lib/perl5/5.8.0 -Duseshrplib -Dusethreads -Duseithreads -Duselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef'
 useithreads=define usemultiplicity=
    useperlio= d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=un uselongdouble=
    usemymalloc=, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
    optimize='',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING -fno-strict-aliasing -I/usr/local/include -I/usr/include/gdbm'
    ccversion='', gccversion='3.2.2 20030222 (Red Hat Linux 3.2.2-5)', gccosandvers=''
gccversion='3.2.2 200302'
    intsize=r, longsize=r, ptrsize=5, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long'
k', ivsize=4'
ivtype='l, nvtype='double'
o_nonbl', nvsize=, Off_t='', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='gcc'
l', ldflags =' -L/u'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -lgdbm -ldb -ldl -lm -lpthread -lc -lcrypt -lutil
    perllibs=
    libc=/lib/libc-2.3.2.so, so=so, useshrplib=true, libperl=libper
    gnulibc_version='2.3.2'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so', d_dlsymun=undef, ccdlflags='-rdynamic -Wl,-rpath,/usr/lib/perl5/5.8.0/i386-linux-thread-multi/CORE'
    cccdlflags='-fPIC'
ccdlflags='-rdynamic -Wl,-rpath,/usr/lib/perl5', lddlflags='s Unicode/Normalize XS/A'

Locally applied patches:
    MAINT18379


@INC for perl v5.8.0:
    /usr/lib/perl5/5.8.0/i386-linux-thread-multi
    /usr/lib/perl5/5.8.0
    /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.0
    /usr/lib/perl5/site_perl
    /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.0
    /usr/lib/perl5/vendor_perl
    /usr/lib/perl5/5.8.0/i386-linux-thread-multi
    /usr/lib/perl5/5.8.0
    .


Environment for perl v5.8.0:
    HOME=/home/jamie
    LANG=en_GB.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/home/jamie/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash
    dlflags='-share (unset)

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Mar 14, 2004

From @iabyn

On Sun, Mar 14, 2004 at 07​:50​:20AM -0000, Jamie Lokier wrote​:

The following test case causes Perl to write "panic​:
magic_killbackrefs during global destruction." and return a non-zero
exit status. It is clearly not an operator error​: slight changes to
the program such as adding more code make the error go away.

Hmm, I can't reproduce it here with any of the 5.8.x perls nor any of the
5.9.x snaphots I have lying around. Anyone else have any luck?
(I tried on redhat 7.2).

I know that some bugs have been fixed recently relating to weak refs
being returned by threads, but I don't know whether that covers your
specific problem.

Dave.

--
Standards (n). Battle insignia or tribal totems.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Mar 14, 2004

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Mar 15, 2004

From @jlokier

Dave Mitchell via RT wrote​:

On Sun, Mar 14, 2004 at 07​:50​:20AM -0000, Jamie Lokier wrote​:

The following test case causes Perl to write "panic​:
magic_killbackrefs during global destruction." and return a non-zero
exit status. It is clearly not an operator error​: slight changes to
the program such as adding more code make the error go away.

To make the test case, I started with a large program which triggered
it, and then pruned it line by line, function by function. Why I
found is that it's highly sensitive to the number of "somethings" in a
program.

That is, I'd delete one statement, and it would go away. I'd
introduce one statement elsewhere, and it'd come back. Yet, sometimes
I could delete several large functions and it would stay.

To reduce the program, I used a tiny script which appended "$x=1;\n"
to one of the files and ran Perl on them, repeatedly appending until
the bug would be triggered.

Then I deleted whole functions from the original program and ran that
script, which would append enough "$x=1;\n"'s to restore the bug.
Most of the time when I deleted a function, that would restore the bug
within 1-200 extra lines being added. Yet, sometimes I'd delete a
function or even a single line, and after hundreds of those extra
lines I still wouldn't see the bug. So I'd back out that deletion,
and try to narrow it down.

Eventually I got close to what you have in the bug report, except that
a couple of files had hundreds of "$x=1;\n" lines -- just the right
number to tickle the bug.

Then I decided to trim those files some more, and see if I could
replace 298 lines of "$x=1;\n" with "eval '$x=1;'x298" or some similar
number. Much to my surprise, the empty evals were enough to replace
all those lines.

When I copied the files from my test machine to the machine where I
send mails, I tested them there and the bug didn't appear :( Both
machines have the same binary Perl and libc packages. That's when I
started looking at the environment variables, and found the number of
them to be critical -- the length and values don't seem to matter,
except that LANG=en_GB.UTF-8 is important​: LANG=en_GB doesn't trigger it!

So the bug appears to be highly sensitive to numbers of something.
Probably it's dependent on one or more memory allocatord, and won't
show up _with that exact test case_ on a different version of libc, or
maybe even a different memory layout of Perl. Probably there's a
slightly different test case which tickles it on a different system.

If possible, I recommend installing Red Hat 9 i386 with all current
updates, including perl-5.8.0-88.3 and glibc-2.3.2-27.9.7, just to
recreate the environment where this test case fails so you can debug
it. It's a lot of work I know :/

Maybe you will have luck running it under a memory debugger -- perhaps
it's a user-after-free or double-free bug?

Hmm, I can't reproduce it here with any of the 5.8.x perls nor any of the
5.9.x snaphots I have lying around. Anyone else have any luck?
(I tried on redhat 7.2).

I know that some bugs have been fixed recently relating to
weak refs being returned by threads, but I don't know whether that
covers your specific problem.

If it's thread related, note that Red Hat 9 and Red Hat 7.2 have
_very_ different threading implementations. RH9 uses the NPTL thread
library; RH7.2 uses LinuxThreads. The patterns of memory allocation
are likely to be very different as a result.

-- Jamie

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Mar 15, 2004

From nick.ing-simmons@elixent.com

Jamie Lokier <perl5-porters@​perl.org> writes​:

# New Ticket Created by Jamie Lokier
# Please include the string​: [perl #27630]
# in the subject line of all future correspondence about this issue.
# <URL​: http​://rt.perl.org​:80/rt3/Ticket/Display.html?id=27630 >

This is a bug report for perl from jamie@​shareable.org,
generated with the help of perlbug 1.34 running under perl v5.8.0.

-----------------------------------------------------------------
[Please enter your report here]

The following test case causes Perl to write "panic​:
magic_killbackrefs during global destruction." and return a non-zero
exit status. It is clearly not an operator error​: slight changes to
the program such as adding more code make the error go away.

If the bug doesn't show with the current development version, the only
way to be sure it is actually fixed now is if you recognise it
already, or to run the test case with the Perl version that does
trigger it, perhaps requiring the same OS environment, and figure out
the cause from there. The version of Perl I am using is Red Hat 9's
perl-5.8.0-88.3,

Which is known to have a number of Unicode oddities, but I can't
say I have seen it produce this before. But as the other oddities
suggest it has malloc-ed data overrun problems it is certainly
possible.

in a fairly standard Red Hat 9 installation with all
updates currently applied, i686 architecture. The detailed config is
below as usual.

Unfortunately I was unable to reduce it smaller than 6 small files.
This shell script will create them. Then do "sh environ" to run Perl
appropriately to trigger the bug​:

--- START OF SHELL SCRIPT WHICH UNPACKS FILES ---

cat <<'END' >environ
#!/bin/sh
env -i LANG=en_GB.UTF-8 \
ENV2= ENV3= ENV4= ENV5= ENV6= ENV7= ENV8= ENV9= ENV10= ENV11= ENV12= \
ENV13= ENV14= ENV15= ENV16= ENV17= ENV18= ENV19= ENV20= ENV21= ENV22= \
ENV23= ENV24= ENV25= ENV26= ENV27= ENV28= ENV29= ENV30= ENV31= ENV32= \
ENV33= ENV34= ENV35= ENV36= ENV37= ENV38= ENV39= \
perl bug.pl
END

cat <<'END' >bug.pl
# This triggers "panic​: magic_killbackrefs during global destruction".
BEGIN {
require 'bug1.pl';
for my $name (grep { 1 } keys %{'CORE​::'}) {
*{$name} = ${'foo​::'}{$name};
# Writing like this doesn't cause the panic​:
#${__PACKAGE__."​::"}{$name} = ${'foo​::'}{$name};
}
}
END

cat <<'END' >bug1.pl
use bug2;
use bug3;
use threads;
1;
END
cat <<'END' >bug2.pm
sub bug0 {}
eval "";
1;
END

cat <<'END' >bug3.pm
package bug3;
use bug4;
$bug4 = bug4 (0,1);
eval "";
1;
END

cat <<'END' >bug4.pm
package bug4;
@​EXPORT = qw(bug4);
@​ISA = 'Exporter';
require Exporter;
use Scalar​::Util 'weaken';
sub bug4($;$) {
my $pool = [undef, undef, undef];
weaken ($pool->[0] = $pool);
weaken ($pool->[0] = $pool);
return $pool;
}
1;
END

--- END OF SHELL SCRIPT WHICH UNPACKS FILES ---

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags​:
category=core
severity=high
---
Site configuration information for perl v5.8.0​:

Configured by bhcompile'
cf_email='bhcompile at Wed Aug 13 11​:45​:59 EDT 2003.

Summary of my rderl (revision 5.0 version 8 subversion 0) configuration​:
Platform​:
osname=linux, osvers=2.4.21-1.1931.2.382.entsmp, archname=i386-linux-thread-multi
uname='linux str'
config_args='-des -Doptimize=-O2 -g -pipe -march=i386 -mcpu=i686 -Dmyhostname=localhost -Dperladmin=root@​localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dinstallprefix=/usr -Dprefix=/usr -Darchname=i386-linux -Dvendorprefix=/usr -Dsiteprefix=/usr -Dotherlibdirs=/usr/lib/perl5/5.8.0 -Duseshrplib -Dusethreads -Duseithreads -Duselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr'
hint=recommended, useposix=true, d_sigaction=define
usethreads=define use5005threads=undef'
useithreads=define usemultiplicity=
useperlio= d_sfio=undef uselargefiles=define usesocks=undef
use64bitint=undef use64bitall=un uselongdouble=
usemymalloc=, bincompat5005=undef
Compiler​:
cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
optimize='',
cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING -fno-strict-aliasing -I/usr/local/include -I/usr/include/gdbm'
ccversion='', gccversion='3.2.2 20030222 (Red Hat Linux 3.2.2-5)', gccosandvers=''
gccversion='3.2.2 200302'
intsize=r, longsize=r, ptrsize=5, doublesize=8, byteorder=1234
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
ivtype='long'
k', ivsize=4'
ivtype='l, nvtype='double'
o_nonbl', nvsize=, Off_t='', lseeksize=8
alignbytes=4, prototype=define
Linker and Libraries​:
ld='gcc'
l', ldflags =' -L/u'
libpth=/usr/local/lib /lib /usr/lib
libs=-lnsl -lgdbm -ldb -ldl -lm -lpthread -lc -lcrypt -lutil
perllibs=
libc=/lib/libc-2.3.2.so, so=so, useshrplib=true, libperl=libper
gnulibc_version='2.3.2'
Dynamic Linking​:
dlsrc=dl_dlopen.xs, dlext=so', d_dlsymun=undef, ccdlflags='-rdynamic -Wl,-rpath,/usr/lib/perl5/5.8.0/i386-linux-thread-multi/CORE'
cccdlflags='-fPIC'
ccdlflags='-rdynamic -Wl,-rpath,/usr/lib/perl5', lddlflags='s Unicode/Normalize XS/A'

Locally applied patches​:
MAINT18379

---
@​INC for perl v5.8.0​:
/usr/lib/perl5/5.8.0/i386-linux-thread-multi
/usr/lib/perl5/5.8.0
/usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi
/usr/lib/perl5/site_perl/5.8.0
/usr/lib/perl5/site_perl
/usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi
/usr/lib/perl5/vendor_perl/5.8.0
/usr/lib/perl5/vendor_perl
/usr/lib/perl5/5.8.0/i386-linux-thread-multi
/usr/lib/perl5/5.8.0
.

---
Environment for perl v5.8.0​:
HOME=/home/jamie
LANG=en_GB.UTF-8
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/usr/local/bin​:/bin​:/usr/bin​:/usr/X11R6/bin​:/home/jamie/bin
PERL_BADLANG (unset)
SHELL=/bin/bash
dlflags='-share (unset)

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Mar 15, 2004

From nick.ing-simmons@elixent.com

Dave Mitchell <davem@​fdisolutions.com> writes​:

On Sun, Mar 14, 2004 at 07​:50​:20AM -0000, Jamie Lokier wrote​:

The following test case causes Perl to write "panic​:
magic_killbackrefs during global destruction." and return a non-zero
exit status. It is clearly not an operator error​: slight changes to
the program such as adding more code make the error go away.

Hmm, I can't reproduce it here with any of the 5.8.x perls nor any of the
5.9.x snaphots I have lying around. Anyone else have any luck?
(I tried on redhat 7.2).

If you perforce access then you could try getting the

p4 sync //depot/maint-5.8/...@​18379

Which is what I think RedHat's MAINT18379 "locally applied patch"
is supposed to mean. I haven't tried diffing that against their
source RPM though.

I know that some bugs have been fixed recently relating to weak refs
being returned by threads, but I don't know whether that covers your
specific problem.

Dave.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jul 3, 2006

From @smpeters

[nick.ing-simmons@​elixent.com - Mon Mar 15 04​:28​:04 2004]​:

Dave Mitchell <davem@​fdisolutions.com> writes​:

On Sun, Mar 14, 2004 at 07​:50​:20AM -0000, Jamie Lokier wrote​:

The following test case causes Perl to write "panic​:
magic_killbackrefs during global destruction." and return a non-zero
exit status. It is clearly not an operator error​: slight changes to
the program such as adding more code make the error go away.

Hmm, I can't reproduce it here with any of the 5.8.x perls nor any of the
5.9.x snaphots I have lying around. Anyone else have any luck?
(I tried on redhat 7.2).

If you perforce access then you could try getting the

p4 sync //depot/maint-5.8/...@​18379

Which is what I think RedHat's MAINT18379 "locally applied patch"
is supposed to mean. I haven't tried diffing that against their
source RPM though.

I know that some bugs have been fixed recently relating to weak refs
being returned by threads, but I don't know whether that covers your
specific problem.

Dave.

Similarly to when this ticket was opened, I've been unable to reproduce
it with any recent bleadperls. My guess is that this was partially
caused by one of the broken Perl's on RedHat from that time period.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jul 3, 2006

@smpeters - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant