possible bug in PerlIO::encoding

[p5pRT]

Migrated from rt.perl.org#98060 (status was 'open')

Searchable as RT98060$

[p5pRT]

From perlbug@plan9.de

Created by perlbug@plan9.de

While browsing the source for encoding.xs (5.12.4), I saw this comment​:

  /*
  * we now "use Encode ()" here instead of
  * PerlIO/encoding.pm. This avoids SEGV when "​:encoding()"
  * is invoked without prior "use Encode". -- dankogai
  */

the code then uses PUSHSTACKi/POPSTACK, with this extra comment​:

  /* Encode needs a lot of stack - it is likely to move ... */

And encoding.pm contains​:

# Equivalent of this is done in encoding.xs - do not uncomment.
#
# use Encode ();

So apparently this code is an attempt to avoid segfaults due to stack
changes in code that doesn't expect it (and in fact, the perlio layers
loading code does NOT switch to a new stack).

Obviously, doing it from encoding.xs only reduces the chances of a stack
change, it cannot elimiate it, i.e. if Encode uses a lot of stack, then
chances are high that this error will bite, while with this "workaround",
chances are lower only, as the BOOT function of encoding.xs is called from
normal perl code which might cause stack changes already.

If stack changes are indeed a problem, then the problem cannot be in
encoding.xs, but is likely in perlio.c, which calls load_module without
PUSHSTACK/POPSTACK, or in some other layer.

I didn't have time to investigate this in detail, so forgive me when my
analysis is wrong for any reason, but I thought it would be best to report
this, as it smells like a hack around an actual problem.

Perl Info

Flags:
    category=library
    severity=low
    module=PerlIO::encoding

Site configuration information for perl 5.12.4:

Configured by Marc Lehmann at Sun Aug 14 14:59:28 CEST 2011.

Summary of my perl5 (revision 5 version 12 subversion 4) configuration:
   
  Platform:
    osname=linux, osvers=2.6.39-2-amd64, archname=x86_64-linux
    uname='linux cerebro 2.6.39-2-amd64 #1 smp tue jul 5 02:51:22 utc 2011 x86_64 gnulinux '
    config_args='-Duselargefiles -Duse64bitint -Dusemymalloc=n -Dstatic_ext=Fcntl -Dcc=gcc -Dccflags=-DPERL_DISABLE_PMC -DPERL_ARENA_SIZE=16376 -USITEARCH_EXP -USITELIB_EXP -UARCHLIB_EXP -D_GNU_SOURCE  -I/opt/include -ggdb -gdwarf-2 -g3 -Doptimize=-O6 -fno-strict-aliasing -Dcccdlflags=-fPIC -Dldflags=-L/opt/perl/lib -L/opt/lib -Dlibs=-ldl -lm -lcrypt -lgdbm -Dprefix=/opt/perl -Dprivlib=/opt/perl/lib/perl5 -Darchlib=/opt/perl/lib/perl5 -Uusevendorprefix -Dsiteprefix=/opt/perl -Dsitelib=/opt/perl/lib/perl5 -Dsitearch=/opt/perl/lib/perl5 -Dsitebin=/opt/perl/bin -Dman1dir=/opt/perl/man/man1 -Dman3dir=/opt/perl/man/man3 -Dsiteman1dir=/opt/perl/man/man1 -Dsiteman3dir=/opt/perl/man/man3 -Dman1ext=1 -Dman3ext=3 -Dpager=/usr/bin/less -Uafs -Uusesfio -Uusenm -Uuseshrplib -Ud_dosuid -Dusethreads=undef -Duse5005threads=undef -Duseithreads=undef -Dusemultiplicity=undef -Demail=perl-binary@plan9.de -Dcf_email=perl-binary@plan9.de -Dcf_by=Marc Lehmann -Dlocincpth=/opt/perl/include /opt/include -Dmyhostname=localhost -Dmultiarch=undef -Dbin=/opt/perl/bin -Dxxxusedevel -DxxxDEBUGGING -Dxxxuse_debugging_perl -Dxxxuse_debugmalloc -dEs'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=undef
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-DPERL_DISABLE_PMC -DPERL_ARENA_SIZE=16376 -USITEARCH_EXP -USITELIB_EXP -UARCHLIB_EXP -D_GNU_SOURCE -I/opt/include -ggdb -gdwarf-2 -g3 -fno-strict-aliasing -pipe -I/opt/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O6 -fno-strict-aliasing',
    cppflags='-DPERL_DISABLE_PMC -DPERL_ARENA_SIZE=16376 -USITEARCH_EXP -USITELIB_EXP -UARCHLIB_EXP -D_GNU_SOURCE -I/opt/include -ggdb -gdwarf-2 -g3 -fno-strict-aliasing -pipe -I/opt/include'
    ccversion='', gccversion='4.4.5', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags ='-L/opt/perl/lib -L/opt/lib -L/usr/local/lib'
    libpth=/usr/local/lib /lib/../lib /usr/lib/../lib /lib /usr/lib /usr/lib/x86_64-linux-gnu /lib64 /usr/lib64
    libs=-ldl -lm -lcrypt -lgdbm
    perllibs=-ldl -lm -lcrypt
    libc=/lib/libc-2.11.2.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.11.2'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O6 -fno-strict-aliasing -L/opt/perl/lib -L/opt/lib -L/usr/local/lib'

Locally applied patches:
    


@INC for perl 5.12.4:
    /root/src/sex
    /opt/perl/lib/perl5
    /opt/perl/lib/perl5
    .


Environment for perl 5.12.4:
    HOME=/root
    LANG (unset)
    LANGUAGE (unset)
    LC_CTYPE=en_US.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/root/s2:/root/s:/opt/bin:/opt/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11/bin:/usr/games:/usr/local/bin:/usr/local/sbin:/root/pserv:.
    PERL5LIB=/root/src/sex
    PERL5_CPANPLUS_CONFIG=/root/.cpanplus/config
    PERLDB_OPTS=ornaments=0
    PERL_ANYEVENT_DBI_TESTS=1
    PERL_ANYEVENT_LOOP_TESTS=1
    PERL_ANYEVENT_NET_TESTS=1
    PERL_BADLANG (unset)
    PERL_UNICODE=E
    SHELL=/bin/bash

[p5pRT]

From zefram@fysh.org

perlbug@​plan9.de wrote​:

If stack changes are indeed a problem, then the problem cannot be in
encoding.xs, but is likely in perlio.c, which calls load_module without
PUSHSTACK/POPSTACK, or in some other layer.

There are many places that are liable to hold lurking stack-reallocation
bugs. Stacks don't move very often, so a tacit assumption that the
stack won't move will rarely bite. We could do with a debugging option
that forcibly moves the stack at every opportunity (every time EXTEND
is called). That would go nicely with valgrind.

-zefram

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'