false positive of stack extending check with empty arrays

[p5pRT]

Migrated from rt.perl.org#131732 (status was 'resolved')

Searchable as RT131732$

[p5pRT]

From lorenz@math.tu-berlin.de

Created by lorenz@math.tu-berlin.de

Using an implicit return with two empty arrays triggers panic of the stack extending check introduced in​:

commit 87058c3
Author​: David Mitchell <davem@​iabyn.com>
Date​: Tue Jun 13 09​:11​:13 2017 +0100
add PL_curstackinfo->si_stack_hwm

Reduced testcase​:

$ /store/borel/lorenz/prefixes/perlblead/bin/perl -e 'sub test { my @​x; (@​x,@​x); }; test();'
panic​: previous op failed to extend arg stack​: base=7ebb50, sp=7ebb58, hwm=7ebb50

Adding an explicit return or using the returned value in a print works fine​:
$ /store/borel/lorenz/prefixes/perlblead/bin/perl -e 'sub test { my @​x; return (@​x,@​x); }; test();'
$ /store/borel/lorenz/prefixes/perlblead/bin/perl -e 'sub test { my @​x; (@​x,@​x); }; print test();'

Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl 5.27.2:

Configured by lorenz at Mon Jul 10 16:21:11 CEST 2017.

Summary of my perl5 (revision 5 version 27 subversion 2) configuration:
  Commit id: 3072e7590c6345bcdd3e68ceb789160139beb412
  Platform:
    osname=linux
    osvers=4.1.38-50-default
    archname=x86_64-linux
    uname='linux borel 4.1.38-50-default #1 smp preempt sun feb 19 14:35:48 utc 2017 (6b4d8cb) x86_64 x86_64 x86_64 gnulinux '
    config_args='-des -Dprefix=/store/borel/lorenz/prefixes/perlblead -Duseshrplib -DDEBUGGING -Doptimize=-O1 -g -pipe -Dusedevel'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=undef
    usemultiplicity=undef
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
    optimize='-O1 -g -pipe'
    cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion=''
    gccversion='4.8.5'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib64/gcc/x86_64-suse-linux/4.8/include-fixed /usr/lib64/gcc/x86_64-suse-linux/4.8/../../../../x86_64-suse-linux/lib /usr/lib /lib/../lib64 /usr/lib/../lib64 /lib /lib64 /usr/lib64 /usr/local/lib64
    libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
    perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.22.so
    so=so
    useshrplib=true
    libperl=libperl.so
    gnulibc_version='2.22'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E -Wl,-rpath,/store/borel/lorenz/prefixes/perlblead/lib/5.27.2/x86_64-linux/CORE'
    cccdlflags='-fPIC'
    lddlflags='-shared -O1 -g -pipe -L/usr/local/lib -fstack-protector'



@INC for perl 5.27.2:
    /store/borel/lorenz/prefix/lib/perl5
    /store/borel/lorenz/prefix/lib/perl5
    /usr/site-local/lib/perl5/site_perl
    /store/borel/lorenz/prefixes/perlblead/lib/site_perl/5.27.2/x86_64-linux
    /store/borel/lorenz/prefixes/perlblead/lib/site_perl/5.27.2
    /store/borel/lorenz/prefixes/perlblead/lib/5.27.2/x86_64-linux
    /store/borel/lorenz/prefixes/perlblead/lib/5.27.2


Environment for perl 5.27.2:
    HOME=/homes/combi/lorenz
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH=/store/borel/lorenz/prefix/lib:/store/borel/lorenz/prefix/lib:/usr/lib64/mpi/gcc/openmpi/lib64
    LOGDIR (unset)
    PATH=/store/borel/lorenz/prefix/bin:/store/borel/lorenz/prefix/bin:/homes/combi/lorenz/.cabal/bin:/homes/combi/lorenz/.local/bin:/store/borel/lorenz/prefix/bin:/store/borel/lorenz/prefix/bin:/homes/combi/lorenz/.cabal/bin:/homes/combi/lorenz/.local/bin:/net/TeXLive/bin/x86_64-linux:/usr/lib64/mpi/gcc/openmpi/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games:/usr/lib/mit/bin:/usr/lib/mit/sbin:/usr/site-local/bin:/usr/site-local/share/bin
    PERL5LIB=/store/borel/lorenz/prefix/lib/perl5:/store/borel/lorenz/prefix/lib/perl5:/usr/site-local/lib/perl5/site_perl
    PERL_BADLANG (unset)
    PERL_LOCAL_LIB_ROOT=:/store/borel/lorenz/prefix:/store/borel/lorenz/prefix
    SHELL=/bin/bash

[p5pRT]

From @jkeenan

On Mon, 10 Jul 2017 14​:51​:16 GMT, lorenz@​math.tu-berlin.de wrote​:

This is a bug report for perl from lorenz@​math.tu-berlin.de,
generated with the help of perlbug 1.40 running under perl 5.27.2.

-----------------------------------------------------------------
[Please describe your issue here]

Using an implicit return with two empty arrays triggers panic of the
stack extending check introduced in​:

commit 87058c3
Author​: David Mitchell <davem@​iabyn.com>
Date​: Tue Jun 13 09​:11​:13 2017 +0100
add PL_curstackinfo->si_stack_hwm

Reduced testcase​:

$ /store/borel/lorenz/prefixes/perlblead/bin/perl -e 'sub test { my
@​x; (@​x,@​x); }; test();'
panic​: previous op failed to extend arg stack​: base=7ebb50, sp=7ebb58,
hwm=7ebb50

I am unable to reproduce this in perl 5 blead (commit 3072e75) on either Ubuntu Linux 16.04 LTS or FreeBSD-10.3-RELEASE-p1.

Adding an explicit return or using the returned value in a print works
fine​:
$ /store/borel/lorenz/prefixes/perlblead/bin/perl -e 'sub test { my
@​x; return (@​x,@​x); }; test();'
$ /store/borel/lorenz/prefixes/perlblead/bin/perl -e 'sub test { my
@​x; (@​x,@​x); }; print test();'

Example on FreeBSD-10.1​:

#####
[perl] $ ./perl -Ilib -e 'sub test { my @​x; return (@​x,@​x); }; test();'
[perl] $ ./perl -Ilib -e 'sub test { my @​x; (@​x,@​x); }; print test();'
[perl] $ ./perl -Ilib -e 'sub test { my @​x; (@​x,@​x); }; test();'
[perl] $
#####

No evidence of panic.

Thank you very much.
--
James E Keenan (jkeenan@​cpan.org)

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @jkeenan

On Mon, 10 Jul 2017 20​:41​:41 GMT, jkeenan wrote​:

On Mon, 10 Jul 2017 14​:51​:16 GMT, lorenz@​math.tu-berlin.de wrote​:

This is a bug report for perl from lorenz@​math.tu-berlin.de,
generated with the help of perlbug 1.40 running under perl 5.27.2.

-----------------------------------------------------------------
[Please describe your issue here]

Using an implicit return with two empty arrays triggers panic of the
stack extending check introduced in​:

commit 87058c3
Author​: David Mitchell <davem@​iabyn.com>
Date​: Tue Jun 13 09​:11​:13 2017 +0100
add PL_curstackinfo->si_stack_hwm

Reduced testcase​:

$ /store/borel/lorenz/prefixes/perlblead/bin/perl -e 'sub test { my
@​x; (@​x,@​x); }; test();'
panic​: previous op failed to extend arg stack​: base=7ebb50,
sp=7ebb58,
hwm=7ebb50

I am unable to reproduce this in perl 5 blead (commit
3072e75) on either Ubuntu Linux 16.04
LTS or FreeBSD-10.3-RELEASE-p1.

Adding an explicit return or using the returned value in a print
works
fine​:
$ /store/borel/lorenz/prefixes/perlblead/bin/perl -e 'sub test { my
@​x; return (@​x,@​x); }; test();'
$ /store/borel/lorenz/prefixes/perlblead/bin/perl -e 'sub test { my
@​x; (@​x,@​x); }; print test();'

Example on FreeBSD-10.1​:

#####
[perl] $ ./perl -Ilib -e 'sub test { my @​x; return (@​x,@​x); };
test();'
[perl] $ ./perl -Ilib -e 'sub test { my @​x; (@​x,@​x); }; print test();'
[perl] $ ./perl -Ilib -e 'sub test { my @​x; (@​x,@​x); }; test();'
[perl] $
#####

No evidence of panic.

Update​:

This *does* panic under a debugging build. (I previously reported results from my default configurations on both OSes.)

Example on Linux​:

#####
$ sh ./Configure -des -Dusedevel -DDEBUGGING -Doptimize="-O1 -g -pipe"

$ make test_prep

$./perl -Ilib -e 'sub test { my @​x; return (@​x,@​x); }; test();'

$./perl -Ilib -e 'sub test { my @​x; (@​x,@​x); }; print test();'

$./perl -Ilib -e 'sub test { my @​x; (@​x,@​x); }; test();'
panic​: previous op failed to extend arg stack​: base=d75b50, sp=d75b58, hwm=d75b50
#####

Similar results in debugging build on FreeBSD-10.3​:

#####
panic​: previous op failed to extend arg stack​: base=802006800, sp=802006808, hwm=802006800
#####

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

[p5pRT]

From @timbunce

Looks like DBI also fails some tests on 5.27.2 with -DDEBUGGING​:

http​://www.cpantesters.org/cpan/report/c5d1a148-69b3-11e7-b609-ee2501f1587f

On 10 July 2017 at 15​:51, via RT <perlbug-followup@​perl.org> wrote​:

# New Ticket Created by
# Please include the string​: [perl #131732]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=131732 >

This is a bug report for perl from lorenz@​math.tu-berlin.de,
generated with the help of perlbug 1.40 running under perl 5.27.2.

-----------------------------------------------------------------
[Please describe your issue here]

Using an implicit return with two empty arrays triggers panic of the stack
extending check introduced in​:

commit 87058c3
Author​: David Mitchell <davem@​iabyn.com>
Date​: Tue Jun 13 09​:11​:13 2017 +0100
add PL_curstackinfo->si_stack_hwm

Reduced testcase​:

$ /store/borel/lorenz/prefixes/perlblead/bin/perl -e 'sub test { my @​x;
(@​x,@​x); }; test();'
panic​: previous op failed to extend arg stack​: base=7ebb50, sp=7ebb58,
hwm=7ebb50

Adding an explicit return or using the returned value in a print works
fine​:
$ /store/borel/lorenz/prefixes/perlblead/bin/perl -e 'sub test { my @​x;
return (@​x,@​x); }; test();'
$ /store/borel/lorenz/prefixes/perlblead/bin/perl -e 'sub test { my @​x;
(@​x,@​x); }; print test();'

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags​:
category=core
severity=medium
---
Site configuration information for perl 5.27.2​:

Configured by lorenz at Mon Jul 10 16​:21​:11 CEST 2017.

Summary of my perl5 (revision 5 version 27 subversion 2) configuration​:
Commit id​: 3072e75
Platform​:
osname=linux
osvers=4.1.38-50-default
archname=x86_64-linux
uname='linux borel 4.1.38-50-default #1 smp preempt sun feb 19
14​:35​:48 utc 2017 (6b4d8cb) x86_64 x86_64 x86_64 gnulinux '
config_args='-des -Dprefix=/store/borel/lorenz/prefixes/perlblead
-Duseshrplib -DDEBUGGING -Doptimize=-O1 -g -pipe -Dusedevel'
hint=recommended
useposix=true
d_sigaction=define
useithreads=undef
usemultiplicity=undef
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
bincompat5005=undef
Compiler​:
cc='cc'
ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe
-fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
optimize='-O1 -g -pipe'
cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe
-fstack-protector -I/usr/local/include'
ccversion=''
gccversion='4.8.5'
gccosandvers=''
intsize=4
longsize=8
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=3
ivtype='long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries​:
ld='cc'
ldflags =' -fstack-protector -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib64/gcc/x86_64-suse-linux/4.8/include-fixed
/usr/lib64/gcc/x86_64-suse-linux/4.8/../../../../x86_64-suse-linux/lib
/usr/lib /lib/../lib64 /usr/lib/../lib64 /lib /lib64 /usr/lib64
/usr/local/lib64
libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
-lgdbm_compat
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.22.so
so=so
useshrplib=true
libperl=libperl.so
gnulibc_version='2.22'
Dynamic Linking​:
dlsrc=dl_dlopen.xs
dlext=so
d_dlsymun=undef
ccdlflags='-Wl,-E -Wl,-rpath,/store/borel/
lorenz/prefixes/perlblead/lib/5.27.2/x86_64-linux/CORE'
cccdlflags='-fPIC'
lddlflags='-shared -O1 -g -pipe -L/usr/local/lib -fstack-protector'

---
@​INC for perl 5.27.2​:
/store/borel/lorenz/prefix/lib/perl5
/store/borel/lorenz/prefix/lib/perl5
/usr/site-local/lib/perl5/site_perl
/store/borel/lorenz/prefixes/perlblead/lib/site_perl/5.27.
2/x86_64-linux
/store/borel/lorenz/prefixes/perlblead/lib/site_perl/5.27.2
/store/borel/lorenz/prefixes/perlblead/lib/5.27.2/x86_64-linux
/store/borel/lorenz/prefixes/perlblead/lib/5.27.2

---
Environment for perl 5.27.2​:
HOME=/homes/combi/lorenz
LANG=en_US.UTF-8
LANGUAGE (unset)
LD_LIBRARY_PATH=/store/borel/lorenz/prefix/lib​:/store/
borel/lorenz/prefix/lib​:/usr/lib64/mpi/gcc/openmpi/lib64
LOGDIR (unset)
PATH=/store/borel/lorenz/prefix/bin​:/store/borel/
lorenz/prefix/bin​:/homes/combi/lorenz/.cabal/bin​:/
homes/combi/lorenz/.local/bin​:/store/borel/lorenz/prefix/
bin​:/store/borel/lorenz/prefix/bin​:/homes/combi/lorenz/.cabal/bin​:/homes/
combi/lorenz/.local/bin​:/net/TeXLive/bin/x86_64-linux​:/usr/
lib64/mpi/gcc/openmpi/bin​:/usr/local/bin​:/usr/bin​:/bin​:/
usr/bin/X11​:/usr/games​:/usr/lib/mit/bin​:/usr/lib/mit/sbin​:
/usr/site-local/bin​:/usr/site-local/share/bin
PERL5LIB=/store/borel/lorenz/prefix/lib/perl5​:/store/borel/
lorenz/prefix/lib/perl5​:/usr/site-local/lib/perl5/site_perl
PERL_BADLANG (unset)
PERL_LOCAL_LIB_ROOT=​:/store/borel/lorenz/prefix​:/store/
borel/lorenz/prefix
SHELL=/bin/bash

[p5pRT]

From @arc

This reduces to​:

$ ./miniperl -e 'my @​x; @​x; ()'

This isn't in fact a false positive​: a scalar- or void-context pp_list will always yield exactly one stack result, even if (as here) there are no arguments on the stack because the input array was empty. It must therefore extend the stack to account for its result.

Fixed in b54564c

--
Aaron Crane ** http​://aaroncrane.co.uk/

[p5pRT]

@arc - Status changed from 'open' to 'resolved'

[p5pRT]

From @iabyn

On Sun, Jul 16, 2017 at 03​:46​:06PM +0100, Tim Bunce wrote​:

Looks like DBI also fails some tests on 5.27.2 with -DDEBUGGING​:

http​://www.cpantesters.org/cpan/report/c5d1a148-69b3-11e7-b609-ee2501f1587f

Now fixed in blead with​:

commit 978b185
Author​: David Mitchell <davem@​iabyn.com>
AuthorDate​: Sun Jul 16 20​:00​:01 2017 +0100
Commit​: David Mitchell <davem@​iabyn.com>
CommitDate​: Sun Jul 16 20​:00​:01 2017 +0100

  PL_curstackinfo->si_stack_hwm​: gently restore
 
  RT #131732
 
  With v5.27.1-66-g87058c3, I introduced a DEBUGGING-only mechanism in the
  runops loop for checking whether an op extended the stack by as many slots
  as values it returned on the stack. It did this by setting a
  high-water-mark just before calling each pp function, and checking its
  result on return.
 
  It saved and restored the old value of PL_curstackinfo->si_stack_hwm
  whenever it entered or left a runops loop or did a JMPENV_PUSH /
  JMPENV_POP. However, the restoring could restore to an old value that was
  smaller than the current value, leading to false-positive stack-extend
  panics. So only restore if the old value was larger.
 
  In particular this was causing false positives in DBI.

--
No matter how many dust sheets you use, you will get paint on the carpet.