'x' operator on list causes segfault and confuses valgrind, 64-bit version #14972

p5pRT · 2015-10-09T03:03:12Z

Migrated from rt.perl.org#126309 (status was 'resolved')

Searchable as RT126309$

p5pRT · 2015-10-09T03:03:12Z

From @dcollinsn

Greetings Porters,

I have compiled bleadperl with the afl-gcc compiler using:

./Configure -Dusedevel -Dprefix='/usr/local/perl-afl' -Dcc='ccache afl-gcc' -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -des
AFL_HARDEN=1 make && make test

And then fuzzed the resulting binary using:

AFL_NO_VAR_CHECK=1 afl-fuzz -i in -o out bin/perl @@

After reducing testcases using `afl-tmin` and performing additional minimization by hand, I have located the following testcase that triggers a segmentation fault in the perl interpreter. The testcase is the file:

@0[(0)x~0]

As you might expect, ~0 is functioning as a really big number here, and this also segfaults:

@0[(0)x18446744073709551615]

However, this does not:

@0[(0)x8446744073709551615] => Out of memory during array extend at -e line 1.

And this gets awkward:

@0[(0)x1446744073709551615] => panic: realloc, size=11573952589676414152 at -e line 1.

This appears to be related to [perl #125937], although this persists after that bug was fixed (and this bug is the bigger badder 64-bit brother of that one)

**GDB**

(gdb) run
Starting program: /usr/local/perl-afl/bin/perl -e @0\[$0$x\~0\]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff6d4f735 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) bt
#0 0x00007ffff6d4f735 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x0000000000805220 in memcpy (__len=131072, __src=0x11f3a30,
__dest=<optimized out>) at /usr/include/x86_64-linux-gnu/bits/string3.h:51
#2 Perl_repeatcpy (to=0x11f3a30 "\030` \001", from=0x11f3a28 "\030` \001", len=8,
count=9223372036854775806) at util.c:3133
#3 0x0000000000a7ac5b in Perl_pp_repeat () at pp.c:1744
#4 0x00000000007dec7f in Perl_runops_debug () at dump.c:2224
#5 0x0000000000544469 in S_run_body (oldscope=1) at perl.c:2456
#6 perl_run (my_perl=<optimized out>) at perl.c:2379
#7 0x000000000042c6f8 in main (argc=3, argv=0x7fffffffe348, env=0x7fffffffe368)
at perlmain.c:116
(gdb) f 2
#2 Perl_repeatcpy (to=0x11f3a30 "\030` \001", from=0x11f3a28 "\030` \001", len=8,
count=9223372036854775806) at util.c:3133
3133 memcpy(p, to, size);
(gdb) info locals
size = 131072
p = <optimized out>
items = <optimized out>
linear = 4
half = 4611686018427387903
__PRETTY_FUNCTION__ = "Perl_repeatcpy"
(gdb) f 3
#3 0x0000000000a7ac5b in Perl_pp_repeat () at pp.c:1744
1744 }
(gdb) info locals
max = 9223372036854775807
sv = 0x7fffffffffffffff
infnan = false
__PRETTY_FUNCTION__ = "Perl_pp_repeat"

**VALGRIND**

Valgrind had a bad time with this one:

==50624== Memcheck, a memory error detector
==50624== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==50624== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==50624== Command: ../bin/perl -e @0[(0)x~0]
==50624==
==50624== Invalid write of size 8
==50624== at 0x4C2C363: memcpy@@GLIBC_2.14 (vg_replace_strmem.c:1018)
==50624== by 0x80521F: memcpy (string3.h:51)
==50624== by 0x80521F: Perl_repeatcpy (util.c:3133)
==50624== by 0xA7AC5A: Perl_pp_repeat (pp.c:1744)
==50624== by 0x7DEC7E: Perl_runops_debug (dump.c:2224)
==50624== by 0x544468: S_run_body (perl.c:2456)
==50624== by 0x544468: perl_run (perl.c:2379)
==50624== by 0x42C6F7: main (perlmain.c:116)
==50624== Address 0x5f54fe0 is 0 bytes after a block of size 1,024 alloc'd
==50624== at 0x4C27C0F: malloc (vg_replace_malloc.c:299)
==50624== by 0x7EFD6C: Perl_safesysmalloc (util.c:153)
==50624== by 0x8C3468: Perl_av_extend_guts (av.c:182)
==50624== by 0xB0E1BF: Perl_new_stackinfo (scope.c:56)
==50624== by 0x520F5A: Perl_init_stacks (perl.c:4063)
==50624== by 0x521515: perl_construct (perl.c:249)
==50624== by 0x42C403: main (perlmain.c:110)
==50624==
==50624== Invalid read of size 8
==50624== at 0x4C2C36E: memcpy@@GLIBC_2.14 (vg_replace_strmem.c:1018)
==50624== by 0x80521F: memcpy (string3.h:51)
==50624== by 0x80521F: Perl_repeatcpy (util.c:3133)
==50624== by 0xA7AC5A: Perl_pp_repeat (pp.c:1744)
==50624== by 0x7DEC7E: Perl_runops_debug (dump.c:2224)
==50624== by 0x544468: S_run_body (perl.c:2456)
==50624== by 0x544468: perl_run (perl.c:2379)
==50624== by 0x42C6F7: main (perlmain.c:116)
==50624== Address 0x5f54fe0 is 0 bytes after a block of size 1,024 alloc'd
==50624== at 0x4C27C0F: malloc (vg_replace_malloc.c:299)
==50624== by 0x7EFD6C: Perl_safesysmalloc (util.c:153)
==50624== by 0x8C3468: Perl_av_extend_guts (av.c:182)
==50624== by 0xB0E1BF: Perl_new_stackinfo (scope.c:56)
==50624== by 0x520F5A: Perl_init_stacks (perl.c:4063)
==50624== by 0x521515: perl_construct (perl.c:249)
==50624== by 0x42C403: main (perlmain.c:110)
==50624==

valgrind: m_mallocfree.c:303 (get_bszB_as_is): Assertion 'bszB_lo == bszB_hi' failed.
valgrind: Heap block lo/hi size mismatch: lo = 1088, hi = 100058968.
This is probably caused by your program erroneously writing past the
end of a heap block and corrupting heap metadata. If you fix any
invalid writes reported by Memcheck, this assertion failure will
probably go away. Please try that before reporting this as a bug.

host stacktrace:
==50624== at 0x38084838: show_sched_status_wrk (m_libcassert.c:343)
==50624== by 0x38084954: report_and_quit (m_libcassert.c:415)
==50624== by 0x38084AE1: vgPlain_assert_fail (m_libcassert.c:481)
==50624== by 0x380922ED: get_bszB_as_is (m_mallocfree.c:301)
==50624== by 0x380922ED: get_bszB (m_mallocfree.c:311)
==50624== by 0x380922ED: get_pszB (m_mallocfree.c:385)
==50624== by 0x380922ED: vgPlain_describe_arena_addr (m_mallocfree.c:1527)
==50624== by 0x3807DEEB: vgPlain_describe_addr (m_addrinfo.c:186)
==50624== by 0x3807C763: vgMemCheck_update_Error_extra (mc_errors.c:1141)
==50624== by 0x3808092A: vgPlain_maybe_record_error (m_errormgr.c:813)
==50624== by 0x3807BD2A: vgMemCheck_record_address_error (mc_errors.c:760)
==50624== by 0x3805A018: mc_LOADVn_slow (mc_main.c:1392)
==50624== by 0x803C85AF7: ???
==50624== by 0x80355BF2F: ???
==50624== by 0x80200833F: ???
==50624== by 0x80521F: memcpy (string3.h:51)
==50624== by 0x80521F: Perl_repeatcpy (util.c:3133)
==50624== by 0x80200833F: ???
==50624== by 0x80521F: memcpy (string3.h:51)
==50624== by 0x80521F: Perl_repeatcpy (util.c:3133)

sched status:
running_tid=1

Thread 1: status = VgTs_Runnable (lwpid 50624)
==50624== at 0x4C2C360: memcpy@@GLIBC_2.14 (vg_replace_strmem.c:1018)
==50624== by 0x80521F: memcpy (string3.h:51)
==50624== by 0x80521F: Perl_repeatcpy (util.c:3133)
==50624== by 0xA7AC5A: Perl_pp_repeat (pp.c:1744)
==50624== by 0x7DEC7E: Perl_runops_debug (dump.c:2224)
==50624== by 0x544468: S_run_body (perl.c:2456)
==50624== by 0x544468: perl_run (perl.c:2379)
==50624== by 0x42C6F7: main (perlmain.c:116)

Note: see also the FAQ in the source distribution.
It contains workarounds to several common problems.
In particular, if Valgrind aborted or crashed after
identifying problems in your program, there's a good chance
that fixing those problems will prevent Valgrind aborting or
crashing, especially if it happened in m_mallocfree.c.

If that doesn't help, please report this bug to: www.valgrind.org

In the bug report, send all the above text, the valgrind
version, and what OS and version you are using. Thanks.

**PERL -V**

Summary of my perl5 (revision 5 version 23 subversion 4) configuration:
Commit id: 94757bf
Platform:
osname=linux, osvers=3.16.0-4-amd64, archname=x86_64-linux-ld
uname='linux nightshade64 3.16.0-4-amd64 #1 smp debian 3.16.7-ckt11-1+deb8u4 (2015-09-19) x86_64 gnulinux '
config_args='-Dusedevel -Dprefix=/usr/local/perl-afl -Dcc=ccache afl-gcc -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -DDEBUGGING -DPERL_POISON -des'
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
use64bitint=define, use64bitall=define, uselongdouble=define
usemymalloc=n, bincompat5005=undef
Compiler:
cc='ccache afl-gcc', ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-g',
cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
ccversion='', gccversion='4.9.2', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
ivtype='long', ivsize=8, nvtype='long double', nvsize=16, Off_t='off_t', lseeksize=8
alignbytes=16, prototype=define
Linker and Libraries:
ld='ccache afl-gcc', ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.19.so, so=so, useshrplib=false, libperl=libperl.a
gnulibc_version='2.19'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fPIC', lddlflags='-shared -g -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl):
Compile-time options: DEBUGGING HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE
PERL_DONT_CREATE_GVSV
PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL
USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE
USE_LOCALE_COLLATE USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_LONG_DOUBLE
USE_PERLIO USE_PERL_ATOF
Built under linux
Compiled at Sep 30 2015 16:39:59
@INC:
/usr/local/perl-afl/lib/site_perl/5.23.4/x86_64-linux-ld
/usr/local/perl-afl/lib/site_perl/5.23.4
/usr/local/perl-afl/lib/5.23.4/x86_64-linux-ld
/usr/local/perl-afl/lib/5.23.4
.

p5pRT · 2015-10-19T20:32:24Z

From @dcollinsn

This now crashes with the following message, without a segfault, on both debugging and nondebugging perls on the same platform:

panic: av_extend_guts() negative count (-9223372036854775681)

dcollins@nightshade64:/usr/local/perl-afl/out$ ../bin/perl -V
Summary of my perl5 (revision 5 version 23 subversion 4) configuration:
Commit id: a7dba6f
Platform:
osname=linux, osvers=3.16.0-4-amd64, archname=x86_64-linux-ld
uname='linux nightshade64 3.16.0-4-amd64 #1 smp debian 3.16.7-ckt11-1+deb8u4 (2015-09-19) x86_64 gnulinux '
config_args='-Dusedevel -Dprefix=/usr/local/perl-afl -Dcc=ccache afl-gcc -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -DDEBUGGING -DDEBUG_LEAKING_SCALARS -des'
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
use64bitint=define, use64bitall=define, uselongdouble=define
usemymalloc=n, bincompat5005=undef
Compiler:
cc='ccache afl-gcc', ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-g',
cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
ccversion='', gccversion='4.9.2', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
ivtype='long', ivsize=8, nvtype='long double', nvsize=16, Off_t='off_t', lseeksize=8
alignbytes=16, prototype=define
Linker and Libraries:
ld='ccache afl-gcc', ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.19.so, so=so, useshrplib=false, libperl=libperl.a
gnulibc_version='2.19'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fPIC', lddlflags='-shared -g -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl):
Compile-time options: DEBUGGING HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE
PERL_DONT_CREATE_GVSV
PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL
USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE
USE_LOCALE_COLLATE USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_LONG_DOUBLE
USE_PERLIO USE_PERL_ATOF
Built under linux
Compiled at Oct 15 2015 20:35:14
@INC:
/usr/local/perl-afl/lib/site_perl/5.23.4/x86_64-linux-ld
/usr/local/perl-afl/lib/site_perl/5.23.4
/usr/local/perl-afl/lib/5.23.4/x86_64-linux-ld
/usr/local/perl-afl/lib/5.23.4
.

p5pRT · 2015-10-20T20:34:24Z

From @iabyn

On Mon, Oct 19, 2015 at 01:32:24PM -0700, Dan Collins via RT wrote:

This now crashes with the following message, without a segfault, on both
debugging and nondebugging perls on the same platform:

panic: av_extend_guts() negative count (-9223372036854775681)

Technically that's correct, in that its catching the error and throwing
a panic, but the error message could be better.

I've just pushed out a branch for smoking, smoke-me/davem/repeat that
changes the panic to

Out of memory during stack extend

and also fixes a spurious g++ compiler warning from my earlier messing
about with EXTEND() etc.

--
Monto Blanco... scorchio!

p5pRT · 2015-10-20T20:34:24Z

The RT System itself - Status changed from 'new' to 'open'

p5pRT · 2015-10-28T14:03:22Z

From @iabyn

On Tue, Oct 20, 2015 at 04:46:50PM +0100, Dave Mitchell wrote:

On Mon, Oct 19, 2015 at 01:32:24PM -0700, Dan Collins via RT wrote:

This now crashes with the following message, without a segfault, on both
debugging and nondebugging perls on the same platform:

panic: av_extend_guts() negative count (-9223372036854775681)

Technically that's correct, in that its catching the error and throwing
a panic, but the error message could be better.

I've just pushed out a branch for smoking, smoke-me/davem/repeat that
changes the panic to
Out of memory during stack extend
and also fixes a spurious g++ compiler warning from my earlier messing
about with EXTEND() etc.

and now merged as:

e9548aa

--
My Dad used to say 'always fight fire with fire', which is probably why
he got thrown out of the fire brigade.

p5pRT · 2015-10-29T11:15:20Z

@iabyn - Status changed from 'open' to 'pending release'

p5pRT · 2016-05-13T04:03:49Z

From @khwilliamson

Thank you for submitting this report. You have helped make Perl better.

With the release of Perl 5.24.0 on May 9, 2016, this and 149 other issues have been resolved.

Perl 5.24.0 may be downloaded via https://metacpan.org/release/RJBS/perl-5.24.0

p5pRT · 2016-05-13T04:03:49Z

@khwilliamson - Status changed from 'pending release' to 'resolved'

p5pRT closed this as completed May 13, 2016

p5pRT added the Severity Low label Oct 19, 2019

'x' operator on list causes segfault and confuses valgrind, 64-bit version #14972

'x' operator on list causes segfault and confuses valgrind, 64-bit version #14972

p5pRT commented Oct 9, 2015

p5pRT commented Oct 9, 2015

p5pRT commented Oct 19, 2015

p5pRT commented Oct 20, 2015

p5pRT commented Oct 20, 2015

p5pRT commented Oct 28, 2015

p5pRT commented Oct 29, 2015

p5pRT commented May 13, 2016

p5pRT commented May 13, 2016

'x' operator on list causes segfault and confuses valgrind, 64-bit version #14972

'x' operator on list causes segfault and confuses valgrind, 64-bit version #14972

Comments

p5pRT commented Oct 9, 2015

p5pRT commented Oct 9, 2015

From @dcollinsn

p5pRT commented Oct 19, 2015

From @dcollinsn

p5pRT commented Oct 20, 2015

From @iabyn

p5pRT commented Oct 20, 2015

p5pRT commented Oct 28, 2015

From @iabyn

p5pRT commented Oct 29, 2015

p5pRT commented May 13, 2016

From @khwilliamson

p5pRT commented May 13, 2016