Perl_op_free: Assertion `!(o->op_private & ~PL_op_private_valid[type])' failed (op.c:721)

[p5pRT]

Migrated from rt.perl.org#123763 (status was 'resolved')

Searchable as RT123763$

[p5pRT]

From @geeknik

Built v5.21.9 (v5.21.8-237-g3c47da3) with the following command line​:

/Configure -des -Dusedevel -DDEBUGGING -Dcc=afl-gcc -Doptimize=-O2\ -g && AFL_HARDEN=1 make -j6 test-prep

Bug found with AFL (http​://lcamtuf.coredump.cx/afl)

GDB output​:
gdb-peda$ file ~/perl/perl
gdb-peda$ set args abort76.pl
gdb-peda$ r
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Use of my $_ is experimental at abort76.pl line 5.
Content-type​: text/html

Hello World.
Heres the form info​:<P>
perl​: av.c​:70​: Perl_av_extend​: Assertion `((svtype)((av)->sv_flags & 0xff)) == SVt_PVAV' failed.

Program received signal SIGABRT, Aborted.
[----------------------------------registers-----------------------------------]
RAX​: 0x0
RBX​: 0x7fffffffe60f --> 0x3a6469006c726570 ('perl')
RCX​: 0xffffffffffffffff
RDX​: 0x6
RSI​: 0x9a93
RDI​: 0x9a93
RBP​: 0x7ffff6ea9a07 --> 0x257325732500203a ('​: ')
RSP​: 0x7fffffffde48 --> 0x7ffff6d923e0 (<*__GI_abort+384>​: mov rdx,QWORD PTR fs​:0x10)
RIP​: 0x7ffff6d8f165 (<*__GI_raise+53>​: cmp rax,0xfffffffffffff000)
R8 : 0x7ffff7fdd700 (0x00007ffff7fdd700)
R9 : 0x27564156505f7456 ("Vt_PVAV'")
R10​: 0x8
R11​: 0x206
R12​: 0xefd3e8 ("((svtype)((av)->sv_flags & 0xff)) == SVt_PVAV")
R13​: 0xf6e922 ("Perl_av_extend")
R14​: 0x7ffff6ea9a07 --> 0x257325732500203a ('​: ')
R15​: 0x46 ('F')
EFLAGS​: 0x206 (carry PARITY adjust zero sign trap INTERRUPT direction overflow)
[-------------------------------------code-------------------------------------]
  0x7ffff6d8f15b <*__GI_raise+43>​: movsxd rdi,eax
  0x7ffff6d8f15e <*__GI_raise+46>​: mov eax,0xea
  0x7ffff6d8f163 <*__GI_raise+51>​: syscall
=> 0x7ffff6d8f165 <*__GI_raise+53>​: cmp rax,0xfffffffffffff000
  0x7ffff6d8f16b <*__GI_raise+59>​: ja 0x7ffff6d8f182 <*__GI_raise+82>
  0x7ffff6d8f16d <*__GI_raise+61>​: repz ret
  0x7ffff6d8f16f <*__GI_raise+63>​: nop
  0x7ffff6d8f170 <*__GI_raise+64>​: test eax,eax
[------------------------------------stack-------------------------------------]
0000| 0x7fffffffde48 --> 0x7ffff6d923e0 (<*__GI_abort+384>​: mov rdx,QWORD PTR fs​:0x10)
0008| 0x7fffffffde50 --> 0xefd3e8 ("((svtype)((av)->sv_flags & 0xff)) == SVt_PVAV")
0016| 0x7fffffffde58 --> 0x7ffff6eab9c1 --> 0x706c6568007325 ('%s')
0024| 0x7fffffffde60 --> 0x7fffffffde80 --> 0x3000000018
0032| 0x7fffffffde68 --> 0x46 ('F')
0040| 0x7fffffffde70 --> 0x7fffffffdf70 --> 0x7fffffffe60f --> 0x3a6469006c726570 ('perl')
0048| 0x7fffffffde78 --> 0x7ffff6dc3fe6 (<__fxprintf+310>​: lea rsp,[rbp-0x20])
0056| 0x7fffffffde80 --> 0x3000000018
[------------------------------------------------------------------------------]
Legend​: code, data, rodata, value
Stopped reason​: SIGABRT
0x00007ffff6d8f165 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c​:64
64 ../nptl/sysdeps/unix/sysv/linux/raise.c​: No such file or directory.

Debian 7, kernel v3.2.63-2+deb7u2 x86_64, libc6 v2.13-38+deb7u7, GCC 4.9.2

[p5pRT]

From @geeknik

abort76.pl

[p5pRT]

From @geeknik

Built v5.21.9 (v5.21.8-200-ga57d3d4) using the following command line​:

./Configure -des -Dusedevel -DDEBUGGING -Dcc=afl-gcc -Doptimize=-O2\ -g && AFL_HARDEN=1 make -j6 test-prep

Bug found with AFL (http​://lcamtuf.coredump.cx/afl)

GDB output​:
gdb-peda$ file ~/perl/perl
gdb-peda$ set args test76
gdb-peda$ r
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Use of my $_ is experimental at test76 line 1.
Can't modify constant item in scalar assignment at test76 line 1, at EOF
Execution of test76 aborted due to compilation errors.
perl​: op.c​:721​: Perl_op_free​: Assertion `!(o->op_private & ~PL_op_private_valid[type])' failed.

Program received signal SIGABRT, Aborted.
[----------------------------------registers-----------------------------------]
RAX​: 0x0
RBX​: 0x7fffffffe659 --> 0x736574006c726570 ('perl')
RCX​: 0xffffffffffffffff
RDX​: 0x6
RSI​: 0x9ba4
RDI​: 0x9ba4
RBP​: 0x7ffff6ea9a07 --> 0x257325732500203a ('​: ')
RSP​: 0x7fffffffdf18 --> 0x7ffff6d923e0 (<*__GI_abort+384>​: mov rdx,QWORD PTR fs​:0x10)
RIP​: 0x7ffff6d8f165 (<*__GI_raise+53>​: cmp rax,0xfffffffffffff000)
R8 : 0x7ffff7fdd700 (0x00007ffff7fdd700)
R9 : 0x2027295d65707974 ("type])' ")
R10​: 0x8
R11​: 0x202
R12​: 0xefd0d8 ("!(o->op_private & ~PL_op_private_valid[type])")
R13​: 0xf11132 ("Perl_op_free")
R14​: 0x7ffff6ea9a07 --> 0x257325732500203a ('​: ')
R15​: 0x2d1
EFLAGS​: 0x202 (carry parity adjust zero sign trap INTERRUPT direction overflow)
[-------------------------------------code-------------------------------------]
  0x7ffff6d8f15b <*__GI_raise+43>​: movsxd rdi,eax
  0x7ffff6d8f15e <*__GI_raise+46>​: mov eax,0xea
  0x7ffff6d8f163 <*__GI_raise+51>​: syscall
=> 0x7ffff6d8f165 <*__GI_raise+53>​: cmp rax,0xfffffffffffff000
  0x7ffff6d8f16b <*__GI_raise+59>​: ja 0x7ffff6d8f182 <*__GI_raise+82>
  0x7ffff6d8f16d <*__GI_raise+61>​: repz ret
  0x7ffff6d8f16f <*__GI_raise+63>​: nop
  0x7ffff6d8f170 <*__GI_raise+64>​: test eax,eax
[------------------------------------stack-------------------------------------]
0000| 0x7fffffffdf18 --> 0x7ffff6d923e0 (<*__GI_abort+384>​: mov rdx,QWORD PTR fs​:0x10)
0008| 0x7fffffffdf20 --> 0xefd0d8 ("!(o->op_private & ~PL_op_private_valid[type])")
0016| 0x7fffffffdf28 --> 0x7ffff6eab9c1 --> 0x706c6568007325 ('%s')
0024| 0x7fffffffdf30 --> 0x7fffffffdf50 --> 0x3000000018
0032| 0x7fffffffdf38 --> 0x2d1
0040| 0x7fffffffdf40 --> 0x7fffffffe040 --> 0x7fffffffe659 --> 0x736574006c726570 ('perl')
0048| 0x7fffffffdf48 --> 0x7ffff6dc3fe6 (<__fxprintf+310>​: lea rsp,[rbp-0x20])
0056| 0x7fffffffdf50 --> 0x3000000018
[------------------------------------------------------------------------------]
Legend​: code, data, rodata, value
Stopped reason​: SIGABRT
0x00007ffff6d8f165 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c​:64
64 ../nptl/sysdeps/unix/sysv/linux/raise.c​: No such file or directory.

Test case​: my($_);0=split
Hexdump​:
0000000 796d 2428 295f 303b 733d 6c70 7469
000000e

Debian 7, kernel v3.2.63-2+deb7u2 x86_64, libc6 v2.13-38+deb7u7, GCC 4.9.2

[p5pRT]

From @geeknik

test76

[p5pRT]

From @hvds

On Sun Feb 08 12​:52​:53 2015, brian.carpenter@​gmail.com wrote​:

Test case​: my($_);0=split

The op is OP_PUSHRE, and the flag in question is OPpTARGET_MY being set in Perl_ck_match.

If this is ok, I think the fix is as simple as the patch below followed by regen/opcode.pl. I can't tell if it is, though​: pp_pushre supplies its return value via XPUSHs which is in neither the safe nor the unsafe lists in the docs for the flag in regen/op_private.

Hugo

--- a/regen/op_private
+++ b/regen/op_private
@​@​ -381,7 +381,7 @​@​ addbits($_, 4 => qw(OPpTARGET_MY TARGMY))
  for ops_with_flag('T'),
  # This flag is also used to indicate matches against implicit $_,
  # where $_ is lexical; e.g. my $_; ....; /foo/
- qw(match subst trans transr);
+ qw(match subst pushre trans transr);
;

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @cpansprout

On Tue Feb 10 16​:05​:15 2015, hv wrote​:

On Sun Feb 08 12​:52​:53 2015, brian.carpenter@​gmail.com wrote​:

Test case​: my($_);0=split

The op is OP_PUSHRE, and the flag in question is OPpTARGET_MY being
set in Perl_ck_match.

Yes, that fix is right. The flag does nothing on pushre, but turning it off would be a waste of CPU cycles.

It gets turned on because the first argument to split (whether explicit or not) is a match op (which gets the flag if a lexical $_ is in scope), but it gets converted into a pushre op. Ultimately, whether lexical $_ was seen becomes irrelevant.

If this is ok, I think the fix is as simple as the patch below
followed by regen/opcode.pl. I can't tell if it is, though​: pp_pushre
supplies its return value via XPUSHs which is in neither the safe nor
the unsafe lists in the docs for the flag in regen/op_private.

That would only be a problem if it caused PL_opargs[OP_PUSHRE] & OA_TARGLEX to be true, which it does not. (Even then, I suspect the flag would still be harmless, because a lone pushre would never be on the rhs of an assignment.)

--- a/regen/op_private
+++ b/regen/op_private
@​@​ -381,7 +381,7 @​@​ addbits($_, 4 => qw(OPpTARGET_MY TARGMY))
for ops_with_flag('T'),
# This flag is also used to indicate matches against implicit $_,
# where $_ is lexical; e.g. my $_; ....; /foo/
- qw(match subst trans transr);
+ qw(match subst pushre trans transr);
;

--

Father Chrysostomos

[p5pRT]

From @hvds

"Father Chrysostomos via RT" <perlbug-followup@​perl.org> wrote​:
:Yes, that fix is right. The flag does nothing on pushre, but turning
:it off would be a waste of CPU cycles.

Thanks for the confirmation. I'll aim to add a test case and push
tomorrow.

Hugo

[p5pRT]

From @hvds

This case reduces to 'my($_); @​x = split'; I'm going to merge it into [perl #123763] since fixes for both issues will need to take account of each other.

Hugo

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @hvds

Ah, it might be more complicated than that​: rt123762 reduces to the near-identical 'my $_; @​x = split', and is not fixed by this change.

At the point OPpTARGET_MY is set in ck_match, we also set o->op_targ; later in pp_split if op_targ is set it is taken to be the pad offset for the array.

I think this may have been introduced by​:

commit fd017c0
Author​: Father Chrysostomos <sprout@​cpan.org>
Date​: Sat Oct 11 23​:33​:40 2014 -0700

  Optimise @​lexarray = split...
 
  <E2><80><98>@​pkgary = split //, $foo<E2><80><99> gets optimised such that the split writes
  directly to the array and the assignment doesn<E2><80><99>t have to happen.
 
  This commit makes it work also with lexical arrays. It only works for
  arrays declared previously; <E2><80><98>my @​a = split<E2><80><99> doesn<E2><80><99>t get optimised,
  just as <E2><80><98>local @​a = split<E2><80><99> doesn<E2><80><99>t.
 
  The pad offset is stored in the op_targ field of the pushre op, just
  as the GV is stored in its op_pmreplrootu field.

Hugo

[p5pRT]

From @cpansprout

On Wed Feb 11 03​:02​:11 2015, hv wrote​:

Ah, it might be more complicated than that​: rt123762 reduces to the
near-identical 'my $_; @​x = split', and is not fixed by this change.

At the point OPpTARGET_MY is set in ck_match, we also set o->op_targ;
later in pp_split if op_targ is set it is taken to be the pad offset
for the array.

I think this may have been introduced by​:

commit fd017c0
Author​: Father Chrysostomos <sprout@​cpan.org>
Date​: Sat Oct 11 23​:33​:40 2014 -0700

Optimise @​lexarray = split...

Yes, that is likely. We probably need to free the target, if any, in ck_split.

--

Father Chrysostomos

[p5pRT]

From @cpansprout

On Tue Feb 10 18​:37​:31 2015, hv wrote​:

"Father Chrysostomos via RT" <perlbug-followup@​perl.org> wrote​:
:Yes, that fix is right. The flag does nothing on pushre, but turning
:it off would be a waste of CPU cycles.

Thanks for the confirmation. I'll aim to add a test case and push
tomorrow.

I have applied your patch as 26f4cc1 (thank you) and added a test in 179b3fa.

--

Father Chrysostomos

[p5pRT]

From @cpansprout

On Wed Feb 11 09​:13​:38 2015, sprout wrote​:

On Wed Feb 11 03​:02​:11 2015, hv wrote​:

Ah, it might be more complicated than that​: rt123762 reduces to the
near-identical 'my $_; @​x = split', and is not fixed by this change.

At the point OPpTARGET_MY is set in ck_match, we also set o->op_targ;
later in pp_split if op_targ is set it is taken to be the pad offset
for the array.

I think this may have been introduced by​:

commit fd017c0
Author​: Father Chrysostomos <sprout@​cpan.org>
Date​: Sat Oct 11 23​:33​:40 2014 -0700

Optimise @​lexarray = split...

Yes, that is likely. We probably need to free the target, if any, in
ck_split.

And this I have done in 55b3980.

--

Father Chrysostomos

[p5pRT]

@cpansprout - Status changed from 'open' to 'pending release'