S_sublex_done: Assertion `(PL_parser->lex_inwhat) == OP_SUBST || (PL_parser->lex_inwhat) == OP_TRANS' failed (toke.c:2448)

[p5pRT]

Migrated from rt.perl.org#123955 (status was 'resolved')

Searchable as RT123955$

[p5pRT]

From @geeknik

Built v5.21.10 (v5.21.9-43-g2c3f32a) with the following command line​:

./Configure -des -Dusedevel -DDEBUGGING -Dcc=afl-gcc -Doptimize=-O2\ -g && AFL_HARDEN=1 make -j12 test-prep

Bug found with AFL (http​://lcamtuf.coredump.cx/afl)

Valgrind​:
perl​: toke.c​:2448​: S_sublex_done​: Assertion `(PL_parser->lex_inwhat) == OP_SUBST || (PL_parser->lex_inwhat) == OP_TRANS' failed.
==41409==
==41409== Process terminating with default action of signal 6 (SIGABRT)​: dumping core
==41409== at 0x5B55165​: raise (raise.c​:64)
==41409== by 0x5B583DF​: abort (abort.c​:92)
==41409== by 0x5B4E310​: __assert_fail (assert.c​:81)
==41409== by 0x6506D4​: S_sublex_done (toke.c​:2448)
==41409== by 0x5FA620​: Perl_yylex (toke.c​:4548)
==41409== by 0x65B6B4​: Perl_yyparse (perly.c​:322)
==41409== by 0x532070​: S_parse_body (perl.c​:2277)
==41409== by 0x539872​: perl_parse (perl.c​:1611)
==41409== by 0x42AF37​: main (perlmain.c​:114)

GDB​:
gdb-peda$ file ~/perl/perl
gdb-peda$ set args test01-min
gdb-peda$ r
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
perl​: toke.c​:2448​: S_sublex_done​: Assertion `(PL_parser->lex_inwhat) == OP_SUBST || (PL_parser->lex_inwhat) == OP_TRANS' failed.

Program received signal SIGABRT, Aborted.
[----------------------------------registers-----------------------------------]
RAX​: 0x0
RBX​: 0x7fffffffe63b --> 0x736574006c726570 ('perl')
RCX​: 0xffffffffffffffff
RDX​: 0x6
RSI​: 0x353b (';5')
RDI​: 0x353b (';5')
RBP​: 0x7ffff6ea9a07 --> 0x257325732500203a ('​: ')
RSP​: 0x7fffffffd838 --> 0x7ffff6d923e0 (<*__GI_abort+384>​: mov rdx,QWORD PTR fs​:0x10)
RIP​: 0x7ffff6d8f165 (<*__GI_raise+53>​: cmp rax,0xfffffffffffff000)
R8 : 0x7ffff7fdd700 (0x00007ffff7fdd700)
R9 : 0x27534e4152545f50 ("P_TRANS'")
R10​: 0x8
R11​: 0x202
R12​: 0xf056c8 ("(PL_parser->lex_inwhat) == OP_SUBST || (PL_parser->lex_inwhat) == OP_TRANS")
R13​: 0xf07098 ("S_sublex_done")
R14​: 0x7ffff6ea9a07 --> 0x257325732500203a ('​: ')
R15​: 0x990
EFLAGS​: 0x202 (carry parity adjust zero sign trap INTERRUPT direction overflow)
[-------------------------------------code-------------------------------------]
  0x7ffff6d8f15b <*__GI_raise+43>​: movsxd rdi,eax
  0x7ffff6d8f15e <*__GI_raise+46>​: mov eax,0xea
  0x7ffff6d8f163 <*__GI_raise+51>​: syscall
=> 0x7ffff6d8f165 <*__GI_raise+53>​: cmp rax,0xfffffffffffff000
  0x7ffff6d8f16b <*__GI_raise+59>​: ja 0x7ffff6d8f182 <*__GI_raise+82>
  0x7ffff6d8f16d <*__GI_raise+61>​: repz ret
  0x7ffff6d8f16f <*__GI_raise+63>​: nop
  0x7ffff6d8f170 <*__GI_raise+64>​: test eax,eax
[------------------------------------stack-------------------------------------]
0000| 0x7fffffffd838 --> 0x7ffff6d923e0 (<*__GI_abort+384>​: mov rdx,QWORD PTR fs​:0x10)
0008| 0x7fffffffd840 --> 0xf056c8 ("(PL_parser->lex_inwhat) == OP_SUBST || (PL_parser->lex_inwhat) == OP_TRANS")
0016| 0x7fffffffd848 --> 0x7ffff6eab9c1 --> 0x706c6568007325 ('%s')
0024| 0x7fffffffd850 --> 0x7fffffffd870 --> 0x3000000018
0032| 0x7fffffffd858 --> 0x990
0040| 0x7fffffffd860 --> 0x7fffffffd960 --> 0x7fffffffe63b --> 0x736574006c726570 ('perl')
0048| 0x7fffffffd868 --> 0x7ffff6dc3fe6 (<__fxprintf+310>​: lea rsp,[rbp-0x20])
0056| 0x7fffffffd870 --> 0x3000000018
[------------------------------------------------------------------------------]
Legend​: code, data, rodata, value
Stopped reason​: SIGABRT
0x00007ffff6d8f165 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c​:64
64 ../nptl/sysdeps/unix/sysv/linux/raise.c​: No such file or directory.

Test case is 21 bytes, here is a hexdump​:
0000000 3d30 4022 7b30 7330 3020 3030 3b22 7665
0000010 6c61 2422 0022
0000015

Debian 7, Kernel 3.2.65-1+deb7u1 x86_64, GCC 4.9.2, libc 2.13-38+deb7u7

[p5pRT]

From @geeknik

test01-min

[p5pRT]

From @cpansprout

On Sat Feb 28 18​:12​:37 2015, brian.carpenter@​gmail.com wrote​:

Test case is 21 bytes, here is a hexdump​:
0000000 3d30 4022 7b30 7330 3020 3030 3b22 7665
0000010 6c61 2422 0022
0000015

Based on where the assertion is coming from, I’m guessing this has been fixed by 479ae48, but I have not yet confirmed.

--

Father Chrysostomos

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @cpansprout

On Sat Feb 28 22​:14​:51 2015, sprout wrote​:

On Sat Feb 28 18​:12​:37 2015, brian.carpenter@​gmail.com wrote​:

Test case is 21 bytes, here is a hexdump​:
0000000 3d30 4022 7b30 7330 3020 3030 3b22 7665
0000010 6c61 2422 0022
0000015

Based on where the assertion is coming from, I’m guessing this has
been fixed by 479ae48, but I have not yet confirmed.

Actually, no, it’s not fixed.

--

Father Chrysostomos

[p5pRT]

From @cpansprout

On Sun Mar 01 11​:26​:01 2015, sprout wrote​:

On Sat Feb 28 22​:14​:51 2015, sprout wrote​:

On Sat Feb 28 18​:12​:37 2015, brian.carpenter@​gmail.com wrote​:

Test case is 21 bytes, here is a hexdump​:
0000000 3d30 4022 7b30 7330 3020 3030 3b22 7665
0000010 6c61 2422 0022
0000015

Based on where the assertion is coming from, I’m guessing this has
been fixed by 479ae48, but I have not yet confirmed.

Actually, no, it’s not fixed.

It’s related to #123617. It’s not only PL_lex_stuff that needs localisation, but also PL_sublex_info.repl.

When we evaluate "@​0{0s 000";eval"$" the ‘s 000’ is a substitution that causes both PL_lex_stuff and PL_sublex_info.repl to be set. Since the s comes after a 0, the parser immediately sees the syntax error and pops some tokens, causing PL_lex_stuff to be reset to NULL, since it is now localised as of eabab8b. PL_sublex_info.repl is still set, so the double quote that follows is treated as a two-part quote-like operator; hence the assertion failure.

This is now fixed in ce7c414.

--

Father Chrysostomos

[p5pRT]

@cpansprout - Status changed from 'open' to 'pending release'

[p5pRT]

From @khwilliamson

Thank you for submitting this ticket.

The issue should now be resolved with the release today of Perl v5.22, which is available at http​://www.perl.org/get.html
--
Karl Williamson for the Perl 5 team

[p5pRT]

@khwilliamson - Status changed from 'pending release' to 'resolved'