Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S_no_op: Assertion `s >= oldbp' failed. (toke.c:536) #14472

Closed
p5pRT opened this issue Feb 5, 2015 · 22 comments
Closed

S_no_op: Assertion `s >= oldbp' failed. (toke.c:536) #14472

p5pRT opened this issue Feb 5, 2015 · 22 comments

Comments

@p5pRT
Copy link

p5pRT commented Feb 5, 2015

Migrated from rt.perl.org#123737 (status was 'resolved')

Searchable as RT123737$

@p5pRT
Copy link
Author

p5pRT commented Feb 5, 2015

From @geeknik

Built v5.21.9 (v5.21.8-200-ga57d3d4) using the following command line​:

./Configure -des -Dusedevel -DDEBUGGING -Dcc=afl-gcc -Doptimize=-O2\ -g && AFL_HARDEN=1 make -j6 test-prep

Bug found with AFL (http​://lcamtuf.coredump.cx/afl)

GDB output​:
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Scalar found where operator expected at tokeabort line 1, near "0$"
perl​: toke.c​:536​: S_no_op​: Assertion `s >= oldbp' failed.

Program received signal SIGABRT, Aborted.
[----------------------------------registers-----------------------------------]
RAX​: 0x0
RBX​: 0x7fffffffe652 --> 0x6b6f74006c726570 ('perl')
RCX​: 0xffffffffffffffff
RDX​: 0x6
RSI​: 0x2952 ('R)')
RDI​: 0x2952 ('R)')
RBP​: 0x7ffff6ea9a07 --> 0x257325732500203a ('​: ')
RSP​: 0x7fffffffd738 --> 0x7ffff6d923e0 (<*__GI_abort+384>​: mov rdx,QWORD PTR fs​:0x10)
RIP​: 0x7ffff6d8f165 (<*__GI_raise+53>​: cmp rax,0xfffffffffffff000)
R8 : 0x7ffff7fdd700 (0x00007ffff7fdd700)
R9 : 0x6f5f6f6e5f53203a ('​: S_no_o')
R10​: 0x8
R11​: 0x202
R12​: 0xea9cc3 ("s >= oldbp")
R13​: 0xeb7225 --> 0x706f5f6f6e5f53 ('S_no_op')
R14​: 0x7ffff6ea9a07 --> 0x257325732500203a ('​: ')
R15​: 0x218
EFLAGS​: 0x202 (carry parity adjust zero sign trap INTERRUPT direction overflow)
[-------------------------------------code-------------------------------------]
  0x7ffff6d8f15b <*__GI_raise+43>​: movsxd rdi,eax
  0x7ffff6d8f15e <*__GI_raise+46>​: mov eax,0xea
  0x7ffff6d8f163 <*__GI_raise+51>​: syscall
=> 0x7ffff6d8f165 <*__GI_raise+53>​: cmp rax,0xfffffffffffff000
  0x7ffff6d8f16b <*__GI_raise+59>​: ja 0x7ffff6d8f182 <*__GI_raise+82>
  0x7ffff6d8f16d <*__GI_raise+61>​: repz ret
  0x7ffff6d8f16f <*__GI_raise+63>​: nop
  0x7ffff6d8f170 <*__GI_raise+64>​: test eax,eax
[------------------------------------stack-------------------------------------]
0000| 0x7fffffffd738 --> 0x7ffff6d923e0 (<*__GI_abort+384>​: mov rdx,QWORD PTR fs​:0x10)
0008| 0x7fffffffd740 --> 0xea9cc3 ("s >= oldbp")
0016| 0x7fffffffd748 --> 0x7ffff6eab9c1 --> 0x706c6568007325 ('%s')
0024| 0x7fffffffd750 --> 0x7fffffffd770 --> 0x3000000018
0032| 0x7fffffffd758 --> 0x218
0040| 0x7fffffffd760 --> 0x7fffffffd860 --> 0x7fffffffe652 --> 0x6b6f74006c726570 ('perl')
0048| 0x7fffffffd768 --> 0x7ffff6dc3fe6 (<__fxprintf+310>​: lea rsp,[rbp-0x20])
0056| 0x7fffffffd770 --> 0x3000000018
[------------------------------------------------------------------------------]
Legend​: code, data, rodata, value
Stopped reason​: SIGABRT
0x00007ffff6d8f165 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c​:64
64 ../nptl/sysdeps/unix/sysv/linux/raise.c​: No such file or directory.
gdb-peda$ bt
#0 0x00007ffff6d8f165 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c​:64
#1 0x00007ffff6d923e0 in *__GI_abort () at abort.c​:92
#2 0x00007ffff6d88311 in *__GI___assert_fail (assertion=assertion@​entry=0xea9cc3 "s >= oldbp", file=<optimized out>, file@​entry=0xea9a0d "toke.c", line=line@​entry=0x218, function=function@​entry=0xeb7225 "S_no_op") at assert.c​:81
#3 0x00000000005c2701 in S_no_op (what=what@​entry=0xeaa39e "Scalar", s=s@​entry=0x1182032 "{\n;") at toke.c​:536
#4 0x0000000000651615 in Perl_yylex () at toke.c​:5991
#5 0x000000000065c275 in Perl_yyparse (gramtype=<optimized out>) at perly.c​:322
#6 0x000000000052d275 in S_parse_body (env=env@​entry=0x0, xsinit=xsinit@​entry=0x42d080 <xs_init>) at perl.c​:2273
#7 0x000000000053324f in perl_parse (my_perl=<optimized out>, xsinit=0x42d080 <xs_init>, argc=<optimized out>, argv=<optimized out>, env=0x0) at perl.c​:1607
#8 0x000000000042cc8c in main (argc=0x2, argv=0x7fffffffe3c8, env=0x7fffffffe3e0) at perlmain.c​:114
#9 0x00007ffff6d7bead in __libc_start_main (main=<optimized out>, argc=<optimized out>, ubp_av=<optimized out>, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe3b8) at libc-start.c​:244
#10 0x000000000042cfa5 in _start ()

Test case hexdump​:
0000000 2430 0a7b
0000004

@p5pRT
Copy link
Author

p5pRT commented Feb 5, 2015

From @geeknik

tokeabort.gz

@p5pRT
Copy link
Author

p5pRT commented Feb 6, 2015

From @cpansprout

Fixed in 488bc57.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Feb 6, 2015

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Feb 6, 2015

@cpansprout - Status changed from 'open' to 'pending release'

@p5pRT
Copy link
Author

p5pRT commented Mar 17, 2015

From @geeknik

This bugger is back, albeit in a slight different part of toke.c.

Built this from git source​: (v5.21.10 (v5.21.9-259-g88d9f32)). I've
attached the test case which is a meager 6-bytes. Here is the hexdump
output​:
0000000 1f40 2324 0a7b
0000006

GDB​:
gdb-peda$ file ~/perl/perl
gdb-peda$ set args test43
gdb-peda$ r
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Use of literal control characters in variable names is deprecated at test43
line 1.
Array length found where operator expected at test43 line 1, near "@​ $#"
perl​: toke.c​:539​: S_no_op​: Assertion `s >= oldbp' failed.

Program received signal SIGABRT, Aborted.
[----------------------------------registers-----------------------------------]
RAX​: 0x0
RBX​: 0x7fffffffe626 --> 0x736574006c726570 ('perl')
RCX​: 0xffffffffffffffff
RDX​: 0x6
RSI​: 0xdea0
RDI​: 0xdea0
RBP​: 0x7ffff6ea9c67 --> 0x257325732500203a ('​: ')
RSP​: 0x7fffffffd798 --> 0x7ffff6d933e0 (<*__GI_abort+384>​: mov rdx,QWORD
PTR fs​:0x10)
RIP​: 0x7ffff6d90165 (<*__GI_raise+53>​: cmp rax,0xfffffffffffff000)
R8 : 0x7ffff7fdd700 (0x00007ffff7fdd700)
R9 : 0x6f5f6f6e5f53203a ('​: S_no_o')
R10​: 0x8
R11​: 0x206
R12​: 0xeef495 ("s >= oldbp")
R13​: 0xefc4f9 --> 0x706f5f6f6e5f53 ('S_no_op')
R14​: 0x7ffff6ea9c67 --> 0x257325732500203a ('​: ')
R15​: 0x21b
EFLAGS​: 0x206 (carry PARITY adjust zero sign trap INTERRUPT direction
overflow)
[-------------------------------------code-------------------------------------]
  0x7ffff6d9015b <*__GI_raise+43>​: movsxd rdi,eax
  0x7ffff6d9015e <*__GI_raise+46>​: mov eax,0xea
  0x7ffff6d90163 <*__GI_raise+51>​: syscall
=> 0x7ffff6d90165 <*__GI_raise+53>​: cmp rax,0xfffffffffffff000
  0x7ffff6d9016b <*__GI_raise+59>​: ja 0x7ffff6d90182 <*__GI_raise+82>
  0x7ffff6d9016d <*__GI_raise+61>​: repz ret
  0x7ffff6d9016f <*__GI_raise+63>​: nop
  0x7ffff6d90170 <*__GI_raise+64>​: test eax,eax
[------------------------------------stack-------------------------------------]
0000| 0x7fffffffd798 --> 0x7ffff6d933e0 (<*__GI_abort+384>​: mov
rdx,QWORD PTR fs​:0x10)
0008| 0x7fffffffd7a0 --> 0xeef495 ("s >= oldbp")
0016| 0x7fffffffd7a8 --> 0x7ffff6eabc21 --> 0x706c6568007325 ('%s')
0024| 0x7fffffffd7b0 --> 0x7fffffffd7d0 --> 0x3000000018
0032| 0x7fffffffd7b8 --> 0x21b
0040| 0x7fffffffd7c0 --> 0x7fffffffd8c0 --> 0x7fffffffe626 -->
0x736574006c726570 ('perl')
0048| 0x7fffffffd7c8 --> 0x7ffff6dc41b6 (<__fxprintf+310>​: lea
rsp,[rbp-0x20])
0056| 0x7fffffffd7d0 --> 0x3000000018
[------------------------------------------------------------------------------]
Legend​: code, data, rodata, value
Stopped reason​: SIGABRT
0x00007ffff6d90165 in *__GI_raise (sig=<optimized out>) at
../nptl/sysdeps/unix/sysv/linux/raise.c​:64
64 ../nptl/sysdeps/unix/sysv/linux/raise.c​: No such file or directory.
gdb-peda$ bt
#0 0x00007ffff6d90165 in *__GI_raise (sig=<optimized out>) at
../nptl/sysdeps/unix/sysv/linux/raise.c​:64
#1 0x00007ffff6d933e0 in *__GI_abort () at abort.c​:92
#2 0x00007ffff6d89311 in *__GI___assert_fail (assertion=0xeef495 "s >=
oldbp", file=<optimized out>,
  line=0x21b, function=0xefc4f9 "S_no_op") at assert.c​:81
#3 0x00000000005c2e69 in S_no_op () at toke.c​:539
#4 0x00000000005e5585 in Perl_yylex () at toke.c​:6012
#5 0x000000000065acf5 in Perl_yyparse ()
#6 0x0000000000534491 in perl_parse ()
#7 0x000000000042aed8 in main () at perlmain.c​:114
#8 0x00007ffff6d7cead in __libc_start_main (main=<optimized out>,
argc=<optimized out>,
  ubp_av=<optimized out>, init=<optimized out>, fini=<optimized out>,
rtld_fini=<optimized out>,
  stack_end=0x7fffffffe398) at libc-start.c​:244
#9 0x000000000042b1dd in _start ()
gdb-peda$ i r
rax 0x0 0x0
rbx 0x7fffffffe626 0x7fffffffe626
rcx 0xffffffffffffffff 0xffffffffffffffff
rdx 0x6 0x6
rsi 0xdea0 0xdea0
rdi 0xdea0 0xdea0
rbp 0x7ffff6ea9c67 0x7ffff6ea9c67
rsp 0x7fffffffd798 0x7fffffffd798
r8 0x7ffff7fdd700 0x7ffff7fdd700
r9 0x6f5f6f6e5f53203a 0x6f5f6f6e5f53203a
r10 0x8 0x8
r11 0x206 0x206
r12 0xeef495 0xeef495
r13 0xefc4f9 0xefc4f9
r14 0x7ffff6ea9c67 0x7ffff6ea9c67
r15 0x21b 0x21b
rip 0x7ffff6d90165 0x7ffff6d90165 <*__GI_raise+53>
eflags 0x206 [ PF IF ]
cs 0x33 0x33
ss 0x2b 0x2b
ds 0x0 0x0
es 0x0 0x0
fs 0x0 0x0
gs 0x0 0x0

Managed to minize the test case to 5-bytes, here is the hexdump​:
0000000 2430 7b23 000a
0000005

GDB​:
gdb-peda$ file ~/perl/perl
gdb-peda$ set args test43-min
gdb-peda$ r
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Array length found where operator expected at test43-min line 1, near "0$#"
perl​: toke.c​:539​: S_no_op​: Assertion `s >= oldbp' failed.

Program received signal SIGABRT, Aborted.
[----------------------------------registers-----------------------------------]
RAX​: 0x0
RBX​: 0x7fffffffe622 --> 0x736574006c726570 ('perl')
RCX​: 0xffffffffffffffff
RDX​: 0x6
RSI​: 0xaa82
RDI​: 0xaa82
RBP​: 0x7ffff6ea9c67 --> 0x257325732500203a ('​: ')
RSP​: 0x7fffffffd788 --> 0x7ffff6d933e0 (<*__GI_abort+384>​: mov rdx,QWORD
PTR fs​:0x10)
RIP​: 0x7ffff6d90165 (<*__GI_raise+53>​: cmp rax,0xfffffffffffff000)
R8 : 0x7ffff7fdd700 (0x00007ffff7fdd700)
R9 : 0x6f5f6f6e5f53203a ('​: S_no_o')
R10​: 0x8
R11​: 0x206
R12​: 0xeef495 ("s >= oldbp")
R13​: 0xefc4f9 --> 0x706f5f6f6e5f53 ('S_no_op')
R14​: 0x7ffff6ea9c67 --> 0x257325732500203a ('​: ')
R15​: 0x21b
EFLAGS​: 0x206 (carry PARITY adjust zero sign trap INTERRUPT direction
overflow)
[-------------------------------------code-------------------------------------]
  0x7ffff6d9015b <*__GI_raise+43>​: movsxd rdi,eax
  0x7ffff6d9015e <*__GI_raise+46>​: mov eax,0xea
  0x7ffff6d90163 <*__GI_raise+51>​: syscall
=> 0x7ffff6d90165 <*__GI_raise+53>​: cmp rax,0xfffffffffffff000
  0x7ffff6d9016b <*__GI_raise+59>​: ja 0x7ffff6d90182 <*__GI_raise+82>
  0x7ffff6d9016d <*__GI_raise+61>​: repz ret
  0x7ffff6d9016f <*__GI_raise+63>​: nop
  0x7ffff6d90170 <*__GI_raise+64>​: test eax,eax
[------------------------------------stack-------------------------------------]
0000| 0x7fffffffd788 --> 0x7ffff6d933e0 (<*__GI_abort+384>​: mov
rdx,QWORD PTR fs​:0x10)
0008| 0x7fffffffd790 --> 0xeef495 ("s >= oldbp")
0016| 0x7fffffffd798 --> 0x7ffff6eabc21 --> 0x706c6568007325 ('%s')
0024| 0x7fffffffd7a0 --> 0x7fffffffd7c0 --> 0x3000000018
0032| 0x7fffffffd7a8 --> 0x21b
0040| 0x7fffffffd7b0 --> 0x7fffffffd8b0 --> 0x7fffffffe622 -->
0x736574006c726570 ('perl')
0048| 0x7fffffffd7b8 --> 0x7ffff6dc41b6 (<__fxprintf+310>​: lea
rsp,[rbp-0x20])
0056| 0x7fffffffd7c0 --> 0x3000000018
[------------------------------------------------------------------------------]
Legend​: code, data, rodata, value
Stopped reason​: SIGABRT
0x00007ffff6d90165 in *__GI_raise (sig=<optimized out>) at
../nptl/sysdeps/unix/sysv/linux/raise.c​:64
64 ../nptl/sysdeps/unix/sysv/linux/raise.c​: No such file or directory.
gdb-peda$ bt
#0 0x00007ffff6d90165 in *__GI_raise (sig=<optimized out>) at
../nptl/sysdeps/unix/sysv/linux/raise.c​:64
#1 0x00007ffff6d933e0 in *__GI_abort () at abort.c​:92
#2 0x00007ffff6d89311 in *__GI___assert_fail (assertion=0xeef495 "s >=
oldbp", file=<optimized out>,
  line=0x21b, function=0xefc4f9 "S_no_op") at assert.c​:81
#3 0x00000000005c2e69 in S_no_op () at toke.c​:539
#4 0x00000000005e5585 in Perl_yylex () at toke.c​:6012
#5 0x000000000065acf5 in Perl_yyparse ()
#6 0x0000000000534491 in perl_parse ()
#7 0x000000000042aed8 in main () at perlmain.c​:114
#8 0x00007ffff6d7cead in __libc_start_main (main=<optimized out>,
argc=<optimized out>,
  ubp_av=<optimized out>, init=<optimized out>, fini=<optimized out>,
rtld_fini=<optimized out>,
  stack_end=0x7fffffffe388) at libc-start.c​:244
#9 0x000000000042b1dd in _start ()
gdb-peda$ i r
rax 0x0 0x0
rbx 0x7fffffffe622 0x7fffffffe622
rcx 0xffffffffffffffff 0xffffffffffffffff
rdx 0x6 0x6
rsi 0xaa82 0xaa82
rdi 0xaa82 0xaa82
rbp 0x7ffff6ea9c67 0x7ffff6ea9c67
rsp 0x7fffffffd788 0x7fffffffd788
r8 0x7ffff7fdd700 0x7ffff7fdd700
r9 0x6f5f6f6e5f53203a 0x6f5f6f6e5f53203a
r10 0x8 0x8
r11 0x206 0x206
r12 0xeef495 0xeef495
r13 0xefc4f9 0xefc4f9
r14 0x7ffff6ea9c67 0x7ffff6ea9c67
r15 0x21b 0x21b
rip 0x7ffff6d90165 0x7ffff6d90165 <*__GI_raise+53>
eflags 0x206 [ PF IF ]
cs 0x33 0x33
ss 0x2b 0x2b
ds 0x0 0x0
es 0x0 0x0
fs 0x0 0x0
gs 0x0 0x0

@p5pRT
Copy link
Author

p5pRT commented Mar 17, 2015

From @geeknik

test43

@p5pRT
Copy link
Author

p5pRT commented Mar 17, 2015

From @geeknik

test43-min

@p5pRT
Copy link
Author

p5pRT commented Mar 23, 2015

From @hvds

On Tue Mar 17 01​:52​:36 2015, brian.carpenter@​gmail.com wrote​:

This bugger is back, albeit in a slight different part of toke.c.

I just noticed that the ticket was not reopened, I'll do that now.

Brian's new test case is​:

% echo '0$#{' | ./miniperl -c
Array length found where operator expected at - line 1, near "0$#"
miniperl​: toke.c​:539​: S_no_op​: Assertion `s >= oldbp' failed.
Aborted (core dumped)
%

Hopefully Father C will get time to take a look.

Hugo

@p5pRT
Copy link
Author

p5pRT commented Mar 23, 2015

@hvds - Status changed from 'pending release' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Apr 24, 2015

From @wolfsage

On Sun, Mar 22, 2015 at 8​:41 PM, Hugo van der Sanden via RT <
perlbug-followup@​perl.org> wrote​:

On Tue Mar 17 01​:52​:36 2015, brian.carpenter@​gmail.com wrote​:

This bugger is back, albeit in a slight different part of toke.c.

I just noticed that the ticket was not reopened, I'll do that now.

Brian's new test case is​:

% echo '0$#{' | ./miniperl -c
Array length found where operator expected at - line 1, near "0$#"
miniperl​: toke.c​:539​: S_no_op​: Assertion `s >= oldbp' failed.
Aborted (core dumped)
%

I'm not sure if the original fix was enough or the right place to fix
things.

Back in 5.18.4, these both worked and reported the correct errors​:

  mhorsfall@​dory​:~$ ~/dpppperls/debug/perl-5.18.4/bin/perl5.18.4 -ce '0$#{'
  Array length found where operator expected at -e line 1, near "0$#"
  (Missing operator before $#?)
  syntax error at -e line 1, near "0$#"
  Missing right curly or square bracket at -e line 1, at end of line
  -e had compilation errors.

  mhorsfall@​dory​:~$ ~/dpppperls/debug/perl-5.18.4/bin/perl5.18.4 -ce '0${'
  Scalar found where operator expected at -e line 1, near "0$"
  (Missing operator before $?)
  syntax error at -e line 1, near "0$"
  Missing right curly or square bracket at -e line 1, at end of line
  -e had compilation errors.

In 5.19.5 with the following commit, these started panicing​:

  good - zero exit from ./perl -Ilib /home/mhorsfall/crash.pl
  a49b10d is the first bad commit
  commit a49b10d
  Author​: Brian Fraser <fraserbn@​gmail.com>
  Date​: Sun Sep 1 20​:41​:26 2013 -0300

  toke.c, scan_ident()​: use PEEKSPACE() to skip over whitespace.

  This fixes a number of bugs regarding whitespace and line numbers
  in scan_ident(), such as ${\nfoo\n} not increasing the line number,
  or ${\ntime\n[1]} not working.

  It goes through a number of hoops to get the correct line number for
  warnings emmitted from scan_ident, and reverts CopLINE to its
  original value if scan_ident() is giving up and returning from the
  point of the opening bracket, like in the case of ${\n\nfoo()}.

  :040000 040000 fea9796b35814ce4842f64bf81366bad5ee381ba
0bc66bc8eeb87e6160264cb0ae12e38e45803c1b M t
  :100644 100644 53ad9f85ce0b819b1cf33fd53bf57c21b43b6c21
682fe67af183d23171c93ecc4499949d3fd2cfe2 M toke.c
  bisect run success
  That took 742 seconds.

Later in 5.21.5, they started working again in non-debug builds, but begain
reporting errors incorrectly​:

("fixed" by​:)

  commit 59685a4
  Author​: Yves Orton <demerphq@​gmail.com>
  Date​: Tue Sep 23 01​:34​:27 2014 +0200

  add an assert that the length arg for UTF8f is non-negative

  If we dont we will just hit a different more confusing assert
  later. In production builds we zero elen so the args is assumed
  empty.

  mhorsfall@​dory​:~/p5/perl$ runperls -dm 5.21.5 -e '0${'
  /home/mhorsfall/dpppperls/default/perl-5.21.5/bin/perl5.21.5 -e '0${'
2>&1
  Scalar found where operator expected at -e line 1, near "0$"
  (Missing operator before ?)
  syntax error at -e line 1, near "0$"
  Missing right curly or square bracket at -e line 1, at end of line
  Execution of -e aborted due to compilation errors.
  child exited with value 255

  mhorsfall@​dory​:~/p5/perl$ runperls -dm 5.21.5 -e '0$#{'
  /home/mhorsfall/dpppperls/default/perl-5.21.5/bin/perl5.21.5 -e '0$#{'
2>&1
  Array length found where operator expected at -e line 1, near "0$#"
  (Missing operator before ?)
  syntax error at -e line 1, near "0$#"
  Missing right curly or square bracket at -e line 1, at end of line
  Execution of -e aborted due to compilation errors.
  child exited with value 255

(Notice that "Missing operator before ?)" doesn't have the identifier
anymore)

The fix for this ticket fixed the first case, but are these individual
fixes needed for each case, or is there some more global fix that covers
them all? (I'm really not sure)

Also, here's another broken one​:

  mhorsfall@​tworivers​:~$ perl -e '0@​'
  Array found where operator expected at -e line 1, at end of line
  (Missing operator before ?)
  syntax error at -e line 1, near "0@​
  "
  Execution of -e aborted due to compilation errors.

Though that never appeared to report properly. Also the newline after the @​
is strange...

In any case, I'm not sure if this needs to remain a blocker for 5.22 if we
don't fix it in time since it's been broken since 5.20...

Cheers,

-- Matthew Horsfall (alh)

@p5pRT
Copy link
Author

p5pRT commented Apr 25, 2015

From @iabyn

On Fri, Apr 24, 2015 at 01​:06​:41PM -0400, Matthew Horsfall (alh) wrote​:

In any case, I'm not sure if this needs to remain a blocker for 5.22 if we
don't fix it in time since it's been broken since 5.20...

I've just fixed the 0$#{ case with v5.21.11-17-g310a0d0, but
I'll leave the ticket open in case someone wants to do a more general fix
post 5.22.

--
Never do today what you can put off till tomorrow.

@p5pRT
Copy link
Author

p5pRT commented Nov 25, 2015

From @tonycoz

On Fri Apr 24 10​:07​:10 2015, alh wrote​:

The fix for this ticket fixed the first case, but are these individual
fixes needed for each case, or is there some more global fix that covers
them all? (I'm really not sure)

I think they'll need individual fixes, since they depend on the handler for each token advancing the buffer pointer to provide enough context.

Also, here's another broken one​:

mhorsfall@​tworivers​:~$ perl -e '0@​'
Array found where operator expected at -e line 1, at end of line
(Missing operator before ?)
syntax error at -e line 1, near "0@​
"
Execution of -e aborted due to compilation errors.

Though that never appeared to report properly. Also the newline after the @​
is strange...

The attached improves the Missing operator line, it doesn't try to handle '0@​$foo' but helps for '0@​foo'.

The newline for the syntax error line isn't specific to @​, it occurs for other similar synax errors too​:

$ ./perl -e '0$foo'
Scalar found where operator expected at -e line 1, near "0$foo"
  (Missing operator before $foo?)
syntax error at -e line 1, near "0$foo
"
Execution of -e aborted due to compilation errors.

Tony

@p5pRT
Copy link
Author

p5pRT commented Nov 25, 2015

From @tonycoz

0001-perl-123737-delay-reporting-a-missing-operator-for-a.patch
From 405867c6d927552e43332df4277784f77119b0e8 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Wed, 25 Nov 2015 16:07:51 +1100
Subject: [perl #123737] delay reporting a missing operator for arrays

Previously it was reported a the beginning of the '@' case, without
even skipping the @ symbol.

Make the code more similar to the scalar case and try to parse an
identifier first.
---
 t/lib/croak/toke | 9 +++++++++
 toke.c           | 7 ++++---
 2 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/t/lib/croak/toke b/t/lib/croak/toke
index 64012fb..50394da 100644
--- a/t/lib/croak/toke
+++ b/t/lib/croak/toke
@@ -37,6 +37,15 @@ syntax error at - line 1, near "0$#"
 Missing right curly or square bracket at - line 1, at end of line
 Execution of - aborted due to compilation errors.
 ########
+# NAME (Missing opertaor before @foo) [perl #123737]
+0@foo
+EXPECT
+Array found where operator expected at - line 1, near "0@foo"
+	(Missing operator before @foo?)
+syntax error at - line 1, near "0@foo
+"
+Execution of - aborted due to compilation errors.
+########
 # NAME Unterminated here-doc in string eval
 eval "<<foo"; die $@
 EXPECT
diff --git a/toke.c b/toke.c
index 6d6975c..169c970 100644
--- a/toke.c
+++ b/toke.c
@@ -6353,11 +6353,12 @@ Perl_yylex(pTHX)
 	TOKEN('$');
 
     case '@':
-	if (PL_expect == XOPERATOR)
-	    no_op("Array", s);
-	else if (PL_expect == XPOSTDEREF) POSTDEREF('@');
+        if (PL_expect == XPOSTDEREF)
+            POSTDEREF('@');
 	PL_tokenbuf[0] = '@';
 	s = scan_ident(s, PL_tokenbuf + 1, sizeof PL_tokenbuf - 1, FALSE);
+	if (PL_expect == XOPERATOR)
+	    no_op("Array", s);
 	pl_yylval.ival = 0;
 	if (!PL_tokenbuf[1]) {
 	    PREREF('@');
-- 
2.1.4

@p5pRT
Copy link
Author

p5pRT commented Jan 10, 2016

From @tonycoz

On Tue Nov 24 21​:22​:47 2015, tonyc wrote​:

Also, here's another broken one​:

mhorsfall@​tworivers​:~$ perl -e '0@​'
Array found where operator expected at -e line 1, at end of line
(Missing operator before ?)
syntax error at -e line 1, near "0@​
"
Execution of -e aborted due to compilation errors.

Though that never appeared to report properly. Also the newline after
the @​
is strange...

The attached improves the Missing operator line, it doesn't try to
handle '0@​$foo' but helps for '0@​foo'.

Applied as a7162bf.

Leaving open for further cases.

Tony

@p5pRT
Copy link
Author

p5pRT commented Jan 12, 2016

From @geeknik

I see your fix in the git shortlog (http​://perl5.git.perl.org/perl.git/shortlog), but Perl v5.23.7 (v5.23.6-104-g5dcc841) still SIGABRTs with perl -e '0@​{'​:

toke.c​:539​: S_no_op​: Assertion `s >= oldbp' failed"

Line 539 now as opposed to 536 in my original report.

On Sun Jan 10 15​:48​:38 2016, tonyc wrote​:

Applied as a7162bf.

Leaving open for further cases.

Tony

@p5pRT
Copy link
Author

p5pRT commented Jan 12, 2016

From @tonycoz

On Mon Jan 11 16​:55​:50 2016, brian.carpenter@​gmail.com wrote​:

I see your fix in the git shortlog
(http​://perl5.git.perl.org/perl.git/shortlog), but Perl v5.23.7
(v5.23.6-104-g5dcc841) still SIGABRTs with perl -e '0@​{'​:

toke.c​:539​: S_no_op​: Assertion `s >= oldbp' failed"

Line 539 now as opposed to 536 in my original report.

Oops, you're right, I got too into improving the message for 0@​foo.

The attached fixes it for me.

Tony

@p5pRT
Copy link
Author

p5pRT commented Jan 12, 2016

From @tonycoz

0001-perl-123737-handle-a-non-identifer-after-better-for-.patch
From 25dc4549efb21e888d4f0eaa858fa4fa2341562e Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Tue, 12 Jan 2016 15:39:00 +1100
Subject: [perl #123737] handle a non-identifer after @ better for a missing op

Previously this would assert().
---
 t/lib/croak/toke | 11 ++++++++++-
 toke.c           | 10 ++++++++--
 2 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/t/lib/croak/toke b/t/lib/croak/toke
index 50394da..18dfa24 100644
--- a/t/lib/croak/toke
+++ b/t/lib/croak/toke
@@ -37,7 +37,7 @@ syntax error at - line 1, near "0$#"
 Missing right curly or square bracket at - line 1, at end of line
 Execution of - aborted due to compilation errors.
 ########
-# NAME (Missing opertaor before @foo) [perl #123737]
+# NAME (Missing operator before @foo) [perl #123737]
 0@foo
 EXPECT
 Array found where operator expected at - line 1, near "0@foo"
@@ -46,6 +46,15 @@ syntax error at - line 1, near "0@foo
 "
 Execution of - aborted due to compilation errors.
 ########
+# NAME (Missing operator before @{) [perl #123737]
+0@{
+EXPECT
+Array found where operator expected at - line 1, near "0@{"
+	(Missing operator before @{?)
+syntax error at - line 1, near "0@"
+Missing right curly or square bracket at - line 1, at end of line
+Execution of - aborted due to compilation errors.
+########
 # NAME Unterminated here-doc in string eval
 eval "<<foo"; die $@
 EXPECT
diff --git a/toke.c b/toke.c
index 95ce3fd..23c3521 100644
--- a/toke.c
+++ b/toke.c
@@ -6368,8 +6368,14 @@ Perl_yylex(pTHX)
             POSTDEREF('@');
 	PL_tokenbuf[0] = '@';
 	s = scan_ident(s, PL_tokenbuf + 1, sizeof PL_tokenbuf - 1, FALSE);
-	if (PL_expect == XOPERATOR)
-	    no_op("Array", s);
+	if (PL_expect == XOPERATOR) {
+            d = s;
+            if (PL_bufptr > s) {
+                d = PL_bufptr-1;
+                PL_bufptr = PL_oldbufptr;
+            }
+	    no_op("Array", d);
+        }
 	pl_yylval.ival = 0;
 	if (!PL_tokenbuf[1]) {
 	    PREREF('@');
-- 
2.1.4

@p5pRT
Copy link
Author

p5pRT commented Jan 14, 2016

From @tonycoz

On Mon Jan 11 20​:39​:59 2016, tonyc wrote​:

On Mon Jan 11 16​:55​:50 2016, brian.carpenter@​gmail.com wrote​:

I see your fix in the git shortlog
(http​://perl5.git.perl.org/perl.git/shortlog), but Perl v5.23.7
(v5.23.6-104-g5dcc841) still SIGABRTs with perl -e '0@​{'​:

toke.c​:539​: S_no_op​: Assertion `s >= oldbp' failed"

Line 539 now as opposed to 536 in my original report.

Oops, you're right, I got too into improving the message for 0@​foo.

The attached fixes it for me.

Applied as 61d3025.

I also checked for similar problems in other calls to no_op() and didn't see any other cases I could make crash, so I'll close this ticket.

Tony

@p5pRT
Copy link
Author

p5pRT commented Jan 14, 2016

@tonycoz - Status changed from 'open' to 'pending release'

@p5pRT
Copy link
Author

p5pRT commented May 13, 2016

From @khwilliamson

Thank you for submitting this report. You have helped make Perl better.
 
With the release of Perl 5.24.0 on May 9, 2016, this and 149 other issues have been resolved.

Perl 5.24.0 may be downloaded via https://metacpan.org/release/RJBS/perl-5.24.0

@p5pRT
Copy link
Author

p5pRT commented May 13, 2016

@khwilliamson - Status changed from 'pending release' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant