heap-buffer-overflow in Perl_sv_vcatpvfn_flags (sv.c:12897)

[p5pRT]

Migrated from rt.perl.org#128951 (status was 'resolved')

Searchable as RT128951$

[p5pRT]

From @geeknik

The attached test case triggers a heap-buffer-overflow in Perl_sv_vcatpvfn_flags (sv.c​:12897). This was found with AFL, ASAN and libdislocator.so and affects v5.25.4 (v5.25.3-245-g2e66fe9). Perl 5.20.2 returns an error that says `Unrecognized character \xD7; marked by <-- HERE after !@​{<-- HERE near column -1 at test00 line 1.`

==13440==ERROR​: AddressSanitizer​: heap-buffer-overflow on address 0x60200000dea8 at pc 0x0000004a9880 bp 0x7ffd00eeac90 sp 0x7ffd00eea450
READ of size 10 at 0x60200000dea8 thread T0
  #0 0x4a987f in __asan_memcpy (/home/geeknik/perl/perl+0x4a987f)
  #1 0x980d45 in Perl_sv_vcatpvfn_flags /home/geeknik/perl/sv.c​:12897​:6
  #2 0x963bc6 in Perl_sv_vsetpvfn /home/geeknik/perl/sv.c​:10815​:5
  #3 0x7fef42 in Perl_vmess /home/geeknik/perl/util.c​:1560​:5
  #4 0x7fef42 in Perl_vcroak /home/geeknik/perl/util.c​:1789
  #5 0x7f53ac in Perl_croak /home/geeknik/perl/util.c​:1836​:5
  #6 0x66c3f0 in Perl_yylex /home/geeknik/perl/toke.c​:4901​:9
  #7 0x6ac741 in Perl_yyparse /home/geeknik/perl/perly.c​:334​:19
  #8 0x59bf12 in S_parse_body /home/geeknik/perl/perl.c​:2372​:9
  #9 0x59225c in perl_parse /home/geeknik/perl/perl.c​:1688​:2
  #10 0x4de7d5 in main /home/geeknik/perl/perlmain.c​:121​:18
  #11 0x7fcba632cb44 in __libc_start_main /build/glibc-uPj9cH/glibc-2.19/csu/libc-start.c​:287
  #12 0x4de46c in _start (/home/geeknik/perl/perl+0x4de46c)

0x60200000dea8 is located 8 bytes to the left of 10-byte region [0x60200000deb0,0x60200000deba)
allocated by thread T0 here​:
  #0 0x4c0deb in malloc (/home/geeknik/perl/perl+0x4c0deb)
  #1 0x7f5007 in Perl_safesysmalloc /home/geeknik/perl/util.c​:153​:21

SUMMARY​: AddressSanitizer​: heap-buffer-overflow ??​:0 __asan_memcpy
Shadow bytes around the buggy address​:
  0x0c047fff9b80​: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047fff9b90​: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047fff9ba0​: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047fff9bb0​: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047fff9bc0​: fa fa fa fa fa fa fd fd fa fa 00 02 fa fa 00 02
=>0x0c047fff9bd0​: fa fa 00 02 fa[fa]00 02 fa fa 00 04 fa fa 02 fa
  0x0c047fff9be0​: fa fa 00 02 fa fa 00 03 fa fa 00 02 fa fa 00 00
  0x0c047fff9bf0​: fa fa 00 02 fa fa 00 02 fa fa 00 fa fa fa 00 02
  0x0c047fff9c00​: fa fa 00 00 fa fa 00 00 fa fa 00 06 fa fa 00 fa
  0x0c047fff9c10​: fa fa 00 02 fa fa 05 fa fa fa 00 07 fa fa 00 01
  0x0c047fff9c20​: fa fa 00 02 fa fa 06 fa fa fa 00 02 fa fa 00 03
Shadow byte legend (one shadow byte represents 8 application bytes)​:
  Addressable​: 00
  Partially addressable​: 01 02 03 04 05 06 07
  Heap left redzone​: fa
  Heap right redzone​: fb
  Freed heap region​: fd
  Stack left redzone​: f1
  Stack mid redzone​: f2
  Stack right redzone​: f3
  Stack partial redzone​: f4
  Stack after return​: f5
  Stack use after scope​: f8
  Global redzone​: f9
  Global init order​: f6
  Poisoned by user​: f7
  Container overflow​: fc
  ASan internal​: fe
==13440==ABORTING

[p5pRT]

From @geeknik

test00

[p5pRT]

From @iabyn

On Mon, Aug 15, 2016 at 03​:15​:03PM -0700, Brian Carpenter wrote​:

The attached test case triggers a heap-buffer-overflow in Perl_sv_vcatpvfn_flags (sv.c​:12897). This was found with AFL, ASAN and libdislocator.so and affects v5.25.4 (v5.25.3-245-g2e66fe9). Perl 5.20.2 returns an error that says `Unrecognized character \xD7; marked by <-- HERE after !@​{<-- HERE near column -1 at test00 line 1.`

The src code contains the bytes​:

  @​ { \327 \n

after seeing "@​{" the lexer calls scan_ident(), which sees the \327 as an
ident, then calls S_skipspace_flags() to skip the spaces following the
ident. This moves the current cursor position to the \n, and since that's
a line boundary, its updates PL_linestart and PL_bufptr to point to \n
too.

When it finds that the next char isn't a '}', it does this​:

  /* Didn't find the closing } at the point we expected, so restore
  state such that the next thing to process is the opening { and */
  s = SvPVX(PL_linestr) + bracket; /* let the parser handle it */

i.e. it moves s back to the "{\317" then continues.

However, PL_linestart doesn't get reset, so later when the parser
encounters the \327 and tries to croak with "Unrecognized character %s ...",
when it prints out the section of src code in error, since s < PL_linestr,
negative string lengths and ASAN errors ensue.

I don't know the best way to fix this.

--
I before E. Except when it isn't.

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @cpansprout

On Tue Aug 16 08​:44​:17 2016, davem wrote​:

On Mon, Aug 15, 2016 at 03​:15​:03PM -0700, Brian Carpenter wrote​:

The attached test case triggers a heap-buffer-overflow in
Perl_sv_vcatpvfn_flags (sv.c​:12897). This was found with AFL, ASAN
and libdislocator.so and affects v5.25.4 (v5.25.3-245-g2e66fe9). Perl
5.20.2 returns an error that says `Unrecognized character \xD7;
marked by <-- HERE after !@​{<-- HERE near column -1 at test00 line
1.`

The src code contains the bytes​:

@​ { \327 \n

after seeing "@​{" the lexer calls scan_ident(), which sees the \327 as
an
ident, then calls S_skipspace_flags() to skip the spaces following the
ident. This moves the current cursor position to the \n, and since
that's
a line boundary, its updates PL_linestart and PL_bufptr to point to \n
too.

When it finds that the next char isn't a '}', it does this​:

/* Didn't find the closing } at the point we expected, so restore
state such that the next thing to process is the opening { and */
s = SvPVX(PL_linestr) + bracket; /* let the parser handle it */

i.e. it moves s back to the "{\317" then continues.

However, PL_linestart doesn't get reset, so later when the parser
encounters the \327 and tries to croak with "Unrecognized character %s
...",
when it prints out the section of src code in error, since s <
PL_linestr,
negative string lengths and ASAN errors ensue.

I don't know the best way to fix this.

This looks like fun. Do you mind if I take over?

--

Father Chrysostomos

[p5pRT]

From @iabyn

On Tue, Aug 16, 2016 at 02​:12​:12PM -0700, Father Chrysostomos via RT wrote​:

On Tue Aug 16 08​:44​:17 2016, davem wrote​:

On Mon, Aug 15, 2016 at 03​:15​:03PM -0700, Brian Carpenter wrote​:

The attached test case triggers a heap-buffer-overflow in
Perl_sv_vcatpvfn_flags (sv.c​:12897). This was found with AFL, ASAN
and libdislocator.so and affects v5.25.4 (v5.25.3-245-g2e66fe9). Perl
5.20.2 returns an error that says `Unrecognized character \xD7;
marked by <-- HERE after !@​{<-- HERE near column -1 at test00 line
1.`

The src code contains the bytes​:

@​ { \327 \n

after seeing "@​{" the lexer calls scan_ident(), which sees the \327 as
an
ident, then calls S_skipspace_flags() to skip the spaces following the
ident. This moves the current cursor position to the \n, and since
that's
a line boundary, its updates PL_linestart and PL_bufptr to point to \n
too.

When it finds that the next char isn't a '}', it does this​:

/* Didn't find the closing } at the point we expected, so restore
state such that the next thing to process is the opening { and */
s = SvPVX(PL_linestr) + bracket; /* let the parser handle it */

i.e. it moves s back to the "{\317" then continues.

However, PL_linestart doesn't get reset, so later when the parser
encounters the \327 and tries to croak with "Unrecognized character %s
...",
when it prints out the section of src code in error, since s <
PL_linestr,
negative string lengths and ASAN errors ensue.

I don't know the best way to fix this.

This looks like fun. Do you mind if I take over?

Isn't it handy that people have such differing concepts of what
constitutes 'fun'? Yes, please do take over :-).

--
Never do today what you can put off till tomorrow.

[p5pRT]

From @cpansprout

On Wed Aug 17 00​:09​:07 2016, davem wrote​:

On Tue, Aug 16, 2016 at 02​:12​:12PM -0700, Father Chrysostomos via RT wrote​:

This looks like fun. Do you mind if I take over?

Isn't it handy that people have such differing concepts of what
constitutes 'fun'? Yes, please do take over :-).

It’s practically a continuation of commit 2179133, so it was fairly easy (except for the stupid mistakes that had me wonder why I was getting fleeting crashes; I must be getting rusty).

In any case, it’s now fixed by bf8a9a1. I hope the following commits make the lexer slightly easier (ahem) to comprehend.

--

Father Chrysostomos

[p5pRT]

@cpansprout - Status changed from 'open' to 'pending release'

[p5pRT]

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release today of Perl 5.26.0, this and 210 other issues have been
resolved.

Perl 5.26.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.26.0

If you find that the problem persists, feel free to reopen this ticket.

[p5pRT]

@khwilliamson - Status changed from 'pending release' to 'resolved'