Deserializing a large array of nested objects gives "zend_mm_heap corrupted" #2

bkw · 2012-09-06T16:24:10Z

Unserializing largish object collections aborts php with "zend_mm_heap_corrupted".

Unfortunately I can't produce a testcase yet, since the data this happened with is company confidential.
I try to come up with a synthetic case though.

It is an array, that contains 50 php objects. The serialized payload is 517466 characters.

With USE_ZEND_ALLOC=0 I get this:

php(32501) malloc: *** error for object 0x7fe65ab2b048: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug

Stacktrace:

Application Specific Information:
*** error for object 0x7fbd8232b028: incorrect checksum for freed object - object was probably modified after being freed.

objc[32539]: garbage collection is OFF

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 libsystem_kernel.dylib 0x00007fff87f0182a __kill + 10
1 libsystem_c.dylib 0x00007fff8a571a9c abort + 177
2 libsystem_c.dylib 0x00007fff8a5934ac szone_error + 459
3 libsystem_c.dylib 0x00007fff8a5934e8 free_list_checksum_botch + 29
4 libsystem_c.dylib 0x00007fff8a59a38c tiny_malloc_from_free_list + 145
5 libsystem_c.dylib 0x00007fff8a59b00e szone_malloc_should_clear + 1115
6 libsystem_c.dylib 0x00007fff8a5d03c8 malloc_zone_malloc + 77
7 libsystem_c.dylib 0x00007fff8a5d11a4 malloc + 44
8 php 0x000000010f40fa2f _estrdup + 63
9 php 0x000000010f3e197e sapi_send_headers + 110
10 php 0x000000010f36fc99 php_header + 9
11 php 0x000000010f3e8ece php_ub_body_write + 78
12 php 0x000000010f3d799d php_printf + 157
13 php 0x000000010f3d89a4 php_error_cb + 1492
14 php 0x000000010f42ac45 zend_error + 469
15 msgpack.so 0x000000011270df63 php_msgpack_unserialize + 227 (msgpack.c:254)
16 msgpack.so 0x000000011270d055 zif_msgpack_unserialize + 85 (msgpack.c:331)
17 php 0x000000010f4917f9 zend_do_fcall_common_helper_SPEC + 1081
18 php 0x000000010f491ef1 execute + 609
19 php 0x000000010f42a968 zend_execute_scripts + 440
20 php 0x000000010f3d6872 php_execute_script + 722
21 php 0x000000010f4a8f01 main + 4529
22 php 0x000000010f2637d4 start + 52

bkw · 2012-09-11T18:15:45Z

Here is a test case that is consistantly giving segfaults on my osx 10.7.4, macports php 5.3.15.

 https://gist.github.com/3699796

Can anybody confirm this?

laruence · 2012-09-12T06:55:33Z

I can not get segfault, but:
PHP Warning: msgpack Parse error in /tmp/1.php on line 7
[Wed Sep 12 14:55:03 2012] Script: '/tmp/1.php'
/home/huixinchen/opensource/github/msgpack-php/msgpack_unpack.c(469) : Freeing 0x16CCAC08 (24 bytes), script=/tmp/1.php
[Wed Sep 12 14:55:03 2012] Script: '/tmp/1.php'
/home/huixinchen/opensource/github/msgpack-php/msgpack_unpack.c(477) : Freeing 0x16A33950 (8 bytes), script=/tmp/1.php
=== Total 2 memory leaks detected ===

Dexus · 2012-09-20T06:47:54Z

Are you fixing the memory leaks? i have the same problem with an array bigger than 40 objects and more than 456466 Chars of payload.

laruence · 2012-09-27T03:01:37Z

I can confirm the segfault now, digging

laruence · 2012-09-27T03:02:13Z

@advect could you please look at this too?

laruence · 2012-09-27T15:07:14Z

I got a fix:
$ git diff
diff --git a/msgpack_unpack.h b/msgpack_unpack.h
index 2ffa790..1221c7d 100644
--- a/msgpack_unpack.h
+++ b/msgpack_unpack.h
@@ -4,7 +4,7 @@

#include "ext/standard/php_var.h"

-#define MSGPACK_EMBED_STACK_SIZE 1024
+#define MSGPACK_EMBED_STACK_SIZE 2048

#include "msgpack/unpack_define.h"

but I need more time to confirm it, maybe we should use heap memory for that stack

@Dexus @bkw could you please try with this patch? thanks

bkw · 2012-10-10T20:13:50Z

Hey Laurence, thanks for the patch! Unfortunately, is does not seem to do what You expected:

The test script gives me:
Warning: [msgpack] (php_msgpack_unserialize) Parse error in […]/tests/bug002.php on line 11

And my own test above still segfaults:

Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x0000003644099df9

VM Regions Near 0x3644099df9:
    MALLOC_LARGE           0000000122599000-0000000122c99000 [ 7168K] rw-/rwx SM=PRV  
--> 
    MALLOC_TINY            00007fa2a2c00000-00007fa2a2f00000 [ 3072K] rw-/rwx SM=PRV  

Application Specific Information:
objc[24468]: garbage collection is OFF

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   php                             0x000000010ede2761 gc_zval_possible_root + 113
1   php                             0x000000010edd3353 zend_hash_destroy + 67
2   php                             0x000000010ede5b24 zend_objects_free_object_storage + 52
3   php                             0x000000010ede9401 zend_objects_store_del_ref_by_handle_ex + 401
4   php                             0x000000010ede94ba zend_objects_store_del_ref + 26
5   php                             0x000000010edbb3da _zval_ptr_dtor + 74
6   msgpack.so                      0x00000001120bbdc9 msgpack_unserialize_var_destroy + 89 (msgpack_unpack.c:304)
7   msgpack.so                      0x00000001120b6e90 php_msgpack_unserialize + 192 (msgpack.c:273)
8   msgpack.so                      0x00000001120b5fa5 zif_msgpack_unserialize + 85 (msgpack.c:334)
9   php                             0x000000010ee2d7f9 zend_do_fcall_common_helper_SPEC + 1081
10  php                             0x000000010ee2def1 execute + 609
11  php                             0x000000010edc6888 zend_execute_scripts + 440
12  php                             0x000000010ed72782 php_execute_script + 722
13  php                             0x000000010ee44f01 main + 4529
14  php                             0x000000010ebff624 start + 52

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000122959038  rbx: 0x0000000122a719f8  rcx: 0x0000003521740dc0  rdx: 0x00007fff6e7fbed0
  rdi: 0x0000000122a719f8  rsi: 0x0000000000000000  rbp: 0x00007fff6e7f1c40  rsp: 0x00007fff6e7f1c30
   r8: 0x0000000000000000   r9: 0x0000000000000000  r10: 0x0000000000000481  r11: 0x00000001120bbd70
  r12: 0x0000000000000000  r13: 0x00007fff6e7fc178  r14: 0x0000000122a72138  r15: 0x0000000122a721b8
  rip: 0x000000010ede2761  rfl: 0x0000000000000206  cr2: 0x0000003644099df9
Logical CPU: 0

advect · 2012-10-10T22:45:23Z

https://github.com/kjdev/php-ext-msgpacki

msgpacki is this no problem.
might be good.

bkw · 2012-10-11T11:04:44Z

php-msgpacki looks interesting, but it doesn't compile on osx yet. I'll give it a shot.

advect · 2012-10-11T22:30:21Z

It's a shame that

laruence · 2012-10-12T10:04:07Z

@advect it's better if there is only one msgpack php extension, so maybe we should contact with the msgpacki guy, merge our works together..

asiminiceanu · 2012-12-14T11:09:58Z

Hello,

The test bug002.phpt for this issue makes the RPM build impossible.
Can this test be marked as SKIP until the problem is resolved?

SKIP unpack of template converter: multiple class/variable: class unpacker (array) [tests/136b.phpt] reason: tests in PHP 5.1 or older
XFAIL Bug #2 (Deserializing a large array of nested objects gives "zend_mm_heap corrupted") [tests/bug002.phpt] 
PASS Bug #6 (bug with incorrect packing of mixed arrays) [tests/bug006.phpt] 
=====================================================================
TIME END 2012-12-14 11:50:34

=====================================================================
TEST RESULT SUMMARY
---------------------------------------------------------------------
Exts skipped    :    0
Exts tested     :   30
---------------------------------------------------------------------

Number of tests :  191               113
Tests skipped   :   78 ( 40.8%) --------
Tests warned    :    0 (  0.0%) (  0.0%)
Tests failed    :    0 (  0.0%) (  0.0%)
Expected fail   :    1 (  0.5%) (  0.9%)
Tests passed    :  112 ( 58.6%) ( 99.1%)
---------------------------------------------------------------------
Time taken      :    5 seconds
=====================================================================

=====================================================================
EXPECTED FAILED TEST SUMMARY
---------------------------------------------------------------------
Bug #2 (Deserializing a large array of nested objects gives "zend_mm_heap corrupted") [tests/bug002.phpt]
=====================================================================
+ unset NO_INTERACTION REPORT_EXIT_STATUS
++ find tests -name '*.diff' -type f -print
+ '[' -n tests/bug002.diff ']'
+ exit 1
error: Bad exit status from /var/tmp/rpm-tmp.85923 (%check)


RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.85923 (%check)

laruence · 2012-12-15T04:26:05Z

hey, maybe you can remove that from the rpm package?

thanks

…ck#2 (Deserializing a large array of nested objects gives "zend_mm_heap corrupted") [tests/bug002.phpt]"

asiminiceanu · 2012-12-15T18:47:12Z

Done. Check out the pull request #9

Program received signal SIGSEGV, Segmentation fault. add_next_index_zval (arg=0x0, value=0x159e650) at src/service/php/php-5.4.11/Zend/zend_API.c:1406 1406 return zend_hash_next_index_insert(Z_ARRVAL_P(arg), &value, sizeof(zval *), NULL); (gdb) bt #0 add_next_index_zval (arg=0x0, value=0x159e650) at src/service/php/php-5.4.11/Zend/zend_API.c:1406 msgpack#1 0x00007ffff3aefa52 in msgpack_unserialize_array_item (unpack=0x7fffffff11e0, container=0x7fffffff3290, obj=0x159e650) at src/service/php/msgpack-php/msgpack_unpack.c:498 msgpack#2 0x00007ffff3aea769 in template_execute (ctx=0x7fffffff11e0, data=<value optimized out>, len=<value optimized out>, off=<value optimized out>) at src/service/php/msgpack-php/msgpack/unpack_template.h:338 msgpack#3 0x00007ffff3aeb195 in php_msgpack_unserialize (return_value=0x13328c0, str=<value optimized out>, str_len=165671) at src/service/php/msgpack-php/msgpack.c:267

laruence added a commit that referenced this issue Sep 28, 2012

test script for #2

4e8f11a

laruence closed this as completed Dec 15, 2012

laruence reopened this Dec 15, 2012

asiminiceanu pushed a commit to asiminiceanu/msgpack-php that referenced this issue Dec 15, 2012

Temporary disabled the tests output check in order to skip "Bug msgpa…

ed44891

…ck#2 (Deserializing a large array of nested objects gives "zend_mm_heap corrupted") [tests/bug002.phpt]"

gutweiler mentioned this issue Jan 31, 2013

Fix unpacking empty array / segmentation fault. #11

Merged

joeyhub mentioned this issue May 18, 2015

Segfault 5.5.6 #49

Closed

m6w6 closed this as completed Dec 18, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deserializing a large array of nested objects gives "zend_mm_heap corrupted" #2

Deserializing a large array of nested objects gives "zend_mm_heap corrupted" #2

bkw commented Sep 6, 2012

bkw commented Sep 11, 2012

laruence commented Sep 12, 2012

Dexus commented Sep 20, 2012

laruence commented Sep 27, 2012

laruence commented Sep 27, 2012

laruence commented Sep 27, 2012

bkw commented Oct 10, 2012

advect commented Oct 10, 2012

bkw commented Oct 11, 2012

advect commented Oct 11, 2012

laruence commented Oct 12, 2012

asiminiceanu commented Dec 14, 2012

laruence commented Dec 15, 2012

asiminiceanu commented Dec 15, 2012

Deserializing a large array of nested objects gives "zend_mm_heap corrupted" #2

Deserializing a large array of nested objects gives "zend_mm_heap corrupted" #2

Comments

bkw commented Sep 6, 2012

bkw commented Sep 11, 2012

laruence commented Sep 12, 2012

Dexus commented Sep 20, 2012

laruence commented Sep 27, 2012

laruence commented Sep 27, 2012

laruence commented Sep 27, 2012

bkw commented Oct 10, 2012

advect commented Oct 10, 2012

bkw commented Oct 11, 2012

advect commented Oct 11, 2012

laruence commented Oct 12, 2012

asiminiceanu commented Dec 14, 2012

laruence commented Dec 15, 2012

asiminiceanu commented Dec 15, 2012