Crash requesting jsondata from NSMutableDictionary created by JSONKit

[Vestrit]

When requesting jsondata from an NSMutableDictionary created by jsonkit, jsonkit will crash. It works fine for non mutable dictionaries.

Process:         someApp [33571]
Path:            /Users/USER/Library/Developer/Xcode/DerivedData/someApp-dzxzpbtokpnlvbfzxrgbssffgdva/Build/Products/Debug/someApp.app/Contents/MacOS/someApp
Identifier:      com.myCompany.someApp
Version:         1.0 (1)
Code Type:       X86-64 (Native)
Parent Process:  launchd [152]

Date/Time:       2011-03-14 11:02:51.272 -0700
OS Version:      Mac OS X 10.7 (11A390)
Report Version:  8

Interval Since Last Report:          349830 sec
Crashes Since Last Report:           1
Per-App Interval Since Last Report:  47732 sec
Per-App Crashes Since Last Report:   1
Anonymous UUID:                      92135CCF-450D-4DAA-8FF5-E194218A69AF

Crashed Thread:  0  Dispatch queue: com.apple.main-thread

Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x00000000069dcbc3

VM Regions Near 0x69dcbc3:
-->
   __TEXT                 0000000100000000-0000000100026000 [  152K] r-x/rwx SM=COW  /Users/USER/Library/Developer/Xcode/DerivedData/someApp-dzxzpbtokpnlvbfzxrgbssffgdva/Build/Products/Debug/someApp.app/Contents/MacOS/someApp

Application Specific Information:
objc[33571]: garbage collection is OFF
Performing @selector(someSelector:) from sender NSButton 0x1018170c0

Thread 0 name:  Dispatch queue: com.apple.main-thread
Thread 0 Crashed:
0   com.myCompany.someApp            0x000000010000bab3 jk_encode_add_atom_to_buffer + 1075 (JSONKit.m:2605)
1   com.myCompany.someApp            0x000000010000ea8f jk_encode_add_atom_to_buffer + 13327 (JSONKit.m:2795)
2   com.myCompany.someApp            0x000000010000e3e8 jk_encode_add_atom_to_buffer + 11624 (JSONKit.m:2765)
3   com.myCompany.someApp            0x000000010000afd5 jk_encode + 581 (JSONKit.m:2831)
4   com.myCompany.someApp            0x000000010000ad83 -[NSArray(JSONKit) JSONDataWithOptions:error:] + 67 (JSONKit.m:2891)
5   com.myCompany.someApp            0x000000010000ad38 -[NSArray(JSONKit) JSONData] + 56 (JSONKit.m:2886)
6   com.myCompany.someApp            0x000000010000260d -[someAppAppDelegate someMethod:] + 157 (someAppAppDelegate.m:118)
7   com.apple.CoreFoundation       0x00007fff8ad626ad -[NSObject performSelector:withObject:] + 61
8   com.apple.AppKit               0x00007fff8998ed00 -[NSApplication sendAction:to:from:] + 139
9   com.apple.AppKit               0x00007fff8998ec32 -[NSControl sendAction:to:] + 88
10  com.apple.AppKit               0x00007fff89a14a4e -[NSCell _sendActionFrom:] + 137
11  com.apple.AppKit               0x00007fff89a14355 -[NSCell trackMouse:inRect:ofView:untilMouseUp:] + 2014
12  com.apple.AppKit               0x00007fff89a4313a -[NSButtonCell trackMouse:inRect:ofView:untilMouseUp:] + 489
13  com.apple.AppKit               0x00007fff89a12e7b -[NSControl mouseDown:] + 786
14  com.apple.AppKit               0x00007fff899393fc -[NSWindow sendEvent:] + 6288
15  com.apple.AppKit               0x00007fff898704a6 -[NSApplication sendEvent:] + 4847
16  com.apple.AppKit               0x00007fff89808593 -[NSApplication run] + 541
17  com.apple.AppKit               0x00007fff8980133d NSApplicationMain + 860
18  com.myCompany.someApp            0x0000000100001d32 main + 34 (main.m:13)
19  com.myCompany.someApp            0x0000000100001d04 start + 52

Thread 1:
0   libsystem_kernel.dylib         0x00007fff8cb771b2 __workq_kernreturn + 10
1   libsystem_c.dylib              0x00007fff897414d9 _pthread_wqthread + 758
2   libsystem_c.dylib              0x00007fff89742599 start_wqthread + 13

Thread 2 name:  Dispatch queue: com.apple.libdispatch-manager
Thread 2:
0   libsystem_kernel.dylib         0x00007fff8cb77806 kevent + 10
1   libdispatch.dylib              0x00007fff8cebaa26 _dispatch_mgr_invoke + 916
2   libdispatch.dylib              0x00007fff8cebb877 _dispatch_queue_invoke + 63
3   libdispatch.dylib              0x00007fff8cebb070 _dispatch_worker_thread2 + 198
4   libsystem_c.dylib              0x00007fff8974131f _pthread_wqthread + 316
5   libsystem_c.dylib              0x00007fff89742599 start_wqthread + 13

Thread 3:
0   libsystem_kernel.dylib         0x00007fff8cb771b2 __workq_kernreturn + 10
1   libsystem_c.dylib              0x00007fff897414d9 _pthread_wqthread + 758
2   libsystem_c.dylib              0x00007fff89742599 start_wqthread + 13

Thread 4:
0   libsystem_kernel.dylib         0x00007fff8cb771b2 __workq_kernreturn + 10
1   libsystem_c.dylib              0x00007fff897414d9 _pthread_wqthread + 758
2   libsystem_c.dylib              0x00007fff89742599 start_wqthread + 13

Thread 5 name:  com.apple.NSURLConnectionLoader
Thread 5:
0   libsystem_kernel.dylib         0x00007fff8cb756b6 mach_msg_trap + 10
1   libsystem_kernel.dylib         0x00007fff8cb74dad mach_msg + 73
2   com.apple.CoreFoundation       0x00007fff8ad291ec CFRunLoopServiceMachPort + 188
3   com.apple.CoreFoundation       0x00007fff8ac88a44 __CFRunLoopRun + 1204
4   com.apple.CoreFoundation       0x00007fff8ac88356 CFRunLoopRunSpecific + 230
5   com.apple.Foundation           0x00007fff86e77fca +[NSURLConnection(NSURLConnectionReallyInternal) _resourceLoadLoop:] + 335
6   com.apple.Foundation           0x00007fff86dff681 -[NSThread main] + 68
7   com.apple.Foundation           0x00007fff86dff5f9 __NSThread__main + 1575
8   libsystem_c.dylib              0x00007fff8973f804 _pthread_start + 335
9   libsystem_c.dylib              0x00007fff89742589 thread_start + 13

Thread 6 name:  com.apple.CFSocket.private
Thread 6:
0   libsystem_kernel.dylib         0x00007fff8cb76e12 __select + 10
1   com.apple.CoreFoundation       0x00007fff8aca7e1b __CFSocketManager + 1355
2   libsystem_c.dylib              0x00007fff8973f804 _pthread_start + 335
3   libsystem_c.dylib              0x00007fff89742589 thread_start + 13

Thread 0 crashed with X86 Thread State (64-bit):
 rax: 0x00000000069dcbc3  rbx: 0x00000001018170c0  rcx: 0x0000000000000000  rdx: 0x00007fff5fbfe468
 rdi: 0x00007fff5fbf8e00  rsi: 0x00007fff5fbf8f30  rbp: 0x00007fff5fbe69b0  rsp: 0x00007fff5fbdffa0
  r8: 0x0000000000000000   r9: 0x0000000100515d20  r10: 0x00007fff8ad62310  r11: 0x000000010011a540
 r12: 0x00007fff74c6b9f0  r13: 0x0000000101817400  r14: 0x000000010001e694  r15: 0x000000010182e0c0
 rip: 0x000000010000bab3  rfl: 0x0000000000010246  cr2: 0x00000000069dcbc3
Logical CPU: 4

Binary Images:
      0x100000000 -        0x100025ff7  com.myCompany.someApp (1.0 - 1) <50EC498D-ED9C-321A-B86B-224B4DA9AD55> /Users/USER/Library/Developer/Xcode/DerivedData/someApp-dzxzpbtokpnlvbfzxrgbssffgdva/Build/Products/Debug/someApp.app/Contents/MacOS/someAppsomeApp

     VM Region Summary:
ReadOnly portion of Libraries: Total=135.9M resident=70.0M(52%) swapped_out_or_unallocated=65.9M(48%)
Writable regions: Total=130.6M written=17.8M(14%) resident=27.5M(21%) swapped_out=0K(0%) unallocated=103.1M(79%)

REGION TYPE                      VIRTUAL
===========                      =======
CG backing stores                  5036K
CG image                             32K
CG raster data                      312K
CG shared images                   4496K
Carbon                             2324K
Core Image                           12K
CoreGraphics                         16K
MALLOC                             90.5M        see MALLOC ZONE table below
MALLOC freed, no zone              2588K
MALLOC guard page                    48K
MALLOC metadata                     540K
Memory tag=240                        4K
Memory tag=242                       12K
Memory tag=243                        8K
Memory tag=251                        8K
OpenCL                               12K
SQLite page cache                  1344K
STACK GUARD                        56.0M
Stack                              10.6M
VM_ALLOCATE                        16.1M
__CI_BITMAP                          72K
__DATA                             11.1M
__IMAGE                            1244K
__LINKEDIT                         43.4M
__TEXT                             92.5M
__UNICODE                           544K
mapped file                        35.5M
shared memory                       312K
===========                      =======
TOTAL                             374.3M

                                         VIRTUAL ALLOCATION      BYTES
MALLOC ZONE                                  SIZE      COUNT  ALLOCATED  % FULL
===========                               =======  =========  =========  ======
DefaultMallocZone_0x10003a000               80.4M      70135      13.7M     16%
DispatchContinuations_0x10006c000           6144K          1         32      0%
unnamed_zone_0x1004fa000                    4096K          8        480      0%
CoreGraphics_0x102000c00                      80K        618        12K     15%
x-alloc_0x1008a5400                            0K          2        224
DefaultPurgeableMallocZone_0x101fa8000         0K          0         0K
===========                               =======  =========  =========  ======
TOTAL                                       90.5M      70764      13.7M     15%

Model: MacPro5,1, BootROM MP51.007F.B00, 4 processors, Quad-Core Intel Xeon, 2.8 GHz, 8 GB, SMC 1.39f11
Graphics: ATI Radeon HD 5770, ATI Radeon HD 5770, PCIe, 1024 MB
Memory Module: DIMM 1, 2 GB, DDR3 ECC, 1066 MHz, 0x80AD, 0x484D54313235553742465238432D47372020
Memory Module: DIMM 2, 2 GB, DDR3 ECC, 1066 MHz, 0x80AD, 0x484D54313235553742465238432D47372020
Memory Module: DIMM 3, 2 GB, DDR3 ECC, 1066 MHz, 0x80AD, 0x484D54313235553742465238432D47372020
Memory Module: DIMM 4, 2 GB, DDR3 ECC, 1066 MHz, 0x80AD, 0x484D54313235553742465238432D47372020

[johnezang]

I've tried to recreate this problem, but so far no luck.

It looks like this might be a stack smash from the provided crash dump info.

I did check in 7054853, which includes a number of additional assertions to make sure that the encode cache doesn't copy anything it's not supposed to and accidentally smash the stack in the process.

I can only suggest that you try again with 7054853 and make sure that NS_BLOCK_ASSERTIONS is not enabled, and preferably that the optimizer is turned off. If you continue to have problems, you will probably have to supply the JSON in question so that I can reproduce the problem.

[johnezang]

I have not been able to reproduce this problem, nor have I heard from anyone else who has encountered this problem. If possible, please provide some example code that reproduces this problem.

[Vestrit]

NSData *data = ...; // Data from an NSURLRequest
JSONDecoder *decoder = [ JSONDecoder decoder];
NSMutableDictionary *jsonData = [ decoder mutableObjectWithData:object.data];
[ jsonData JSONData]; // Crash here

We noticed that this seems to only occur on 64 bit builds and not 32 bit builds.. It also seems to perhaps be specific to a dev seed of a newer OS...

[johnezang]

Can you provide the JSON?

[johnezang]

If you need to keep some information about this problem "off the public record", you can email me at john.engelhart AT gmail.com.

Also, I noticed something about the stack trace in the problem description. The problem description states

When requesting jsondata from an NSMutableDictionary created by jsonkit, jsonkit will crash. It works fine for non mutable dictionaries.

However, the stack trace contains:

4   com.myCompany.someApp            0x000000010000ad83 -[NSArray(JSONKit) JSONDataWithOptions:error:] + 67 (JSONKit.m:2891)
5   com.myCompany.someApp            0x000000010000ad38 -[NSArray(JSONKit) JSONData] + 56 (JSONKit.m:2886)
6   com.myCompany.someApp            0x000000010000260d -[someAppAppDelegate someMethod:] + 157 (someAppAppDelegate.m:118)
7   com.apple.CoreFoundation         0x00007fff8ad626ad -[NSObject performSelector:withObject:] + 61

Note that the receiver of the JSONData message has NSArray(JSONKit). This means that the IMP that was selected, based on the class of the receiver, was the NSArray(JSONKit) category version.

It's not possible to tell the class of the receiver from the stack trace. The fact that the stack trace includes the NSArray(JSONKit) JSONData IMP does not necessarily imply the receiver is a JK(Mutable?)Array. Certainly one would never expect the ObjC runtime to dispatch the JKArray(JSONKit) JSONData IMP for a JKDictionary class object.... but if it DID do that, I strongly expect that things would.... not work correctly, and probably crash, as the two object layouts are completely different.

This suggests that adding a quick NSLog("Object class: %@", NSStringFromClass([jsonObject class])); just before the JSONData crashing call would be useful, and report back the result. The expected result would be JKMutableDictionary, and if that is indeed the case AND the stack trace has JKArray(JSONKit), then we've narrowed the problem down a bit..

[Vestrit]

John,

We are working on sanitizing the data and making sure it still occurs
with that version. Once we have that I'll send a sample project over to you.

On Friday, March 25, 2011, johnezang
reply@reply.github.com
wrote:

If you need to keep some information about this problem "off the public record", you can email me at john.engelhart AT gmail.com.

Also, I noticed something about the stack trace in the problem description.  The problem description states

When requesting jsondata from an NSMutableDictionary created by jsonkit, jsonkit will crash. It works fine for non mutable dictionaries.

However, the stack trace contains:

4   com.myCompany.someApp            0x000000010000ad83 -[NSArray(JSONKit) JSONDataWithOptions:error:] + 67 (JSONKit.m:2891)
5   com.myCompany.someApp            0x000000010000ad38 -[NSArray(JSONKit) JSONData] + 56 (JSONKit.m:2886)
6   com.myCompany.someApp            0x000000010000260d -[someAppAppDelegate someMethod:] + 157 (someAppAppDelegate.m:118)
7   com.apple.CoreFoundation         0x00007fff8ad626ad -[NSObject performSelector:withObject:] + 61

Note that the receiver of the JSONData message has NSArray(JSONKit).  This means that the IMP that was selected, based on the class of the receiver, was the NSArray(JSONKit) category version.

It's not possible to tell the class of the receiver from the stack trace.  The fact that the stack trace includes the NSArray(JSONKit) JSONData IMP does not necessarily imply the receiver is a JK(Mutable?)Array.  Certainly one would never expect the ObjC runtime to dispatch the JKArray(JSONKit) JSONData IMP for a JKDictionary class object.... but if it DID do that, I strongly expect that things would.... not work correctly, and probably crash, as the two object layouts are completely different.

This suggests that adding a quick NSLog("Object class: %@", NSStringFromClass([jsonObject class])); just before the JSONData crashing call would be useful, and report back the result.  The expected result would be JKMutableDictionary, and if that is indeed the case AND the stack trace has JKArray(JSONKit), then we've narrowed the problem down a bit..

Reply to this email directly or view it on GitHub:
#9 (comment)

[johnezang]

You might want to try the version of JSONKit.m from commit daf18f3 (and onwards). This commit changes the way in which the immutable / mutable concrete collection classes are done. It also does away with the swizzling hack I had in place which always left me a bit uncomfortable, and I've wondered if that technique didn't play a role in the problem that you're seeing.

[Vestrit]

I will pull an updated version in to our svn Monday and run everything
through our tests again to see if that fixed it!

On Friday, March 25, 2011, johnezang
reply@reply.github.com
wrote:

You might want to try the version of JSONKit.m from commit daf18f3 (and onwards).  This commit changes the way in which the immutable / mutable concrete collection classes are done.  It also does away with the swizzling hack I had in place which always left me a bit uncomfortable, and I've wondered if that technique didn't play a role in the problem that you're seeing.

Reply to this email directly or view it on GitHub:
#9 (comment)

[Vestrit]

I've mailed you a sample project that reproduces the crash when compiled + run.

[johnezang]

This may be a duplicate of issue #15.

[johnezang]

Fixed in commit c2ef692.