Crash pulling doc with unparseable JSON body #271

Closed
jnordberg opened this Issue Mar 6, 2014 · 19 comments

Comments

Projects
None yet
5 participants

This document crashes my iOS app every time it tries to sync. According to jsonlint.com it is perfectly valid JSON, and the sync gateway does not seem to have any problems with it.

Using 1.0-beta2

2014-03-06 14:43:00.380 Clarity[93923:3207] *** ASSERTION FAILED: !_docReader
2014-03-06 14:43:00.380 Clarity[93923:3207] *** Assertion failure in -[CBLBulkDownloader startedPart:](), /Users/jnordberg/Development/couchbase-lite-ios/Source/CBLBulkDownloader.m:137
2014-03-06 14:43:00.382 Clarity[93923:3207] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: '!_docReader'
*** First throw call stack:
(
    0   CoreFoundation                      0x0000000101edf795 __exceptionPreprocess + 165
    1   libobjc.A.dylib                     0x0000000101c42991 objc_exception_throw + 43
    2   CoreFoundation                      0x0000000101edf61a +[NSException raise:format:arguments:] + 106
    3   Foundation                          0x0000000101896707 -[NSAssertionHandler handleFailureInFunction:file:lineNumber:description:] + 169
    4   Clarity                             0x0000000100057aa8 _AssertFailed + 385
    5   Clarity                             0x00000001000525f6 -[CBLBulkDownloader appendToPart:] + 0
    6   Clarity                             0x000000010002e196 -[CBLMultipartReader appendData:] + 914
    7   Clarity                             0x000000010005236d -[CBLBulkDownloader connection:didReceiveData:] + 132
    8   Foundation                          0x0000000101933e9b __65-[NSURLConnectionInternal _withConnectionAndDelegate:onlyActive:]_block_invoke + 48
    9   Foundation                          0x00000001017e76ab -[NSURLConnectionInternal _withConnectionAndDelegate:onlyActive:] + 210
    10  Foundation                          0x00000001017e75bc -[NSURLConnectionInternal _withActiveConnectionAndDelegate:] + 69
    11  Foundation                          0x00000001017e7907 _NSURLConnectionDidReceiveData + 79
    12  CFNetwork                           0x0000000100359520 ___ZN27URLConnectionClient_Classic29_delegate_didReceiveDataArrayEv_block_invoke + 302
    13  CFNetwork                           0x0000000100358942 ___ZN27URLConnectionClient_Classic18_withDelegateAsyncEPKcU13block_pointerFvP16_CFURLConnectionPK33CFURLConnectionClientCurrent_VMaxE_block_invoke_2 + 84
    14  CFNetwork                           0x000000010039bf74 ___ZNK17CoreSchedulingSet13_performAsyncEPKcU13block_pointerFvvE_block_invoke + 25
    15  CoreFoundation                      0x0000000101e86114 CFArrayApplyFunction + 68
    16  CFNetwork                           0x00000001002cbbeb _ZN19RunloopBlockContext7performEv + 115
    17  CFNetwork                           0x00000001002cba31 _ZN17MultiplexerSource7performEv + 247
    18  CFNetwork                           0x00000001002cb854 _ZN17MultiplexerSource8_performEPv + 72
    19  CoreFoundation                      0x0000000101e6eec1 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17
    20  CoreFoundation                      0x0000000101e6e792 __CFRunLoopDoSources0 + 242
    21  CoreFoundation                      0x0000000101e8a61f __CFRunLoopRun + 767
    22  CoreFoundation                      0x0000000101e89f33 CFRunLoopRunSpecific + 467
    23  Foundation                          0x000000010184fb1e -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 253
    24  Clarity                             0x000000010001026f -[CBL_Server runServerThread] + 307
    25  Foundation                          0x000000010184ae0f __NSThread__main__ + 1167
    26  libsystem_pthread.dylib             0x000000010264e899 _pthread_body + 138
    27  libsystem_pthread.dylib             0x000000010264e72a _pthread_struct_init + 0
    28  libsystem_pthread.dylib             0x0000000102652fc9 thread_start + 13
)
libc++abi.dylib: terminating with uncaught exception of type NSException

Looks like it might be scientific notation that is killing it, {"something": 3.0872572109458717e+307}

Member

snej commented Mar 6, 2014

I'm sorry, but this document is too silly.
(read in best Graham Chapman voice)

Member

snej commented Mar 6, 2014

I don't think it's the JSON itself. It looks like the state machine in the multipart reader is getting messed up somehow. Do you have a database URL I could try pulling from, to reproduce this? Or could you send me a database file? (jens at couchbase)

Member

snej commented Mar 6, 2014

No, it is the JSON; NSJSONSerialization returns an error "The data couldn’t be read because it isn’t in the correct format." (Number wound up as NaN around character 1231.)" So this doc is an edge case, being syntactically valid JSON but not parseable to an object tree. I'd guess that almost any platform would have trouble with that number since it isn't representable in IEEE 64-bit floating point. They might not report it as an error when parsing, but if so the number would probably have turned into a NaN in the parsed form and would be written back out incorrectly.

Member

snej commented Mar 6, 2014

I put the document into a server database and tried pulling it with LiteServ, but it didn't crash. I know there have been some changes to the multipart handling code since beta 2, so it may be more robust now. If you can, could you try building the latest Couchbase Lite and seeing if your crash still occurs?

😄

I can't get the pull replication to work using the latest build, iv'e been meaning to create a issue for it but since 1.0-beta2 works fine i've just kept using that...

When i patched my random document generator to not generate numbers with scientific notation everything works fine.

Member

snej commented Mar 7, 2014

Hm, please file an issue about replication on master — we want to release a beta 3 next week and it would be really good to fix anything that's making replication not work.

Member

snej commented Mar 7, 2014

FYI, scientific notation is fine, just lower the maximum exponent. (Not sure what the limit of a 64-bit float is, but you can look it up.)

Managed to reproduce it with the latest build. The numbers i'm generating are between Number.MIN_VALUE and Number.MAX_VALUE so I think they should fit in a 64bit float.

{
    "_id": "5Vab6S4PJ6bYElZy7hljRFQFFl7q",
    "_rev": "1-de873a6506454d4d638bbaad3e1c780b",
    "_revisions": {
        "ids": [
            "de873a6506454d4d638bbaad3e1c780b"
        ],
        "start": 1
    },
    "chesterfield": "Fizzle-zebra dong wombat minnow",
    "egad": {
        "ducttape": "Cake."
    },
    "knickersbevy": [
        3.29092141767461e+307
    ],
    "members": [
        "8177"
    ],
    "owner": "204882",
    "pistachio": -78373
}
13:20:58.323‖ �WARNING*** : CBLMultipartDocumentReader[_id="(null)"]: received unparseable JSON data '{"_id":"5Vab6S4PJ6bYElZy7hljRFQFFl7q","_rev":"1-de873a6506454d4d638bbaad3e1c780b","_revisions":{"ids":["de873a6506454d4d638bbaad3e1c780b"],"start":1},"chesterfield":"Fizzle-zebra dong wombat minnow","egad":{"ducttape":"Cake."},"knickersbevy":[3.29092141767461e+307],"members":["8177"],"owner":"204882","pistachio":-78373}'
2014-03-07 13:20:58.324 Clarity[64861:3207] *** ASSERTION FAILED: !_docReader
2014-03-07 13:20:58.324 Clarity[64861:3207] *** Assertion failure in -[CBLBulkDownloader startedPart:](), /Users/jnordberg/Development/couchbase-lite-ios/Source/CBLBulkDownloader.m:138
2014-03-07 13:20:58.398 Clarity[64861:3207] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: '!_docReader'
*** First throw call stack:
(
    0   CoreFoundation                      0x01eb75e4 __exceptionPreprocess + 180
    1   libobjc.A.dylib                     0x01c3a8b6 objc_exception_throw + 44
    2   CoreFoundation                      0x01eb7448 +[NSException raise:format:arguments:] + 136
    3   Foundation                          0x018e7720 -[NSAssertionHandler handleFailureInFunction:file:lineNumber:description:] + 101
    4   Clarity                             0x00063a5c _AssertFailed + 341
    5   Clarity                             0x0005dd6f -[CBLBulkDownloader appendToPart:] + 0
    6   Clarity                             0x00038132 -[CBLMultipartReader appendData:] + 1086
    7   Clarity                             0x0005dac9 -[CBLBulkDownloader connection:didReceiveData:] + 136
    8   Foundation                          0x01a04b90 ___NSURLConnectionDidReceiveData_block_invoke + 50
    9   Foundation                          0x0199b3e1 __65-[NSURLConnectionInternal _withConnectionAndDelegate:onlyActive:]_block_invoke + 62
    10  Foundation                          0x01823a6e -[NSURLConnectionInternalConnection invokeForDelegate:] + 119
    11  Foundation                          0x018239d6 -[NSURLConnectionInternal _withConnectionAndDelegate:onlyActive:] + 208
    12  Foundation                          0x018238e8 -[NSURLConnectionInternal _withActiveConnectionAndDelegate:] + 76
    13  Foundation                          0x01823c68 _NSURLConnectionDidReceiveData + 87
    14  CFNetwork                           0x003ec941 ___ZN27URLConnectionClient_Classic29_delegate_didReceiveDataArrayEv_block_invoke + 304
    15  CFNetwork                           0x003ebbac ___ZN27URLConnectionClient_Classic18_withDelegateAsyncEPKcU13block_pointerFvP16_CFURLConnectionPK33CFURLConnectionClientCurrent_VMaxE_block_invoke_2 + 104
    16  CFNetwork                           0x004360ef ___ZNK17CoreSchedulingSet13_performAsyncEPKcU13block_pointerFvvE_block_invoke + 25
    17  CoreFoundation                      0x01e58d59 CFArrayApplyFunction + 57
    18  CFNetwork                           0x003555af _ZN19RunloopBlockContext7performEv + 139
    19  CFNetwork                           0x004366d2 _ZThn16_N19RunloopBlockContext24multiplexerClientPerformEv + 20
    20  CFNetwork                           0x003553dd _ZN17MultiplexerSource7performEv + 299
    21  CFNetwork                           0x003551f2 _ZN17MultiplexerSource8_performEPv + 76
    22  CoreFoundation                      0x01e4083f __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 15
    23  CoreFoundation                      0x01e401cb __CFRunLoopDoSources0 + 235
    24  CoreFoundation                      0x01e5d29e __CFRunLoopRun + 910
    25  CoreFoundation                      0x01e5cac3 CFRunLoopRunSpecific + 467
    26  CoreFoundation                      0x01e5c8db CFRunLoopRunInMode + 123
    27  Foundation                          0x018979c5 -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 284
    28  Clarity                             0x0001615b -[CBL_Server runServerThread] + 349
    29  Foundation                          0x01892597 -[NSThread main] + 76
    30  Foundation                          0x018924f6 __NSThread__main__ + 1275
    31  libsystem_pthread.dylib             0x025d65fb _pthread_body + 144
    32  libsystem_pthread.dylib             0x025d6485 _pthread_struct_init + 0
    33  libsystem_pthread.dylib             0x025dbcf2 thread_start + 34
)
libc++abi.dylib: terminating with uncaught exception of type NSException
Member

snej commented Mar 7, 2014

I've verified that NSJSONSerialization returns an error when parsing any numeric literal greater than about 1.0e+147. Which is a bug since the range of the 'double' type goes up to about 3e+307. I filed a bug with Apple (rdar://16260573).

But this shouldn't cause the multipart parser to crash. And I couldn't get it to crash given such a document. Any chance you could send me a database, or give me HTTP access to one?

@snej snej added P3: major and removed question labels Mar 10, 2014

@snej snej added this to the G.A. milestone Mar 10, 2014

@snej snej self-assigned this Mar 10, 2014

Member

snej commented Mar 11, 2014

Just noticed that this is the same exception as in #232. These are likely the same bug.

@snej snej closed this in 24f6381 Mar 12, 2014

snej added a commit that referenced this issue Mar 24, 2014

Fixed another assertion-failure when a _bulk_get download can’t parse…
… a JSON doc

Found a piece that was missing from the prior fix 24f6381. The same assertion failure was occurring,
under slightly different circumstances.
Fixes #288 (speculatively). Also see #271, #232.

snej added a commit that referenced this issue Mar 24, 2014

Fixed another assertion-failure when a _bulk_get download can’t parse…
… a JSON doc

Found a piece that was missing from the prior fix 24f6381. The same assertion failure was occurring,
under slightly different circumstances.
Fixes #288 (speculatively). Also see #271, #232.

evsukov89 pushed a commit to evsukov89/couchbase-lite-ios that referenced this issue Aug 27, 2014

evsukov89 pushed a commit to evsukov89/couchbase-lite-ios that referenced this issue Aug 27, 2014

Fixed another assertion-failure when a _bulk_get download can’t parse…
… a JSON doc

Found a piece that was missing from the prior fix 24f6381. The same assertion failure was occurring,
under slightly different circumstances.
Fixes #288 (speculatively). Also see #271, #232.

maelp commented Dec 23, 2014

I still have the same bug -- when I put a document in Couchbaselite with a small value, it stores it as:

1.977020072466465e-160

and when it tries to load it back from the store, I get:

WARNING: CBL_Body: couldn't parse JSON:
...valid JSON...
(error=Error Domain=NSCocoaErrorDomain Code=3840 "The operation couldn’t be completed.
(Cocoa error 3840.)" (Number wound up as NaN around character 918.)
UserInfo=0x170c608c0 {NSDebugDescription=Number wound up as NaN around character 918.})

Where character 918 corresponds to the value above

Member

snej commented Dec 23, 2014

@maelp: It's a bug in Apple's JSON parser. I suggest working around it by detecting the value and replacing it with 0 before saving the document.

maelp commented Dec 23, 2014

Alright! Perhaps this should be documented in a FAQ, not sure if it was?

The problem is my object can contain numbers at various location, so I should probably write a general-purpose helper which creates a min or detect an almost null value & replace it with 0

How do I know what range of values is considered valid by the Apple JSON parser?

Member

snej commented Dec 23, 2014

Well, you're only the second person ever to run into it, and you found this issue which can serve as an FAQ. (I posted earlier that it's an Apple bug.)

How do I know what range of values is considered valid by the Apple JSON parser?

I don't know. Experiment?

maelp commented Dec 23, 2014

Hey just to make it clear I wasn't complaining :) just asking if you knew the specifics of the bug since you seem to know where it came from

snej - Did you get an update from Apple on rdar://16260573 ?

I'm running into the same issue parsing some JSON for a different source.

Member

snej commented Apr 22, 2015

Let's see … it was closed as a duplicate of rdar://12229466, which is still open.

Tanyd31 commented Oct 26, 2016

failure in -CBLBulkDownloader startedPart:, /Users/jenkins/jenkins/workspace/build_cblite_ios_1031-enterprise/couchbase-lite-ios/Source/CBLBulkDownloader.m:138

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment