Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Crash in CBLBatcher #187

Closed
nl opened this issue Nov 9, 2013 · 8 comments
Closed

Crash in CBLBatcher #187

nl opened this issue Nov 9, 2013 · 8 comments
Labels
f: Replication
Milestone
Beta Two

Comments

@nl
Copy link

nl commented Nov 9, 2013 

(lldb) bt
* thread #12: tid = 0x188c9c, 0x38a6cb22 libobjc.A.dylib`objc_msgSend + 2, name = 'CouchbaseLite, stop reason = EXC_BAD_ACCESS (code=1, address=0x3000000)
    frame #0: 0x38a6cb22 libobjc.A.dylib`objc_msgSend + 2
    frame #1: 0x00194db2 storyplayr`-[CBLBatcher flushAll](self=0x15f4fbf0, _cmd=<unavailable>) + 142 at CBLBatcher.m:118
    frame #2: 0x00189ff2 storyplayr`-[CBL_Puller stop](self=<unavailable>, _cmd=<unavailable>) + 290 at CBL_Puller.m:149
    frame #3: 0x2ed4e03a Foundation`__NSThreadPerformPerform + 386
    frame #4: 0x2e33618a CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 14
    frame #5: 0x2e33565a CoreFoundation`__CFRunLoopDoSources0 + 206
    frame #6: 0x2e333e4e CoreFoundation`__CFRunLoopRun + 622
    frame #7: 0x2e29ece6 CoreFoundation`CFRunLoopRunSpecific + 522
    frame #8: 0x2e29eaca CoreFoundation`CFRunLoopRunInMode + 106
    frame #9: 0x2ec8b57a Foundation`-[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 254
    frame #10: 0x001847ee storyplayr`-[CBL_Server runServerThread](self=0x15d6df70, _cmd=<unavailable>) + 310 at CBL_Server.m:109
    frame #11: 0x2ed4de26 Foundation`__NSThread__main__ + 1062
    frame #12: 0x39098c1c libsystem_pthread.dylib`_pthread_body + 140
    frame #13: 0x39098b8e libsystem_pthread.dylib`_pthread_start + 102
    frame #14: 0x39096c90 libsystem_pthread.dylib`thread_start + 8
@snej
Copy link
Contributor

snej commented Nov 10, 2013

I got a nearly-identical crash report via email from JV this morning, too.

Looks like in the flushAll method, _inbox is a bad pointer during the test at the top of the while loop. I think self got dealloced, because I don't see a way for _inbox itself to become invalid while the batcher is alive.

@snej
Copy link
Contributor

snej commented Nov 10, 2013

Also, note to self: The batcher here is CBLPuller's _downloadsToInsert, not _batcher. This gets released in the -stopped method, so perhaps that's being called during the -insertDownloads call that -flushAll invokes.

@nl
Copy link
Author

nl commented Nov 11, 2013

For what it's worth, it happened when I stopped a quite large, non-continuous, non-persistent pull replication soon after starting it (~2-3 seconds). But ironically enough, I can't replicate it.

@PiranhaTodd
Copy link

I'm seeing this one also. Well, it came in as a Crashlytics crash report from one of our pre-release users. Didn't actually see it myself. Here's a copy/paste from the Crashlytics report:

Fatal Exception NSInvalidArgumentException
-[__NSCFString count]: unrecognized selector sent to instance 0x17d453b0
0
CoreFoundation
__exceptionPreprocess + 130
1
libobjc.A.dylib
objc_exception_throw + 38
2
CoreFoundation
-[NSObject(NSObject) doesNotRecognizeSelector:] + 202
3
CoreFoundation
___forwarding___ + 706
4
CoreFoundation
_CF_forwarding_prep_0 + 24
5
Hammersmith
CBLBatcher.m line 118
-[CBLBatcher flushAll]
6
Hammersmith
CBL_Puller.m line 149
-[CBL_Puller stop]
7
Hammersmith
CBL_Replicator.m line 127
-[CBL_Replicator databaseClosing]
8
Hammersmith
CBLDatabase+Internal.m line 454
-[CBLDatabase(Internal) closeInternal]
9
Hammersmith
CBLDatabase.m line 172
-[CBLDatabase close]
10
Hammersmith
CBLDatabase.m line 186
-[CBLDatabase closeForDeletion]
11
Foundation
__NSThreadPerformPerform + 386
16
CoreFoundation
CFRunLoopRunInMode + 106
17
Foundation
-[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 254
18
Hammersmith
CBL_Server.m line 109
-[CBL_Server runServerThread]
19
Foundation
__NSThread__main__ + 1062
22
libsystem_pthread.dylib
thread_start + 8
17 Threads
com.apple.main-thread
0
libsystem_kernel.dylib
__semwait_signal + 24
2
libsystem_c.dylib
usleep + 52
3
Hammersmith
CBL_Shared.m line 106
-[CBL_Shared forgetDatabaseNamed:]
4
Hammersmith
CBLManager.m line 558
-[CBLManager(Internal) _forgetDatabase:]
5
Hammersmith
CBLDatabase.m line 187
-[CBLDatabase closeForDeletion]
6
Hammersmith
CBLDatabase.m line 196
-[CBLDatabase deleteDatabase:]
7
Hammersmith
ProjectManager.mm line 795
-[DMProjectManager updateLocalProjects]
8
Hammersmith
ProjectManager.mm line 1725
-[DMProjectManager uniqueTitleForCopyOf:]
9
Hammersmith
ProjectManager.mm line 1595
-[DMProjectManager cloneDocumentWithUUID:]
10
Hammersmith
ProjectManager.mm line 1862
-[DMProjectManager cloneDatabase:]
11
Hammersmith
HSProjectBrowserViewController.mm line 358
-[HSProjectBrowserViewController cloneAndOpenTemplate:]
12
Hammersmith
HSProjectSelectionViewController.mm line 695
-[HSProjectSelectionView getStartedButtonTapped:]
13
UIKit
-[UIApplication sendAction:to:from:forEvent:] + 90
14
UIKit
-[UIApplication sendAction:toTarget:fromSender:forEvent:] + 38
15
UIKit
-[UIControl sendAction:to:forEvent:] + 46
16
UIKit
-[UIControl _sendActionsForEvents:withEvent:] + 374
17
UIKit
-[UIControl touchesEnded:withEvent:] + 594
18
UIKit
-[UIWindow _sendTouchesForEvent:] + 528
19
UIKit
-[UIWindow sendEvent:] + 832
20
UIKit
-[UIApplication sendEvent:] + 196
21
UIKit
_UIApplicationHandleEventQueue + 7116
22
CoreFoundation
__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 14
28
UIKit
UIApplicationMain + 1136
29
Hammersmith
main.m line 14
main
com.apple.libdispatch-manager
0
libsystem_kernel.dylib
kevent64 + 24
2
libdispatch.dylib
_dispatch_mgr_thread + 38
com.apple.root.default-overcommit-priority
0
libsystem_kernel.dylib
__workq_kernreturn + 8
1
libsystem_pthread.dylib
_pthread_wqthread + 310
com.apple.root.default-overcommit-priority
0
libsystem_kernel.dylib
__workq_kernreturn + 8
1
libsystem_pthread.dylib
_pthread_wqthread + 310
com.apple.root.default-overcommit-priority
0
libsystem_kernel.dylib
__workq_kernreturn + 8
1
libsystem_pthread.dylib
_pthread_wqthread + 310
com.apple.root.default-overcommit-priority
0
libsystem_kernel.dylib
__workq_kernreturn + 8
1
libsystem_pthread.dylib
_pthread_wqthread + 310
com.crashlytics.MachExceptionServer
0
libsystem_kernel.dylib
mach_msg_trap + 20
1
libsystem_kernel.dylib
mach_msg + 40
2
Hammersmith
CLSMachExceptionServer + 120
3
libsystem_pthread.dylib
_pthread_body + 140
4
libsystem_pthread.dylib
_pthread_start + 102
com.apple.root.default-overcommit-priority
0
libsystem_kernel.dylib
__workq_kernreturn + 8
1
libsystem_pthread.dylib
_pthread_wqthread + 310
com.apple.root.default-overcommit-priority
0
libsystem_kernel.dylib
__workq_kernreturn + 8
1
libsystem_pthread.dylib
_pthread_wqthread + 310
com.apple.NSURLConnectionLoader
0
libsystem_kernel.dylib
mach_msg_trap + 20
5
CoreFoundation
CFRunLoopRunInMode + 106
6
Foundation
+[NSURLConnection(Loader) _resourceLoadLoop:] + 320
7
Foundation
__NSThread__main__ + 1062
9
libsystem_pthread.dylib
_pthread_start + 102
com.apple.CFSocket.private
0
libsystem_kernel.dylib
__select + 20
3
libsystem_pthread.dylib
_pthread_start + 102
com.apple.root.default-overcommit-priority
0
libsystem_kernel.dylib
__workq_kernreturn + 8
1
libsystem_pthread.dylib
_pthread_wqthread + 310
com.apple.root.default-overcommit-priority
0
libsystem_kernel.dylib
mach_msg_trap + 20
5
CoreFoundation
CFRunLoopRunInMode + 106
6
Foundation
-[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 254
7
Foundation
-[NSRunLoop(NSRunLoop) run] + 80
8
Hammersmith
+[TFURLConnectionOperation _runNetworkThread:] + 212
9
Foundation
__NSThread__main__ + 1062
11
libsystem_pthread.dylib
_pthread_start + 102
com.apple.root.default-overcommit-priority
0
libsystem_kernel.dylib
__workq_kernreturn + 8
1
libsystem_pthread.dylib
_pthread_wqthread + 310
WebThread
0
libsystem_kernel.dylib
mach_msg_trap + 20
5
CoreFoundation
CFRunLoopRunInMode + 106
6
WebCore
RunWebThread(void*) + 420
7
libsystem_pthread.dylib
_pthread_body + 140
8
libsystem_pthread.dylib
_pthread_start + 102
com.apple.root.default-overcommit-priority
0
libsystem_kernel.dylib
mach_msg_trap + 20
9
libsystem_pthread.dylib
_pthread_start + 102
CouchbaseLite Crashed
0
libsystem_kernel.dylib
__pthread_kill + 8
10
CoreFoundation
CFRunLoopRunInMode + 106
11
Foundation
-[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 254
12
Hammersmith
CBL_Server.m line 109
-[CBL_Server runServerThread]
13
Foundation
__NSThread__main__ + 1062
15
libsystem_pthread.dylib
_pthread_start + 102

@snej snej mentioned this issue Nov 11, 2013
Closed
@PiranhaTodd
Copy link

Still seeing this issue.

* thread #15: tid = 0xd37f2, 0x3b163b22 libobjc.A.dylib`objc_msgSend + 2, name = 'CouchbaseLite, stop reason = EXC_BAD_ACCESS (code=1, address=0x1)
    frame #0: 0x3b163b22 libobjc.A.dylib`objc_msgSend + 2
    frame #1: 0x00ae4ff4 Hammersmith`-[CBLBatcher flushAll](self=0x145f08d0, _cmd=0x00caae2f) + 60 at CBLBatcher.m:118
    frame #2: 0x00ad1e6e Hammersmith`-[CBL_Puller stop](self=0x147733f0, _cmd=0x337ae5c0) + 526 at CBL_Puller.m:150
    frame #3: 0x00acaae6 Hammersmith`-[CBL_Replicator databaseClosing](self=0x147733f0, _cmd=0x00ca9a0e) + 78 at CBL_Replicator.m:127
    frame #4: 0x00aa5430 Hammersmith`-[CBLDatabase(self=0x145f5690, _cmd=0x00ca9fff) closeInternal] + 1312 at CBLDatabase+Internal.m:453
    frame #5: 0x00b1813e Hammersmith`-[CBLDatabase close](self=0x145f5690, _cmd=0x3376e9e8) + 90 at CBLDatabase.m:172
    frame #6: 0x00b18378 Hammersmith`-[CBLDatabase closeForDeletion](self=0x145f5690, _cmd=0x00ca9ad7) + 464 at CBLDatabase.m:186
    frame #7: 0x00aa7346 Hammersmith`__35-[CBLDatabase(.block_descriptor=<unavailable>) dbChanged:]_block_invoke309 + 118 at CBLDatabase+Internal.m:656
    frame #8: 0x00b430d0 Hammersmith`-[NSObject(self=0x1585e3f0, _cmd=0x00caf189) my_run_as_block] + 24 at MYBlockUtils.m:22
    frame #9: 0x313b5e9a Foundation`__NSThreadPerformPerform + 386
    frame #10: 0x3099d182 CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 14
    frame #11: 0x3099c652 CoreFoundation`__CFRunLoopDoSources0 + 206
    frame #12: 0x3099ae46 CoreFoundation`__CFRunLoopRun + 622
    frame #13: 0x30905c26 CoreFoundation`CFRunLoopRunSpecific + 522
    frame #14: 0x30905a0a CoreFoundation`CFRunLoopRunInMode + 106
    frame #15: 0x312f33da Foundation`-[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 254
    frame #16: 0x00ac4716 Hammersmith`-[CBL_Server runServerThread](self=0x146eaea0, _cmd=0x00caab44) + 530 at CBL_Server.m:109
    frame #17: 0x313b5c86 Foundation`__NSThread__main__ + 1062
    frame #18: 0x3b78fc1c libsystem_pthread.dylib`_pthread_body + 140
    frame #19: 0x3b78fb8e libsystem_pthread.dylib`_pthread_start + 102

@PiranhaTodd
Copy link

Here's the relevant log:
https://gist.github.com/PiranhaTodd/b851b158c36c2cc0d509
(haven't used gists much, so if you can't access, let me know)

We have an active pull replication started, and then start another one. On occasion, this fails, but not 100%. It's often enough that it's a problem, though. :)

@snej snej mentioned this issue Nov 14, 2013
Closed
snej added a commit that referenced this issue Nov 14, 2013
@snej
Fix problems stopping a replicator
b834d56 
The -stop method can end up being called re-entrantly, and it wasn’t
expecting that, so Bad Stuff could happen.

Also made the CBLReplicator’s -stop method simply clear the batcher
instead of flushing it, since any queued changes in it shouldn’t be
processed anyway (they’d result in remote requests that would just
immediately be stopped.)

For #187, #191, #193
@PiranhaTodd
Copy link

Looks fixed in commit b834d56 "Fix problems stopping a replicator". Thanks!

@snej
Copy link
Contributor

snej commented Nov 14, 2013

Whew!

@snej snej closed this as completed Nov 14, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
f: Replication
Projects
None yet
Milestone
Beta Two
Development

No branches or pull requests
3 participants
@nl @snej @PiranhaTodd