Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check index validity on the respective scan queue #2016

Closed
skurfer opened this issue Jan 16, 2015 · 7 comments
Closed

Check index validity on the respective scan queue #2016

skurfer opened this issue Jan 16, 2015 · 7 comments
Assignees

Comments

@skurfer
Copy link
Member

skurfer commented Jan 16, 2015

Got a hang the other day (which is rare for me). It looks like the main thread got stuck on indexIsValid for one of the entries.

The title of this issue might be too specific. Generally, we need to move more of the catalog related stuff out of the main thread. (There are many cases where checking for validity requires hitting the disk or network.)

@skurfer
Copy link
Member Author

skurfer commented Jan 16, 2015

For the curious:

Call graph:
    2618 Thread_6830359   DispatchQueue_1: com.apple.main-thread  (serial)
    + 2618 start  (in libdyld.dylib) + 1  [0x7fff898e75c9]
    +   2618 NSApplicationMain  (in AppKit) + 1832  [0x7fff865b52d4]
    +     2618 -[NSApplication run]  (in AppKit) + 594  [0x7fff865c9e23]
    +       2618 -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:]  (in AppKit) + 194  [0x7fff865d5e80]
    +         2618 _DPSNextEvent  (in AppKit) + 964  [0x7fff865d66d1]
    +           2618 _BlockUntilNextEventMatchingListInModeWithFilter  (in HIToolbox) + 71  [0x7fff8b96cffb]
    +             2618 ReceiveNextEventCommon  (in HIToolbox) + 179  [0x7fff8b96d0be]
    +               2618 RunCurrentEventLoopInMode  (in HIToolbox) + 235  [0x7fff8b96d43f]
    +                 2618 CFRunLoopRunSpecific  (in CoreFoundation) + 296  [0x7fff8c6b9838]
    +                   2618 __CFRunLoopRun  (in CoreFoundation) + 910  [0x7fff8c6b9e0e]
    +                     2618 __CFRunLoopDoBlocks  (in CoreFoundation) + 341  [0x7fff8c6ba655]
    +                       2618 __CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK__  (in CoreFoundation) + 12  [0x7fff8c6c854c]
    +                         2618 __44-[NSApplication _reactToScreenInvalidation:]_block_invoke  (in AppKit) + 75  [0x7fff86914c2d]
    +                           2618 -[NSApplication _reactToScreenInvalidationImmediately:]  (in AppKit) + 379  [0x7fff86914dee]
    +                             2618 -[NSNotificationCenter postNotificationName:object:userInfo:]  (in Foundation) + 66  [0x7fff93e95ea1]
    +                               2618 _CFXNotificationPost  (in CoreFoundation) + 3140  [0x7fff8c6581b4]
    +                                 2618 __CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER__  (in CoreFoundation) + 12  [0x7fff8c766cbc]
    +                                   2618 -[QSObjectSource invalidateSelf]  (in QSCore) + 104  [0x10c344ac8]
    +                                     2618 -[NSNotificationCenter postNotificationName:object:userInfo:]  (in Foundation) + 66  [0x7fff93e95ea1]
    +                                       2618 _CFXNotificationPost  (in CoreFoundation) + 3140  [0x7fff8c6581b4]
    +                                         2618 __CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER__  (in CoreFoundation) + 12  [0x7fff8c766cbc]
    +                                           2618 -[QSLibrarian reloadSource:]  (in QSCore) + 425  [0x10c329569]
    +                                             2618 -[QSCatalogEntry scanForced:]  (in QSCore) + 400  [0x10c31d38f]
    +                                               2618 -[QSCatalogEntry indexIsValid]  (in QSCore) + 187  [0x10c31cd47]
    +                                                 2618 _dispatch_barrier_sync_f_slow  (in libdispatch.dylib) + 597  [0x7fff8d1b2557]
    +                                                   2618 _os_semaphore_wait  (in libsystem_platform.dylib) + 16  [0x7fff8bc45c5b]
    +                                                     2618 semaphore_wait_trap  (in libsystem_kernel.dylib) + 10  [0x7fff8f96556a]
    2618 Thread_6830360   DispatchQueue_2: com.apple.libdispatch-manager  (serial)
    + 2618 _dispatch_mgr_thread  (in libdispatch.dylib) + 52  [0x7fff8d1aba6a]
    +   2618 kevent64  (in libsystem_kernel.dylib) + 10  [0x7fff8f96b22e]
    2618 Thread_6830490
    + 2618 thread_start  (in libsystem_pthread.dylib) + 13  [0x7fff883e44b1]
    +   2618 _pthread_start  (in libsystem_pthread.dylib) + 176  [0x7fff883e6279]
    +     2618 _pthread_body  (in libsystem_pthread.dylib) + 131  [0x7fff883e62fc]
    +       2618 __NSThread__main__  (in Foundation) + 1345  [0x7fff93efbb7a]
    +         2618 kevent  (in libsystem_kernel.dylib) + 10  [0x7fff8f96b216]
    2618 Thread_6830516
    + 2618 thread_start  (in libsystem_pthread.dylib) + 13  [0x7fff883e44b1]
    +   2618 _pthread_start  (in libsystem_pthread.dylib) + 176  [0x7fff883e6279]
    +     2618 _pthread_body  (in libsystem_pthread.dylib) + 131  [0x7fff883e62fc]
    +       2618 _NSEventThread  (in AppKit) + 137  [0x7fff867397a7]
    +         2618 CFRunLoopRunSpecific  (in CoreFoundation) + 296  [0x7fff8c6b9838]
    +           2618 __CFRunLoopRun  (in CoreFoundation) + 1371  [0x7fff8c6b9fdb]
    +             2618 __CFRunLoopServiceMachPort  (in CoreFoundation) + 212  [0x7fff8c6bab14]
    +               2618 mach_msg  (in libsystem_kernel.dylib) + 55  [0x7fff8f96469f]
    +                 2618 mach_msg_trap  (in libsystem_kernel.dylib) + 10  [0x7fff8f96552e]
    2618 Thread_6830595: com.apple.NSURLConnectionLoader
    + 2618 thread_start  (in libsystem_pthread.dylib) + 13  [0x7fff883e44b1]
    +   2618 _pthread_start  (in libsystem_pthread.dylib) + 176  [0x7fff883e6279]
    +     2618 _pthread_body  (in libsystem_pthread.dylib) + 131  [0x7fff883e62fc]
    +       2618 __NSThread__main__  (in Foundation) + 1345  [0x7fff93efbb7a]
    +         2618 +[NSURLConnection(Loader) _resourceLoadLoop:]  (in CFNetwork) + 434  [0x7fff8c4e4d20]
    +           2618 CFRunLoopRunSpecific  (in CoreFoundation) + 296  [0x7fff8c6b9838]
    +             2618 __CFRunLoopRun  (in CoreFoundation) + 1371  [0x7fff8c6b9fdb]
    +               2618 __CFRunLoopServiceMachPort  (in CoreFoundation) + 212  [0x7fff8c6bab14]
    +                 2618 mach_msg  (in libsystem_kernel.dylib) + 55  [0x7fff8f96469f]
    +                   2618 mach_msg_trap  (in libsystem_kernel.dylib) + 10  [0x7fff8f96552e]
    2618 Thread_6830599: com.apple.CFSocket.private
    + 2618 thread_start  (in libsystem_pthread.dylib) + 13  [0x7fff883e44b1]
    +   2618 _pthread_start  (in libsystem_pthread.dylib) + 176  [0x7fff883e6279]
    +     2618 _pthread_body  (in libsystem_pthread.dylib) + 131  [0x7fff883e62fc]
    +       2618 __select  (in libsystem_kernel.dylib) + 10  [0x7fff8f96a3f6]
    2618 Thread_7234385   DispatchQueue_157: QSCatalogEntry scanQueue: QSPresetDisplays  (serial)
    + 2618 start_wqthread  (in libsystem_pthread.dylib) + 13  [0x7fff883e44a1]
    +   2618 _pthread_wqthread  (in libsystem_pthread.dylib) + 729  [0x7fff883e66cb]
    +     2618 _dispatch_worker_thread3  (in libdispatch.dylib) + 91  [0x7fff8d1b9fe4]
    +       2618 _dispatch_root_queue_drain  (in libdispatch.dylib) + 935  [0x7fff8d1ab88f]
    +         2618 _dispatch_client_callout  (in libdispatch.dylib) + 8  [0x7fff8d1a8c13]
    +           2618 _dispatch_call_block_and_release  (in libdispatch.dylib) + 12  [0x7fff8d1ad323]
    +             2618 __42-[QSLibrarian scanCatalogIgnoringIndexes:]_block_invoke  (in QSCore) + 341  [0x10c32c7a0]
    +               2618 -[QSCatalogEntry scanForced:]  (in QSCore) + 425  [0x10c31d3a8]
    +                 2618 -[QSCatalogEntry scanAndCache]  (in QSCore) + 243  [0x10c31d185]
    +                   2618 _dispatch_barrier_sync_f_invoke  (in libdispatch.dylib) + 57  [0x7fff8d1a9e5e]
    +                     2618 _dispatch_client_callout  (in libdispatch.dylib) + 8  [0x7fff8d1a8c13]
    +                       2618 __30-[QSCatalogEntry scanAndCache]_block_invoke  (in QSCore) + 316  [0x10c31e18b]
    +                         2618 -[NSNotificationCenter postNotificationName:object:userInfo:]  (in Foundation) + 66  [0x7fff93e95ea1]
    +                           2618 _CFXNotificationPost  (in CoreFoundation) + 3140  [0x7fff8c6581b4]
    +                             2618 __CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER__  (in CoreFoundation) + 12  [0x7fff8c766cbc]
    +                               2618 -[QSLibrarian updateScanTask:]  (in QSCore) + 632  [0x10c32a199]
    +                                 2618 -[QSTask setStatus:]  (in QSCore) + 153  [0x10c35b733]
    +                                   2618 _dispatch_barrier_sync_f_slow  (in libdispatch.dylib) + 597  [0x7fff8d1b2557]
    +                                     2618 _os_semaphore_wait  (in libsystem_platform.dylib) + 16  [0x7fff8bc45c5b]
    +                                       2618 semaphore_wait_trap  (in libsystem_kernel.dylib) + 10  [0x7fff8f96556a]
    2618 Thread_7236784
      2618 start_wqthread  (in libsystem_pthread.dylib) + 13  [0x7fff883e44a1]
        2618 __workq_kernreturn  (in libsystem_kernel.dylib) + 10  [0x7fff8f96a946]

Total number in stack (recursive counted multiple, when >=5):

Sort by top of stack, same collapsed (when >= 5):
        mach_msg_trap  (in libsystem_kernel.dylib)        5236
        semaphore_wait_trap  (in libsystem_kernel.dylib)        5236
        __select  (in libsystem_kernel.dylib)        2618
        __workq_kernreturn  (in libsystem_kernel.dylib)        2618
        kevent  (in libsystem_kernel.dylib)        2618
        kevent64  (in libsystem_kernel.dylib)        2618

@tiennou
Copy link
Member

tiennou commented Jul 26, 2016

That's fixed by task-cleanup :

    +                               2618 -[QSLibrarian updateScanTask:]  (in QSCore) + 632  [0x10c32a199]
    +                                 2618 -[QSTask setStatus:]  (in QSCore) + 153  [0x10c35b733]
    +                                   2618 _dispatch_barrier_sync_f_slow  (in libdispatch.dylib) + 597  [0x7fff8d1b2557]
    +                                     2618 _os_semaphore_wait  (in libsystem_platform.dylib) + 16  [0x7fff8bc45c5b]
    +                                       2618 semaphore_wait_trap  (in libsystem_kernel.dylib) + 10  [0x7fff8f96556a]

@skurfer
Copy link
Member Author

skurfer commented Jul 26, 2016

That's fixed by task-cleanup

OK. Let me know when I should take another look at that branch.

@tiennou
Copy link
Member

tiennou commented Aug 1, 2016

That just conflicted with my librarian branch (which is fine, easy fix). I'm just wondering about your reasoning... As it stand I would have made it return NO, because the index cannot be valid, since we're actually rescanning. But anyway, just my 2 cents I think...

@pjrobertson
Copy link
Member

The way I saw it: the reasoning was that: if you're in the middle of scanning, then you don't want to cause another scan, so you return YES for that.

Although now I think about it more: you're right. The correct thing is to return NO and then put a check somewhere else for isScanning, but in the end I don't know if it'd make much of a difference

@tiennou
Copy link
Member

tiennou commented Aug 2, 2016

Like in -scanAndCache 😁 ? Right now, what this does is it just sidestep checking the index moddate in a dispatch_sync, which doesn't matter if you say YES or NO anyway (that's why I wanted the reason for the change).

@skurfer
Copy link
Member Author

skurfer commented Aug 2, 2016

Technically, we don’t know where the scan is, so the real answer is “not sure”. But since it’s boolean, I picked YES because I’m used to thinking of that as “give up and go away” in this context.

I suppose NO would eventually yield the same result, but require more code to be called for no real reason. Right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants