[DNM][SourceKit] Log attempts to mangle a module instead of llvm_unreachable for SourceKit/LSP #28051

johnno1962 · 2019-11-04T15:09:11Z

I'm seeing SourceKit crashes using a form of LSP stress tester siteify which tokenises a source file using SourceKit then attempts to find the definition of all identifiers in a source file via LSP. A typical crash I'm seeing is:


Thread 4 Crashed:: Dispatch queue: sourcekit.swift.ConsumeAST
0   libsystem_kernel.dylib        	0x00007fff6bf892c6 __pthread_kill + 10
1   libsystem_pthread.dylib       	0x00007fff6c044bf1 pthread_kill + 284
2   libsystem_c.dylib             	0x00007fff6bef3745 __abort + 144
3   libsystem_c.dylib             	0x00007fff6bef36b5 abort + 142
4   com.apple.SourceKitService.5.0.20191024101_osx	0x0000000103f4b63e llvm::llvm_unreachable_internal(char const*, char const*, unsigned int) + 462
5   com.apple.SourceKitService.5.0.20191024101_osx	0x00000001020a47dc swift::Mangle::ASTMangler::appendType(swift::Type, swift::ValueDecl const*) + 7884 (ASTMangler.cpp:762)
6   com.apple.SourceKitService.5.0.20191024101_osx	0x00000001020a5279 swift::Mangle::ASTMangler::appendDeclType(swift::ValueDecl const*, bool) + 185 (ASTMangler.cpp:2416)
7   com.apple.SourceKitService.5.0.20191024101_osx	0x00000001020a514f swift::Mangle::ASTMangler::mangleDeclType(swift::ValueDecl const*) + 47 (ASTMangler.cpp:402)
8   com.apple.SourceKitService.5.0.20191024101_osx	0x000000010222ac4f swift::ide::printDeclTypeUSR(swift::ValueDecl const*, llvm::raw_ostream&) + 367 (USRGeneration.cpp:46)
9   com.apple.SourceKitService.5.0.20191024101_osx	0x000000010143acfa passCursorInfoForDecl(swift::SourceFile*, swift::ValueDecl const*, swift::ModuleDecl const*, swift::Type, bool, bool, swift::ide::ResolvedCursorInfo, llvm::Optional<unsigned int>, swift::SourceLoc, llvm::ArrayRef<SourceKit::RefactoringInfo>, SourceKit::SwiftLangSupport&, swift::CompilerInvocation const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, llvm::ArrayRef<llvm::IntrusiveRefCntPtr<SourceKit::ImmutableTextSnapshot> >, std::__1::function<void (SourceKit::RequestResult<SourceKit::CursorInfoData> const&)>) + 1642 (SwiftSourceDocInfo.cpp:793)
10  com.apple.SourceKitService.5.0.20191024101_osx	0x0000000101439254 resolveCursor(SourceKit::SwiftLangSupport&, llvm::StringRef, unsigned int, unsigned int, bool, llvm::IntrusiveRefCntPtr<SourceKit::SwiftInvocation>, bool, bool, llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>, std::__1::function<void (SourceKit::RequestResult<SourceKit::CursorInfoData> const&)>)::CursorInfoConsumer::handlePrimaryAST(llvm::IntrusiveRefCntPtr<SourceKit::ASTUnit>) + 4340 (SwiftSourceDocInfo.cpp:1352)
11  com.apple.SourceKitService.5.0.20191024101_osx	0x00000001013f5868 void SourceKit::WorkQueue::DispatchData::callAndDelete<SourceKit::ASTUnit::Implementation::consumeAsync(std::__1::shared_ptr<SourceKit::SwiftASTConsumer>, llvm::IntrusiveRefCntPtr<SourceKit::ASTUnit>)::$_0>(void*) + 88 (IntrusiveRefCntPtr.h:188)
12  libdispatch.dylib             	0x00007fff6be0163d _dispatch_client_callout + 8
13  libdispatch.dylib             	0x00007fff6be078e0 _dispatch_lane_serial_drain + 602
14  libdispatch.dylib             	0x00007fff6be08396 _dispatch_lane_invoke + 385
15  libdispatch.dylib             	0x00007fff6be106ed _dispatch_workloop_worker_thread + 598
16  libsystem_pthread.dylib       	0x00007fff6c041611 _pthread_wqthread + 421
17  libsystem_pthread.dylib       	0x00007fff6c0413fd start_wqthread + 13

Inserting some logging turns out this is where "siteify" tries to look up an identifier for a module reaching the llvm_unreachable that I've changed to an fprintf to standard error. This is the most common SourceKit crash processing the list of packages from https://github.com/daveverwer/SwiftPMLibrary

John

nathawes · 2019-11-05T00:32:02Z

Thanks for finding this! It seems like SourceKit should probably catch this before it gets to the mangling logic, but isn't for some reason. Could you point us to an example file + line in one of packages you mention where jump-to-definition is hitting this?

johnno1962 · 2019-11-05T12:35:06Z

Thanks for picking this up! It’s difficult to give you a line number and file due to the error arriving on stderr after the and the call has been made but if you want to replicate the problem, clone the project https://github.com/johnno1962/siteify, do a swift build then clone https://github.com/3qax/swiftyoled.git, swift build, and run the siteify/.build/debug/siteify binary in the second project directory. This should give a crash dump which is removed by the change in this PR.

There is a second, far rarer crash which affects the project https://github.com/Alamofire/Alamofire.git that I haven’t looked into yet which should give you the following crash when using siteify which seems to give you the line and column:

Process:               SourceKitService [67930]
Path:                  /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2019-11-01-a.xctoolchain/usr/lib/sourcekitd.framework/Versions/A/XPCServices/SourceKitService.xpc/Contents/MacOS/SourceKitService
Identifier:            SourceKitService
Version:               1.0 (5.0.20191101101)
Code Type:             X86-64 (Native)
Parent Process:        ??? [1]
Responsible:           sourcekit-lsp [67897]
User ID:               501

Date/Time:             2019-11-05 13:29:43.230 +0100
OS Version:            Mac OS X 10.14.5 (18F132)
Report Version:        12
Anonymous UUID:        D9F67A20-F003-AAE0-9411-8AA523D1366D


Time Awake Since Boot: 22000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: sourcekit.swift.ConsumeAST

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
dyld2 mode
Stack dump:
0.	While evaluating request USRGenerationRequest(ResponseTests.(file).TestError.error(error:)@/Volumes/Elements/pkgs/single3/Alamofire/Tests/ResponseTests.swift:529:10)
 
abort() called

Thread 0 Crashed:: Dispatch queue: sourcekit.swift.ConsumeAST
0   libsystem_kernel.dylib        	0x00007fff5d6582c6 __pthread_kill + 10
1   libsystem_pthread.dylib       	0x00007fff5d713bf1 pthread_kill + 284
2   libsystem_c.dylib             	0x00007fff5d5c2745 __abort + 144
3   libsystem_c.dylib             	0x00007fff5d5c26b5 abort + 142
4   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000109546fef swift::Mangle::Mangler::verify(llvm::StringRef) (.cold.2) + 63
5   com.apple.SourceKitService.5.0.20191101101_osx	0x00000001088d86bd swift::Mangle::Mangler::verify(llvm::StringRef) + 781
6   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000106af7a4b swift::Mangle::ASTMangler::mangleDeclAsUSR(swift::ValueDecl const*, llvm::StringRef) + 443 (ASTMangler.cpp:523)
7   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000106c8ac6d swift::USRGenerationRequest::evaluate(swift::Evaluator&, swift::ValueDecl const*) const + 1101 (USRGeneration.cpp:255)
8   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000106869bf1 swift::SimpleRequest<swift::USRGenerationRequest, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > (swift::ValueDecl const*), (swift::CacheKind)1>::evaluateRequest(swift::USRGenerationRequest const&, swift::Evaluator&) + 17 (SimpleRequest.h:219)
9   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000106c8d6dc llvm::Expected<swift::USRGenerationRequest::OutputType> swift::Evaluator::getResultUncached<swift::USRGenerationRequest>(swift::USRGenerationRequest const&) + 380 (Evaluator.h:381)
10  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000106c8d20f llvm::Expected<swift::USRGenerationRequest::OutputType> swift::Evaluator::getResultCached<swift::USRGenerationRequest, (void*)0>(swift::USRGenerationRequest const&) + 239 (Evaluator.h:423)
11  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000106c8c16b llvm::Expected<swift::USRGenerationRequest::OutputType> swift::Evaluator::operator()<swift::USRGenerationRequest>(swift::USRGenerationRequest const&) + 107 (SetVector.h:129)
12  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000106c8b482 swift::USRGenerationRequest::OutputType swift::evaluateOrDefault<swift::USRGenerationRequest>(swift::Evaluator&, swift::USRGenerationRequest, swift::USRGenerationRequest::OutputType) + 34 (Error.h:554)
13  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000106c8b3f0 swift::ide::printValueDeclUSR(swift::ValueDecl const*, llvm::raw_ostream&) + 96 (USRGeneration.cpp:280)
14  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000105e7aca8 passCursorInfoForDecl(swift::SourceFile*, swift::ValueDecl const*, swift::ModuleDecl const*, swift::Type, bool, bool, swift::ide::ResolvedCursorInfo, llvm::Optional<unsigned int>, swift::SourceLoc, llvm::ArrayRef<SourceKit::RefactoringInfo>, SourceKit::SwiftLangSupport&, swift::CompilerInvocation const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, llvm::ArrayRef<llvm::IntrusiveRefCntPtr<SourceKit::ImmutableTextSnapshot> >, std::__1::function<void (SourceKit::RequestResult<SourceKit::CursorInfoData> const&)>) + 744 (SwiftSourceDocInfo.cpp:773)
15  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000105e79584 resolveCursor(SourceKit::SwiftLangSupport&, llvm::StringRef, unsigned int, unsigned int, bool, llvm::IntrusiveRefCntPtr<SourceKit::SwiftInvocation>, bool, bool, llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>, std::__1::function<void (SourceKit::RequestResult<SourceKit::CursorInfoData> const&)>)::CursorInfoConsumer::handlePrimaryAST(llvm::IntrusiveRefCntPtr<SourceKit::ASTUnit>) + 4340 (SwiftSourceDocInfo.cpp:1352)
16  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000105e35b18 void SourceKit::WorkQueue::DispatchData::callAndDelete<SourceKit::ASTUnit::Implementation::consumeAsync(std::__1::shared_ptr<SourceKit::SwiftASTConsumer>, llvm::IntrusiveRefCntPtr<SourceKit::ASTUnit>)::$_0>(void*) + 88 (IntrusiveRefCntPtr.h:188)
17  libdispatch.dylib             	0x00007fff5d4d063d _dispatch_client_callout + 8
18  libdispatch.dylib             	0x00007fff5d4d68e0 _dispatch_lane_serial_drain + 602
19  libdispatch.dylib             	0x00007fff5d4d7396 _dispatch_lane_invoke + 385
20  libdispatch.dylib             	0x00007fff5d4df6ed _dispatch_workloop_worker_thread + 598
21  libsystem_pthread.dylib       	0x00007fff5d710611 _pthread_wqthread + 421
22  libsystem_pthread.dylib       	0x00007fff5d7103fd start_wqthread + 13

johnno1962 · 2019-11-05T15:26:50Z

If you want to stress test the crashes remaining after this fix you can use this script ("timed out" is where there is a clangd assertion failure or a genuine time out after 5 minutes):

#!/usr/bin/env ruby

packages = [
  "https://github.com/123flo321/pogoprotos-swift.git", # 6 timed out
  "https://github.com/adam-fowler/aws-signer.git", # 39 timed out
      "https://github.com/adam-fowler/dictionary-encoder.git", # 40
      "https://github.com/adam-fowler/xml-encoder.git", # 41
  "https://github.com/agisboye/swiftlmdb.git", # 53 timed out
  "https://github.com/akane/AnnotationInject.git", # 61 timed out
      "https://github.com/Alamofire/Alamofire.git", # 71
      "https://github.com/Alamofire/AlamofireImage.git", # 72
      "https://github.com/Alecrim/AlecrimCoreData.git", # 75
  "https://github.com/alexdrone/dispatchstore.git", # 88 timed out
  "https://github.com/alexdrone/Store.git", # 90 timed out
      "https://github.com/allaboutapps/Fetch.git", # 102
  "https://github.com/altavares/swiftyshell.git", # 110 timed out
  "https://github.com/AlwaysRightInstitute/SwiftyExpat.git", # 119 timed out
  "https://github.com/americanexpress/xcprojectlint.git", # 126 timed out
  "https://github.com/amzn/smoke-aws-generate.git", # 137 timed out
  "https://github.com/amzn/smoke-http.git", # 141 timed out
  "https://github.com/amzn/smoke-framework.git", # 140 timed out
  "https://github.com/ancestry/carttool.git", # 142 timed out
  "https://github.com/AndrewBarba/ed25519.git", # 146 timed out
      "https://github.com/angelolloqui/swiftkotlin.git", # 151
      "https://github.com/aphelionapps/swaggerparser.git", # 171
  "https://github.com/apleshkov/saber.git", # 172 timed out
  "https://github.com/apollographql/apollo-ios.git", # 175 timed out
  "https://github.com/apple/swift-driver.git", # 189 timed out
  "https://github.com/apple/swift-llbuild.git", # 190 timed out
  "https://github.com/apple/swift-nio-extras.git", # 193 timed out
  "https://github.com/apple/swift-nio-http2.git", # 194 timed out
  "https://github.com/apple/swift-nio-ssl.git", # 197 timed out
  "https://github.com/apple/swift-nio-transport-services.git", # 198 timed out
  "https://github.com/apple/swift-nio.git", # 200 timed out
  "https://github.com/apple/swift-package-manager.git", # 201 timed out
  "https://github.com/apple/swift-protobuf.git", # 202 timed out
  "https://github.com/apple/swift-syntax.git", # 203 timed out
      "https://github.com/apple/swift-tools-support-core.git", # 204
  "https://github.com/armadsen/ORSSerialPort.git", # 212 timed out
  "https://github.com/artemnovichkov/carting.git", # 217 timed out
      "https://github.com/ashleymills/Reachability.swift.git", # 230
      "https://github.com/AsyncNinja/AsyncNinja.git", # 234
  "https://github.com/bankex/web3swift.git", # 275 timed out
  "https://github.com/bannzai/ocha.git", # 278 timed out
  "https://github.com/bannzai/ragnarok.git", # 279 timed out
  "https://github.com/bannzai/teapot.git", # 282 timed out
  "https://github.com/bannzai/xcodeproject.git", # 283 timed out
  "https://github.com/barrault01/firebasehelpers.git", # 284 timed out
  "https://github.com/bersaelor/swiftybeagle.git", # 325 timed out
  "https://github.com/bloombox/Swift.git", # 354 timed out
  "https://github.com/Boilertalk/Keystore.swift.git", # 358 timed out
  "https://github.com/Boilertalk/Web3.swift.git", # 363 timed out
      "https://github.com/bouke/lark.git", # 374
  "https://github.com/bow-swift/bow.git", # 379 timed out
      "https://github.com/bradlarson/gpuimage2.git", # 384
  "https://github.com/brainfinance/stackdriverlogging.git", # 385 timed out
  "https://github.com/brian-schick/stainlesssteed.git", # 393 timed out
  "https://github.com/broadwaylamb/gottagofast.git", # 397 timed out
  "https://github.com/broadwaylamb/OpenCombine.git", # 398 timed out
  "https://github.com/brokenhandsio/SteamPress.git", # 402 timed out
  "https://github.com/brycesteve/perfect-view2pdf.git", # 409 timed out
      "https://github.com/bwetherfield/pitchspeller.git", # 414
  "https://github.com/carthage/carthage.git", # 431 timed out
      "https://github.com/cats-oss/cujira.git", # 433
      "https://github.com/cbguder/CBGPromise.git", # 439
  "https://github.com/e-sites/Natrium.git", # 762 timed out
      "https://github.com/eBardX/XestiMonitors.git", # 765
]

include Process
require 'timeout'

swift = "/Library/Developer/Toolchains/swift-latest.xctoolchain/usr/bin/swift"
processes = 2
running = 0
dir = 0

for package in packages do
    if running >= processes then
        wait()
        running -= 1
    end

    fork do
        begin
            Timeout::timeout(600) {
                system "git clone #{package} #{dir} && cd #{dir} && #{swift} build && ~/bin/siteify"
            }
        rescue Timeout::Error
            puts "**** timeout #{package}"
        end
        exit
    end

    running += 1
    dir += 1
end

while running > 0 do
    wait()
    running -= 1
end

johnno1962 · 2019-11-05T18:54:37Z

A little more information about the second rarer crash I mentioned above. It's probably best to use the project https://github.com/adam-fowler/dictionary-encoder.git which is smaller and reliably produces the second SourceKit crash. The AlamoFire project only reliably produces a crash with the master snapshot of the 1st November, not that of 24th October.

johnno1962 · 2019-11-11T17:39:18Z

Thanks @nathawes for properly resolving the most common crash I mentioned with your commit of Friday. I've been looking at the few remaining probelms and found two "solutions" of a sort. The First Type of crash is something along the lines of:

SourceKitService_2019-11-10-181007_Users-iMac.crash:0.	While walking into decl declaration 0x7ffed3a26e60 (at /Volumes/Elements/pkgs/single5/Alamofire/Source/NetworkReachabilityManager.swift:25:1)
SourceKitService_2019-11-10-181007_Users-iMac.crash:Assertion failed: ((hasBeenBound() || canNeverBeBound()) && "Extension must have already been bound (by bindExtensions)"), function getExtendedNominal, file /Volumes/Elements/swift-lsp/swift/lib/AST/Decl.cpp, line 1203.

#if !(os(watchOS) || os(Linux))

This is where "siteify" was looking up the identifier #if and this causes a SourceKit crash which is quite easily resolved at the client by checking for # at the start of an identifier. The second type of crash was along the lines of:

SourceKitService_2019-11-10-182121-1_Users-iMac.crash:0.	While walking into decl declaration 0x7fcf6182f378 (at /Volumes/Elements/pkgs/single5/XestiMonitors/Sources/Core/UIKit/Accessibility/AccessibilityStatusMonitor.swift:10:1)
SourceKitService_2019-11-10-182121-1_Users-iMac.crash:Assertion failed: (0 && "Attribute's TokenNodes already consumed?"), function handleSpecialDeclAttribute, file /Volumes/Elements/swift-lsp/swift/lib/IDE/SyntaxModel.cpp, line 1187.

#if os(iOS) || os(tvOS)

which I "resolved" by commenting out the assertion for now (see last commit on this branch).

I lost a bit of time because "siteify", sourcekit-lsp and perhaps all Cursor Info lookups in SourceKit seems to not be working correctly on master since this commit 4023199. If I take a branch before this commit things are working again.

If you have the time to look at them this leaves a few crashes that I've not been able to get to the bottom of but you can replicate easily using this more focused script + the siteify project:

#!/usr/bin/env ruby

packages = [
      "https://github.com/adam-fowler/dictionary-encoder.git", # 40
      "https://github.com/adam-fowler/xml-encoder.git", # 41
      "https://github.com/Alamofire/Alamofire.git", # 71
      "https://github.com/Alamofire/AlamofireImage.git", # 72
      "https://github.com/Alecrim/AlecrimCoreData.git", # 75
      "https://github.com/allaboutapps/Fetch.git", # 102
      "https://github.com/angelolloqui/swiftkotlin.git", # 151
      "https://github.com/aphelionapps/swaggerparser.git", # 171
      "https://github.com/apple/swift-tools-support-core.git", # 204
      "https://github.com/ashleymills/Reachability.swift.git", # 230
      "https://github.com/AsyncNinja/AsyncNinja.git", # 234
      "https://github.com/bouke/lark.git", # 374
      "https://github.com/bradlarson/gpuimage2.git", # 384
      "https://github.com/bwetherfield/pitchspeller.git", # 414
      "https://github.com/cats-oss/cujira.git", # 433
      "https://github.com/cbguder/CBGPromise.git", # 439
      "https://github.com/eBardX/XestiMonitors.git", # 765
]

include Process
require 'timeout'

swift = "/Library/Developer/Toolchains/swift-latest.xctoolchain/usr/bin/swift"
processes = 2
running = 0
dir = 0

for package in packages do
    if running >= processes then
        wait()
        running -= 1
    end

    fork do
        begin
            Timeout::timeout(600) {
                system "git clone #{package} && cd `ls -rt | grep -v .log | tail -1` && #{swift} build && ~/bin/siteify"
            }
        rescue Timeout::Error
            puts "**** timeout #{package}"
        end
        exit
    end

    running += 1
    dir += 1
end

while running > 0 do
    wait()
    running -= 1
end

I'm seeing:

SourceKitService_2019-11-11-173204_Users-iMac.crash:Assertion failed: (T->mayHaveMembers()), function TypeTransformContext, file /Volumes/Elements/swift-lsp/swift/lib/AST/ASTPrinter.cpp, line 206.
SourceKitService_2019-11-11-175052_Users-iMac.crash:0.	While walking into decl declaration 0x7f8b2f307ba8 (at /Volumes/Elements/pkgs/single7/AsyncNinja/Sources/appleOS_Observation.swift:23:1)
SourceKitService_2019-11-11-180004_Users-iMac.crash:Assertion failed: (T->mayHaveMembers()), function TypeTransformContext, file /Volumes/Elements/swift-lsp/swift/lib/AST/ASTPrinter.cpp, line 206.
SourceKitService_2019-11-11-180039_Users-iMac.crash:0.	While walking into decl declaration 0x7fdb23014da8 (at /Volumes/Elements/pkgs/single7/AsyncNinja/Sources/appleOS_Observation.swift:23:1)
onseTests.(file).TestError.error(error:)@/Volumes/Elements/pkgs/single7/Fetch/.build/checkouts/Alamofire/Tests/ResponseTests.swift:529:10)
SourceKitService_2019-11-11-175052_Users-iMac.crash:0.	While walking into decl declaration 0x7f8b2f307ba8 (at /Volumes/Elements/pkgs/single7/AsyncNinja/Sources/appleOS_Observation.swift:23:1)
SourceKitService_2019-11-11-175052_Users-iMac.crash:1.	While evaluating request StorageImplInfoRequest(AsyncNinja.(file).ObservationSessionItem.isEnabled@/Volumes/Elements/pkgs/single7/AsyncNinja/Sources/appleOS_Observation.swift:89:9)
SourceKitService_2019-11-11-175052_Users-iMac.crash:2.	While evaluating request IsObjCRequest(AsyncNinja.(file).ObservationSessionItem@/Volumes/Elements/pkgs/single7/AsyncNinja/Sources/appleOS_Observation.swift:88:18)
SourceKitService_2019-11-11-175052_Users-iMac.crash:3.	While evaluating request InheritedDeclsReferencedRequest(AsyncNinja.(file).ObservationSessionItem@/Volumes/Elements/pkgs/single7/AsyncNinja/Sources/appleOS_Observation.swift:88:18, 0)
SourceKitService_2019-11-11-180004_Users-iMac.crash:Assertion failed: (T->mayHaveMembers()), function TypeTransformContext, file /Volumes/Elements/swift-lsp/swift/lib/AST/ASTPrinter.cpp, line 206.
SourceKitService_2019-11-11-180039_Users-iMac.crash:0.	While walking into decl declaration 0x7fdb23014da8 (at /Volumes/Elements/pkgs/single7/AsyncNinja/Sources/appleOS_Observation.swift:23:1)
SourceKitService_2019-11-11-180039_Users-iMac.crash:1.	While evaluating request StorageImplInfoRequest(main.(file).ObservationSessionItem.isEnabled@/Volumes/Elements/pkgs/single7/AsyncNinja/Sources/appleOS_Observation.swift:89:9)
SourceKitService_2019-11-11-180039_Users-iMac.crash:2.	While evaluating request IsObjCRequest(main.(file).ObservationSessionItem@/Volumes/Elements/pkgs/single7/AsyncNinja/Sources/appleOS_Observation.swift:88:18)
SourceKitService_2019-11-11-180039_Users-iMac.crash:3.	While evaluating request InheritedDeclsReferencedRequest(main.(file).ObservationSessionItem@/Volumes/Elements/pkgs/single7/AsyncNinja/Sources/appleOS_Observation.swift:88:18, 0)
SourceKitService_2019-11-11-180053-1_Users-iMac.crash:0.	While evaluating request USRGenerationRequest(FilterOperationTypes.(file).FilterOperationType.custom(filterSetupFunction:)@/Volumes/Elements/pkgs/single7/gpuimage2/examples/Mac/FilterShowcase/FilterShowcase/FilterOperationTypes.swift:14:10)
SourceKitService_2019-11-11-180053_Users-iMac.crash:0.	While evaluating request USRGenerationRequest(Errors.(file).NetworkResult.Success@/Volumes/Elements/pkgs/single7/CBGPromise/Examples/Errors.swift:7:10)
SourceKitService_2019-11-11-180054_Users-iMac.crash:0.	While evaluating request USRGenerationRequest(enums.(file).SDKException.network@/Volumes/Elements/pkgs/single7/swiftkotlin/Assets/Tests/KotlinTokenizer/enums.swift:42:10)

This is after processing the first 765 swift packages in Dave's list. Thanks very much for your help!

benlangmuir · 2019-11-11T18:01:06Z

I lost a bit of time because "siteify", sourcekit-lsp and perhaps all Cursor Info lookups in SourceKit have not been working correctly on master since this commit 4023199. If I take a branch before this commit things are working again.

What is the problem you're seeing? I downloaded the master snapshot from Nov 8 which includes that commit and it appears to be working in general.

johnno1962 · 2019-11-11T18:16:28Z

If I build SourceKitService from master and copy it into the toolchain from Nov 1st, sourcekit-lsp requests for definition and references don't return anything and siteify does not link to definition or any USR references. If I build master from before the commit I mention with no other changes and apply nathan's fix things work fine.

benlangmuir · 2019-11-11T19:01:03Z

build SourceKitService from master and copy it into the toolchain from Nov 1st

Oh, ok. This could just be because SourceKitService is out of sync with the stdlib, which is not supported.

johnno1962 · 2019-11-11T19:04:51Z

Ah, I've download the toolchain for the 8th and am rebuilding master now. IndexDB format change?

benlangmuir · 2019-11-11T19:09:17Z

No, but any lock-step changes between the compiler and stdlib can cause trouble. It's not a configuration we can keep working in sourcekit since the compiler doesn't support it either.

johnno1962 · 2019-11-11T23:00:16Z

Up and running again. I downloaded the 11-08 toolchain which is compatible with master. From the look of the PR I mentioned it changed the module format which must have been the problem.

What I reported above is correct but right now I'm looking at two files which crash the SourceKit tokeniser 😱 (syntax map on file open) - project https://github.com/cats-oss/cujira.git, files Sources/Core/Common/Enumerable.swift and Sources/Core/Common/Extension/Gap.swift, if you use the toolchain in Xcode you can see the crash.

johnno1962 · 2019-11-12T13:51:20Z

I've commented out another assertion which "solves" the Extension must have already been bound (by bindExtensions) problem. I was wrong above when I said this was encountered when looking up definition of #if but was occurring during the syntaxMap phase which I wasn't expecting. The file I mentioned in the previous comment will replicate it and if you could find the time to look at this this it would be a great help. Interestingly this is not a problem using the Xcode 11.1 toolchain but a regression in master that seems to date back to this commit 4bc4030. Likewise the Attribute's TokenNodes already consumed crash relates to a fairly recent commit: 218f490

johnno1962 · 2019-11-19T18:35:46Z

I've added a commit commenting out the remaining assertions that where causing SourceKitService to crash. I can't speak to whether these were valid indications of an underlying problem or recent commits looking to catch problems early without realising the impact on SourceKit. In any case I've had a clean run of the 17 projects I mention in the second ruby script above.

Specifically (if you want to replicate the crash using siteify to get the stack trace):
The change lib/Basic/Mangler.cpp fixes a problem in https://github.com/Alamofire/Alamofire.git

Crashed Thread:        1  Dispatch queue: sourcekit.swift.ConsumeAST

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
dyld2 mode
Stack dump:
0.	While evaluating request USRGenerationRequest(ResponseTests.(file).TestError.error(error:)@/Volumes/Elements/pkgs/single7/Alamofire/Tests/ResponseTests.swift:529:10)
 
abort() called

Thread 0:: Dispatch queue: com.apple.root.default-qos.overcommit
0   libsystem_kernel.dylib        	0x00007fff5af29126 __sigsuspend_nocancel + 10
1   libdispatch.dylib             	0x00007fff5adb2ef3 _dispatch_sigsuspend + 21
2   libdispatch.dylib             	0x00007fff5adb2ede _dispatch_sig_thread + 53

Thread 1 Crashed:: Dispatch queue: sourcekit.swift.ConsumeAST
0   libsystem_kernel.dylib        	0x00007fff5af2b2c6 __pthread_kill + 10
1   libsystem_pthread.dylib       	0x00007fff5afe6bf1 pthread_kill + 284
2   libsystem_c.dylib             	0x00007fff5ae95745 __abort + 144
3   libsystem_c.dylib             	0x00007fff5ae956b5 abort + 142
4   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000105728b8f swift::Mangle::Mangler::verify(llvm::StringRef) (.cold.2) + 63
5   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000104aba40d swift::Mangle::Mangler::verify(llvm::StringRef) + 781
6   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102cd93fb swift::Mangle::ASTMangler::mangleDeclAsUSR(swift::ValueDecl const*, llvm::StringRef) + 443 (ASTMangler.cpp:523)
7   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102e6c9bd swift::USRGenerationRequest::evaluate(swift::Evaluator&, swift::ValueDecl const*) const + 1101 (Error.h:477)
8   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102a49f01 swift::SimpleRequest<swift::USRGenerationRequest, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > (swift::ValueDecl const*), (swift::CacheKind)1>::evaluateRequest(swift::USRGenerationRequest const&, swift::Evaluator&) + 17 (SimpleRequest.h:219)
9   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102e6f42c llvm::Expected<swift::USRGenerationRequest::OutputType> swift::Evaluator::getResultUncached<swift::USRGenerationRequest>(swift::USRGenerationRequest const&) + 380 (Evaluator.h:381)
10  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102e6ef5f llvm::Expected<swift::USRGenerationRequest::OutputType> swift::Evaluator::getResultCached<swift::USRGenerationRequest, (void*)0>(swift::USRGenerationRequest const&) + 239 (Error.h:526)
11  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102e6debb llvm::Expected<swift::USRGenerationRequest::OutputType> swift::Evaluator::operator()<swift::USRGenerationRequest>(swift::USRGenerationRequest const&) + 107 (vector:663)
12  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102e6d1d2 swift::USRGenerationRequest::OutputType swift::evaluateOrDefault<swift::USRGenerationRequest>(swift::Evaluator&, swift::USRGenerationRequest, swift::USRGenerationRequest::OutputType) + 34 (Error.h:554)
13  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102e6d140 swift::ide::printValueDeclUSR(swift::ValueDecl const*, llvm::raw_ostream&) + 96 (string:1421)
14  com.apple.SourceKitService.5.0.20191101101_osx	0x000000010205a838 passCursorInfoForDecl(swift::SourceFile*, swift::ValueDecl const*, swift::ModuleDecl const*, swift::Type, bool, bool, swift::ide::ResolvedCursorInfo, llvm::Optional<unsigned int>, swift::SourceLoc, llvm::ArrayRef<SourceKit::RefactoringInfo>, SourceKit::SwiftLangSupport&, swift::CompilerInvocation const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, llvm::ArrayRef<llvm::IntrusiveRefCntPtr<SourceKit::ImmutableTextSnapshot> >, std::__1::function<void (SourceKit::RequestResult<SourceKit::CursorInfoData> const&)>) + 744 (SwiftSourceDocInfo.cpp:773)
15  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102059114 resolveCursor(SourceKit::SwiftLangSupport&, llvm::StringRef, unsigned int, unsigned int, bool, llvm::IntrusiveRefCntPtr<SourceKit::SwiftInvocation>, bool, bool, llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>, std::__1::function<void (SourceKit::RequestResult<SourceKit::CursorInfoData> const&)>)::CursorInfoConsumer::handlePrimaryAST(llvm::IntrusiveRefCntPtr<SourceKit::ASTUnit>) + 4340 (SwiftSourceDocInfo.cpp:1352)
16  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001020156a8 void SourceKit::WorkQueue::DispatchData::callAndDelete<SourceKit::ASTUnit::Implementation::consumeAsync(std::__1::shared_ptr<SourceKit::SwiftASTConsumer>, llvm::IntrusiveRefCntPtr<SourceKit::ASTUnit>)::$_0>(void*) + 88 (IntrusiveRefCntPtr.h:188)
17  libdispatch.dylib             	0x00007fff5ada363d _dispatch_client_callout + 8
18  libdispatch.dylib             	0x00007fff5ada98e0 _dispatch_lane_serial_drain + 602
19  libdispatch.dylib             	0x00007fff5adaa396 _dispatch_lane_invoke + 385
20  libdispatch.dylib             	0x00007fff5adb26ed _dispatch_workloop_worker_thread + 598
21  libsystem_pthread.dylib       	0x00007fff5afe3611 _pthread_wqthread + 421
22  libsystem_pthread.dylib       	0x00007fff5afe33fd start_wqthread + 13

The changes to lib/AST/ASTScopeLookup.cpp fix the problems in https://github.com/AsyncNinja/AsyncNinja.git (and https://github.com/123flo321/pogoprotos-swift.git)

Crashed Thread:        2  Dispatch queue: com.apple.root.default-qos

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
dyld2 mode
Stack dump:
0.	While walking into decl declaration 0x7f8b2f307ba8 (at /Volumes/Elements/pkgs/single7/AsyncNinja/Sources/appleOS_Observation.swift:23:1)
1.	While evaluating request StorageImplInfoRequest(AsyncNinja.(file).ObservationSessionItem.isEnabled@/Volumes/Elements/pkgs/single7/AsyncNinja/Sources/appleOS_Observation.swift:89:9)
2.	While evaluating request IsObjCRequest(AsyncNinja.(file).ObservationSessionItem@/Volumes/Elements/pkgs/single7/AsyncNinja/Sources/appleOS_Observation.swift:88:18)
3.	While evaluating request InheritedDeclsReferencedRequest(AsyncNinja.(file).ObservationSessionItem@/Volumes/Elements/pkgs/single7/AsyncNinja/Sources/appleOS_Observation.swift:88:18, 0)
 
abort() called

Thread 0:: Dispatch queue: com.apple.root.default-qos.overcommit
0   libsystem_kernel.dylib        	0x00007fff5af29126 __sigsuspend_nocancel + 10
1   libdispatch.dylib             	0x00007fff5adb2ef3 _dispatch_sigsuspend + 21
2   libdispatch.dylib             	0x00007fff5adb2ede _dispatch_sig_thread + 53

Thread 1:
0   libsystem_pthread.dylib       	0x00007fff5afe33f0 start_wqthread + 0

Thread 2 Crashed:: Dispatch queue: com.apple.root.default-qos
0   libsystem_kernel.dylib        	0x00007fff5af2b2c6 __pthread_kill + 10
1   libsystem_pthread.dylib       	0x00007fff5afe6bf1 pthread_kill + 284
2   libsystem_c.dylib             	0x00007fff5ae95745 __abort + 144
3   libsystem_c.dylib             	0x00007fff5ae956b5 abort + 142
4   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000105bc81f1 fatal_error_handler(void*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool) + 65
5   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000105c94f4b llvm::report_fatal_error(llvm::Twine const&, bool) + 251 (string:1421)
6   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000105c94e4b llvm::report_fatal_error(char const*, bool) + 43
7   com.apple.SourceKitService.5.0.20191101101_osx	0x00000001068d7c7b swift::ast_scope::ASTScopeImpl::findStartingScopeForLookup(swift::SourceFile*, swift::DeclName, swift::SourceLoc, swift::DeclContext const*) + 1115 (ASTScopeLookup.cpp:116)
8   com.apple.SourceKitService.5.0.20191101101_osx	0x00000001068d77fd swift::ast_scope::ASTScopeImpl::unqualifiedLookup(swift::SourceFile*, swift::DeclName, swift::SourceLoc, swift::DeclContext const*, swift::namelookup::AbstractASTScopeDeclConsumer&) + 45 (ASTScopeLookup.cpp:47)
9   com.apple.SourceKitService.5.0.20191101101_osx	0x00000001068c5106 swift::ASTScope::unqualifiedLookup(swift::SourceFile*, swift::DeclName, swift::SourceLoc, swift::DeclContext const*, swift::namelookup::AbstractASTScopeDeclConsumer&) + 86 (ASTScope.cpp:47)
10  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000106a3557d (anonymous namespace)::UnqualifiedLookupFactory::lookInASTScopes() + 189
11  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000106a33d5c swift::UnqualifiedLookup::UnqualifiedLookup(swift::DeclName, swift::DeclContext*, swift::SourceLoc, swift::OptionSet<swift::UnqualifiedLookup::Flags, unsigned int>) + 956 (UnqualifiedLookup.cpp:490)
12  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001069ec231 directReferencesForTypeRepr(swift::Evaluator&, swift::ASTContext&, swift::TypeRepr*, swift::DeclContext*) + 1073 (SmallVector.h:127)
13  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001069f0961 swift::InheritedDeclsReferencedRequest::evaluate(swift::Evaluator&, llvm::PointerUnion<swift::TypeDecl*, swift::ExtensionDecl*>, unsigned int) const + 177 (NameLookup.cpp:2114)
14  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001069fb839 swift::SimpleRequest<swift::InheritedDeclsReferencedRequest, llvm::TinyPtrVector<swift::TypeDecl*> (llvm::PointerUnion<swift::TypeDecl*, swift::ExtensionDecl*>, unsigned int), (swift::CacheKind)0>::evaluateRequest(swift::InheritedDeclsReferencedRequest const&, swift::Evaluator&) + 25 (Error.h:477)
15  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001069faf02 llvm::Expected<swift::InheritedDeclsReferencedRequest::OutputType> swift::Evaluator::getResultUncached<swift::InheritedDeclsReferencedRequest>(swift::InheritedDeclsReferencedRequest const&) + 418 (Evaluator.h:381)
16  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001069fa1ab llvm::Expected<swift::InheritedDeclsReferencedRequest::OutputType> swift::Evaluator::operator()<swift::InheritedDeclsReferencedRequest>(swift::InheritedDeclsReferencedRequest const&) + 107 (vector:663)
17  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001069f0ef7 swift::InheritedDeclsReferencedRequest::OutputType swift::evaluateOrDefault<swift::InheritedDeclsReferencedRequest>(swift::Evaluator&, swift::InheritedDeclsReferencedRequest, swift::InheritedDeclsReferencedRequest::OutputType) + 39 (Error.h:554)
18  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001069f21fb swift::getDirectlyInheritedNominalTypeDecls(llvm::PointerUnion<swift::TypeDecl*, swift::ExtensionDecl*>, unsigned int, llvm::SmallVectorImpl<std::__1::pair<swift::SourceLoc, swift::NominalTypeDecl*> >&, bool&) + 187 (PointerIntPair.h:60)
19  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001069f2500 swift::getDirectlyInheritedNominalTypeDecls(llvm::PointerUnion<swift::TypeDecl*, swift::ExtensionDecl*>, bool&) + 144 (Range.h:64)
20  com.apple.SourceKitService.5.0.20191101101_osx	0x000000010690f5f4 swift::ProtocolDecl::getInheritedProtocolsSlow() + 164 (SmallVector.h:127)
21  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001065ad363 swift::IsObjCRequest::evaluate(swift::Evaluator&, swift::ValueDecl*) const + 1875 (Decl.h:4228)
22  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000106615b11 swift::SimpleRequest<swift::IsObjCRequest, bool (swift::ValueDecl*), (swift::CacheKind)2>::evaluateRequest(swift::IsObjCRequest const&, swift::Evaluator&) + 17 (SimpleRequest.h:219)
23  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001069309e4 llvm::Expected<swift::IsObjCRequest::OutputType> swift::Evaluator::getResultUncached<swift::IsObjCRequest>(swift::IsObjCRequest const&) + 372 (Evaluator.h:381)
24  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000106930798 llvm::Expected<swift::IsObjCRequest::OutputType> swift::Evaluator::getResultCached<swift::IsObjCRequest, (void*)0>(swift::IsObjCRequest const&) + 56 (Error.h:526)
25  com.apple.SourceKitService.5.0.20191101101_osx	0x000000010692fc0b llvm::Expected<swift::IsObjCRequest::OutputType> swift::Evaluator::operator()<swift::IsObjCRequest>(swift::IsObjCRequest const&) + 107 (vector:663)
26  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000106906dc2 swift::IsObjCRequest::OutputType swift::evaluateOrDefault<swift::IsObjCRequest>(swift::Evaluator&, swift::IsObjCRequest, swift::IsObjCRequest::OutputType) + 34 (Error.h:554)
27  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000106625ab8 swift::StorageImplInfoRequest::evaluate(swift::Evaluator&, swift::AbstractStorageDecl*) const + 2872 (StorageImpl.h:367)
28  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000106615e01 swift::SimpleRequest<swift::StorageImplInfoRequest, swift::StorageImplInfo (swift::AbstractStorageDecl*), (swift::CacheKind)2>::evaluateRequest(swift::StorageImplInfoRequest const&, swift::Evaluator&) + 17 (SimpleRequest.h:219)
29  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000106950614 llvm::Expected<swift::StorageImplInfoRequest::OutputType> swift::Evaluator::getResultUncached<swift::StorageImplInfoRequest>(swift::StorageImplInfoRequest const&) + 372 (Evaluator.h:381)
30  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001069503c8 llvm::Expected<swift::StorageImplInfoRequest::OutputType> swift::Evaluator::getResultCached<swift::StorageImplInfoRequest, (void*)0>(swift::StorageImplInfoRequest const&) + 56 (Error.h:526)
31  com.apple.SourceKitService.5.0.20191101101_osx	0x000000010694f82b llvm::Expected<swift::StorageImplInfoRequest::OutputType> swift::Evaluator::operator()<swift::StorageImplInfoRequest>(swift::StorageImplInfoRequest const&) + 107 (vector:663)
32  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000106912182 swift::StorageImplInfoRequest::OutputType swift::evaluateOrDefault<swift::StorageImplInfoRequest>(swift::Evaluator&, swift::StorageImplInfoRequest, swift::StorageImplInfoRequest::OutputType) + 34 (Error.h:554)
33  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001069137e8 swift::VarDecl::isSettable(swift::DeclContext const*, swift::DeclRefExpr const*) const + 408 (StorageImpl.h:360)
34  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000105c05fe9 (anonymous namespace)::SwiftDocumentStructureWalker::walkToSubStructurePre(swift::ide::SyntaxStructureNode) + 569 (SwiftEditor.cpp:1184)
35  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000105c055bf (anonymous namespace)::SwiftEditorSyntaxWalker::walkToSubStructurePre(swift::ide::SyntaxStructureNode) + 47 (vector:462)
36  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000105ddd074 (anonymous namespace)::ModelASTWalker::pushStructureNode(swift::ide::SyntaxStructureNode const&, swift::ASTWalker::ParentTy const&) + 772 (SyntaxModel.cpp:1370)
37  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000105ddc0cc (anonymous namespace)::ModelASTWalker::walkToDeclPre(swift::Decl*) + 6956
38  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001068dc54a (anonymous namespace)::Traversal::doIt(swift::Decl*) + 170 (ASTWalker.cpp:1238)
39  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001068e0320 swift::ASTVisitor<(anonymous namespace)::Traversal, swift::Expr*, swift::Stmt*, bool, swift::Pattern*, bool, void>::visit(swift::Pattern*) + 192 (ASTWalker.cpp:1687)
40  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001068e05b2 swift::ASTVisitor<(anonymous namespace)::Traversal, swift::Expr*, swift::Stmt*, bool, swift::Pattern*, bool, void>::visit(swift::Pattern*) + 850 (ASTWalker.cpp:92)
41  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001068e1643 swift::ASTVisitor<(anonymous namespace)::Traversal, swift::Expr*, swift::Stmt*, bool, swift::Pattern*, bool, void>::visit(swift::Decl*) + 2099 (ASTWalker.cpp:92)
42  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001068dc577 (anonymous namespace)::Traversal::doIt(swift::Decl*) + 215 (ASTWalker.cpp:113)
43  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001068e1adb (anonymous namespace)::Traversal::visitNominalTypeDecl(swift::NominalTypeDecl*) + 523 (ASTWalker.cpp:291)
44  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001068dc577 (anonymous namespace)::Traversal::doIt(swift::Decl*) + 215 (ASTWalker.cpp:113)
45  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001068dc48b swift::Decl::walk(swift::ASTWalker&) + 27 (ASTWalker.cpp:1903)
46  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000105ddb259 (anonymous namespace)::ModelASTWalker::walkToDeclPre(swift::Decl*) + 3257 (DenseSet.h:189)
47  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001068dc54a (anonymous namespace)::Traversal::doIt(swift::Decl*) + 170 (ASTWalker.cpp:1238)
48  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001068dc48b swift::Decl::walk(swift::ASTWalker&) + 27 (ASTWalker.cpp:1903)
49  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001069e008e swift::SourceFile::walk(swift::ASTWalker&) + 190 (Module.cpp:1744)
50  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000105dd883b swift::ide::SyntaxModelContext::walk(swift::ide::SyntaxModelWalker&) + 315 (ArrayRef.h:137)
51  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000105bfe6c3 SourceKit::SwiftEditorDocument::readSyntaxInfo(SourceKit::EditorConsumer&) + 1139 (SwiftEditor.cpp:1883)
52  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000105c00a3e SourceKit::SwiftLangSupport::editorOpen(llvm::StringRef, llvm::MemoryBuffer*, SourceKit::EditorConsumer&, llvm::ArrayRef<char const*>, llvm::Optional<SourceKit::VFSOptions>) + 1726 (IntrusiveRefCntPtr.h:164)
53  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000105c3c1aa handleRequestImpl(void*, std::__1::function<void (void*)>) + 11834 (Requests.cpp:2388)
54  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000105c39323 sourcekitd::handleRequest(void*, std::__1::function<void (void*)>) + 275 (functional:1757)
55  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000105bc8748 invocation function for block in sourcekitdServer_peer_event_handler(_xpc_connection_s*, void*) + 216 (functional:1757)
56  libdispatch.dylib             	0x00007fff5ada25f8 _dispatch_call_block_and_release + 12
57  libdispatch.dylib             	0x00007fff5ada363d _dispatch_client_callout + 8
58  libdispatch.dylib             	0x00007fff5ada5a53 _dispatch_queue_override_invoke + 763
59  libdispatch.dylib             	0x00007fff5adb13bc _dispatch_root_queue_drain + 324
60  libdispatch.dylib             	0x00007fff5adb1b46 _dispatch_worker_thread2 + 90
61  libsystem_pthread.dylib       	0x00007fff5afe36b3 _pthread_wqthread + 583
62  libsystem_pthread.dylib       	0x00007fff5afe33fd start_wqthread + 13

The change to lib/Basic/Decl.cpp fixes a problem in https://github.com/adam-fowler/aws-signer.git

Crashed Thread:        3  Dispatch queue: sourcekit.swift.ConsumeAST

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
dyld2 mode
Stack dump:
0.	While evaluating request USRGenerationRequest(AsyncHTTPClient.(file).HTTPClientProxyHandler.WriteItem.write@/Volumes/Elements/pkgs/single8/aws-signer/.build/checkouts/async-http-client/Sources/AsyncHTTPClient/HTTPClientProxyHandler.swift:64:14)
 
abort() called

Thread 0:
0   libsystem_pthread.dylib       	0x00007fff5afe33f0 start_wqthread + 0

Thread 1:: Dispatch queue: com.apple.root.default-qos.overcommit
0   libsystem_kernel.dylib        	0x00007fff5af29126 __sigsuspend_nocancel + 10
1   libdispatch.dylib             	0x00007fff5adb2ef3 _dispatch_sigsuspend + 21
2   libdispatch.dylib             	0x00007fff5adb2ede _dispatch_sig_thread + 53

Thread 2:: Dispatch queue: sourcekit.swift.ASTBuilding
0   libsystem_kernel.dylib        	0x00007fff5af279de __ulock_wait + 10
1   libsystem_pthread.dylib       	0x00007fff5afe86de _pthread_join + 358
2   com.apple.SourceKitService.5.0.20191101101_osx	0x000000010f6297ca llvm::llvm_execute_on_thread(void (*)(void*), void*, unsigned int) + 90
3   libdispatch.dylib             	0x00007fff5ada363d _dispatch_client_callout + 8
4   libdispatch.dylib             	0x00007fff5ada98e0 _dispatch_lane_serial_drain + 602
5   libdispatch.dylib             	0x00007fff5adaa396 _dispatch_lane_invoke + 385
6   libdispatch.dylib             	0x00007fff5adb26ed _dispatch_workloop_worker_thread + 598
7   libsystem_pthread.dylib       	0x00007fff5afe3611 _pthread_wqthread + 421
8   libsystem_pthread.dylib       	0x00007fff5afe33fd start_wqthread + 13

Thread 3 Crashed:: Dispatch queue: sourcekit.swift.ConsumeAST
0   libsystem_kernel.dylib        	0x00007fff5af2b2c6 __pthread_kill + 10
1   libsystem_pthread.dylib       	0x00007fff5afe6bf1 pthread_kill + 284
2   libsystem_c.dylib             	0x00007fff5ae95745 __abort + 144
3   libsystem_c.dylib             	0x00007fff5ae956b5 abort + 142
4   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000112c02b8f swift::Mangle::Mangler::verify(llvm::StringRef) (.cold.2) + 63
5   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000111f9440d swift::Mangle::Mangler::verify(llvm::StringRef) + 781
6   com.apple.SourceKitService.5.0.20191101101_osx	0x00000001101b33fb swift::Mangle::ASTMangler::mangleDeclAsUSR(swift::ValueDecl const*, llvm::StringRef) + 443 (ASTMangler.cpp:523)
7   com.apple.SourceKitService.5.0.20191101101_osx	0x00000001103469bd swift::USRGenerationRequest::evaluate(swift::Evaluator&, swift::ValueDecl const*) const + 1101 (Error.h:477)
8   com.apple.SourceKitService.5.0.20191101101_osx	0x000000010ff23f01 swift::SimpleRequest<swift::USRGenerationRequest, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > (swift::ValueDecl const*), (swift::CacheKind)1>::evaluateRequest(swift::USRGenerationRequest const&, swift::Evaluator&) + 17 (SimpleRequest.h:219)
9   com.apple.SourceKitService.5.0.20191101101_osx	0x000000011034942c llvm::Expected<swift::USRGenerationRequest::OutputType> swift::Evaluator::getResultUncached<swift::USRGenerationRequest>(swift::USRGenerationRequest const&) + 380 (Evaluator.h:381)
10  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000110348f5f llvm::Expected<swift::USRGenerationRequest::OutputType> swift::Evaluator::getResultCached<swift::USRGenerationRequest, (void*)0>(swift::USRGenerationRequest const&) + 239 (Error.h:526)
11  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000110347ebb llvm::Expected<swift::USRGenerationRequest::OutputType> swift::Evaluator::operator()<swift::USRGenerationRequest>(swift::USRGenerationRequest const&) + 107 (vector:663)
12  com.apple.SourceKitService.5.0.20191101101_osx	0x00000001103471d2 swift::USRGenerationRequest::OutputType swift::evaluateOrDefault<swift::USRGenerationRequest>(swift::Evaluator&, swift::USRGenerationRequest, swift::USRGenerationRequest::OutputType) + 34 (Error.h:554)
13  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000110347140 swift::ide::printValueDeclUSR(swift::ValueDecl const*, llvm::raw_ostream&) + 96 (string:1421)
14  com.apple.SourceKitService.5.0.20191101101_osx	0x000000010f534838 passCursorInfoForDecl(swift::SourceFile*, swift::ValueDecl const*, swift::ModuleDecl const*, swift::Type, bool, bool, swift::ide::ResolvedCursorInfo, llvm::Optional<unsigned int>, swift::SourceLoc, llvm::ArrayRef<SourceKit::RefactoringInfo>, SourceKit::SwiftLangSupport&, swift::CompilerInvocation const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, llvm::ArrayRef<llvm::IntrusiveRefCntPtr<SourceKit::ImmutableTextSnapshot> >, std::__1::function<void (SourceKit::RequestResult<SourceKit::CursorInfoData> const&)>) + 744 (SwiftSourceDocInfo.cpp:773)
15  com.apple.SourceKitService.5.0.20191101101_osx	0x000000010f533114 resolveCursor(SourceKit::SwiftLangSupport&, llvm::StringRef, unsigned int, unsigned int, bool, llvm::IntrusiveRefCntPtr<SourceKit::SwiftInvocation>, bool, bool, llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>, std::__1::function<void (SourceKit::RequestResult<SourceKit::CursorInfoData> const&)>)::CursorInfoConsumer::handlePrimaryAST(llvm::IntrusiveRefCntPtr<SourceKit::ASTUnit>) + 4340 (SwiftSourceDocInfo.cpp:1352)
16  com.apple.SourceKitService.5.0.20191101101_osx	0x000000010f4ef6a8 void SourceKit::WorkQueue::DispatchData::callAndDelete<SourceKit::ASTUnit::Implementation::consumeAsync(std::__1::shared_ptr<SourceKit::SwiftASTConsumer>, llvm::IntrusiveRefCntPtr<SourceKit::ASTUnit>)::$_0>(void*) + 88 (IntrusiveRefCntPtr.h:188)
17  libdispatch.dylib             	0x00007fff5ada363d _dispatch_client_callout + 8
18  libdispatch.dylib             	0x00007fff5ada98e0 _dispatch_lane_serial_drain + 602
19  libdispatch.dylib             	0x00007fff5adaa396 _dispatch_lane_invoke + 385
20  libdispatch.dylib             	0x00007fff5adb26ed _dispatch_workloop_worker_thread + 598
21  libsystem_pthread.dylib       	0x00007fff5afe3611 _pthread_wqthread + 421
22  libsystem_pthread.dylib       	0x00007fff5afe33fd start_wqthread + 13

The change to lib/IDE/SyntaxModel.cpp a problem in https://github.com/adam-fowler/xml-encoder.git

Crashed Thread:        3  Dispatch queue: com.apple.root.default-qos

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
dyld2 mode
Stack dump:
0.	While walking into decl 'DictionaryEncoder' (at /Volumes/Elements/pkgs/single7/dictionary-encoder/Sources/DictionaryEncoder/DictionaryEncoder.swift:61:6)
 
Assertion failed: (0 && "Attribute's TokenNodes already consumed?"), function handleSpecialDeclAttribute, file /Users/buildnode/jenkins/workspace/oss-swift-package-osx/swift/lib/IDE/SyntaxModel.cpp, line 1169.
 

Thread 0:
0   libsystem_kernel.dylib        	0x00007fff5af290fe __semwait_signal_nocancel + 10
1   libsystem_c.dylib             	0x00007fff5ae406d8 nanosleep$NOCANCEL + 188
2   libsystem_c.dylib             	0x00007fff5ae6671b sleep$NOCANCEL + 41
3   libdispatch.dylib             	0x00007fff5adb2e76 _dispatch_queue_cleanup2 + 104
4   libsystem_pthread.dylib       	0x00007fff5afe46ab _pthread_tsd_cleanup + 551
5   libsystem_pthread.dylib       	0x00007fff5afe7655 _pthread_exit + 70
6   libsystem_pthread.dylib       	0x00007fff5afe445b pthread_exit + 30
7   libdispatch.dylib             	0x00007fff5adae9b8 dispatch_main + 110
8   libxpc.dylib                  	0x00007fff5b029093 _xpc_objc_main + 580
9   libxpc.dylib                  	0x00007fff5b028b79 xpc_main + 433
10  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102226611 main + 881
11  libdyld.dylib                 	0x00007fff5adf03d5 start + 1

Thread 1:
0   libsystem_pthread.dylib       	0x00007fff5afe33f0 start_wqthread + 0

Thread 2:: Dispatch queue: com.apple.root.default-qos.overcommit
0   libsystem_kernel.dylib        	0x00007fff5af29126 __sigsuspend_nocancel + 10
1   libdispatch.dylib             	0x00007fff5adb2ef3 _dispatch_sigsuspend + 21
2   libdispatch.dylib             	0x00007fff5adb2ede _dispatch_sig_thread + 53

Thread 3 Crashed:: Dispatch queue: com.apple.root.default-qos
0   libsystem_kernel.dylib        	0x00007fff5af2b2c6 __pthread_kill + 10
1   libsystem_pthread.dylib       	0x00007fff5afe6bf1 pthread_kill + 284
2   libsystem_c.dylib             	0x00007fff5ae95745 __abort + 144
3   libsystem_c.dylib             	0x00007fff5ae956b5 abort + 142
4   libsystem_c.dylib             	0x00007fff5ae5e20d __assert_rtn + 324
5   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000104de34d3 (anonymous namespace)::ModelASTWalker::handleSpecialDeclAttribute(swift::DeclAttribute const*, llvm::ArrayRef<swift::Token>) (.cold.2) + 35
6   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102448ed9 (anonymous namespace)::ModelASTWalker::handleSpecialDeclAttribute(swift::DeclAttribute const*, llvm::ArrayRef<swift::Token>) + 313 (SyntaxModel.cpp:1172)
7   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102448b59 (anonymous namespace)::ModelASTWalker::handleAttrRanges(llvm::ArrayRef<std::__1::pair<swift::DeclAttribute const*, swift::SourceRange> >) + 585 (SyntaxModel.cpp:1264)
8   com.apple.SourceKitService.5.0.20191101101_osx	0x000000010243879f (anonymous namespace)::ModelASTWalker::walkToDeclPre(swift::Decl*) + 671 (SyntaxModel.cpp:1190)
9   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102f3823a (anonymous namespace)::Traversal::doIt(swift::Decl*) + 170 (ASTWalker.cpp:1238)
10  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102f3817b swift::Decl::walk(swift::ASTWalker&) + 27 (ASTWalker.cpp:1903)
11  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102439c3c (anonymous namespace)::ModelASTWalker::walkToDeclPre(swift::Decl*) + 5948 (PointerIntPair.h:177)
12  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102f3823a (anonymous namespace)::Traversal::doIt(swift::Decl*) + 170 (ASTWalker.cpp:1238)
13  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102f3d7cb (anonymous namespace)::Traversal::visitNominalTypeDecl(swift::NominalTypeDecl*) + 523 (ASTWalker.cpp:291)
14  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102f38267 (anonymous namespace)::Traversal::doIt(swift::Decl*) + 215 (ASTWalker.cpp:1241)
15  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102f3d7cb (anonymous namespace)::Traversal::visitNominalTypeDecl(swift::NominalTypeDecl*) + 523 (ASTWalker.cpp:291)
16  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102f38267 (anonymous namespace)::Traversal::doIt(swift::Decl*) + 215 (ASTWalker.cpp:1241)
17  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102f3817b swift::Decl::walk(swift::ASTWalker&) + 27 (ASTWalker.cpp:1903)
18  com.apple.SourceKitService.5.0.20191101101_osx	0x000000010303c2ee swift::SourceFile::walk(swift::ASTWalker&) + 190 (Module.cpp:1744)
19  com.apple.SourceKitService.5.0.20191101101_osx	0x000000010243679b swift::ide::SyntaxModelContext::walk(swift::ide::SyntaxModelWalker&) + 315 (SyntaxModel.cpp:508)
20  com.apple.SourceKitService.5.0.20191101101_osx	0x000000010225cb33 SourceKit::SwiftEditorDocument::readSyntaxInfo(SourceKit::EditorConsumer&) + 1139 (SwiftEditor.cpp:1883)
21  com.apple.SourceKitService.5.0.20191101101_osx	0x000000010225eeae SourceKit::SwiftLangSupport::editorOpen(llvm::StringRef, llvm::MemoryBuffer*, SourceKit::EditorConsumer&, llvm::ArrayRef<char const*>, llvm::Optional<SourceKit::VFSOptions>) + 1726 (SwiftEditor.cpp:2184)
22  com.apple.SourceKitService.5.0.20191101101_osx	0x000000010229a61a handleRequestImpl(void*, std::__1::function<void (void*)>) + 11834 (Requests.cpp:2388)
23  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102297793 sourcekitd::handleRequest(void*, std::__1::function<void (void*)>) + 275 (Requests.cpp:349)
24  com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102226bb8 invocation function for block in sourcekitdServer_peer_event_handler(_xpc_connection_s*, void*) + 216 (functional:1757)
25  libdispatch.dylib             	0x00007fff5ada25f8 _dispatch_call_block_and_release + 12
26  libdispatch.dylib             	0x00007fff5ada363d _dispatch_client_callout + 8
27  libdispatch.dylib             	0x00007fff5ada5a53 _dispatch_queue_override_invoke + 763
28  libdispatch.dylib             	0x00007fff5adb13bc _dispatch_root_queue_drain + 324
29  libdispatch.dylib             	0x00007fff5adb1b46 _dispatch_worker_thread2 + 90
30  libsystem_pthread.dylib       	0x00007fff5afe36b3 _pthread_wqthread + 583
31  libsystem_pthread.dylib       	0x00007fff5afe33fd start_wqthread + 13

nathawes · 2019-11-19T19:22:59Z

I think #28226 should have solved the "Extensions must have already been bound" assertion hit and the second in your previous comment. I still need to look into the other two though.

johnno1962 · 2019-11-19T20:55:05Z

Thanks @nathawes, I've tried to give you all the information you need to replicate.

nathawes · 2019-11-19T21:04:20Z

Yeah thanks for that! Having reproducers helps a lot. I'm planning on tackling the mangler/USR generation issue next.

johnno1962 · 2019-11-19T21:16:02Z

From memory the demangler in the verify() returns NULL for Root and the string it was trying to demangle looked a litlle odd e.g. $s13ResponseTests9TestErrorO5error33_855023448C182E12494CBF31350D2856LLXeF and wouldn't de-mangle using swift de-mangle

System Integrity Protection: enabled

Crashed Thread:        1  Dispatch queue: sourcekit.swift.ConsumeAST

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
dyld2 mode
Stack dump:
0.	While evaluating request USRGenerationRequest(ResponseTests.(file).TestError.error(error:)@/Volumes/Elements/pkgs/single7/Alamofire/Tests/ResponseTests.swift:529:10)
 
abort() called

Thread 0:: Dispatch queue: com.apple.root.default-qos.overcommit
0   libsystem_kernel.dylib        	0x00007fff5af29126 __sigsuspend_nocancel + 10
1   libdispatch.dylib             	0x00007fff5adb2ef3 _dispatch_sigsuspend + 21
2   libdispatch.dylib             	0x00007fff5adb2ede _dispatch_sig_thread + 53

Thread 1 Crashed:: Dispatch queue: sourcekit.swift.ConsumeAST
0   libsystem_kernel.dylib        	0x00007fff5af2b2c6 __pthread_kill + 10
1   libsystem_pthread.dylib       	0x00007fff5afe6bf1 pthread_kill + 284
2   libsystem_c.dylib             	0x00007fff5ae95745 __abort + 144
3   libsystem_c.dylib             	0x00007fff5ae956b5 abort + 142
4   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000105728b8f swift::Mangle::Mangler::verify(llvm::StringRef) (.cold.2) + 63
5   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000104aba40d swift::Mangle::Mangler::verify(llvm::StringRef) + 781
6   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102cd93fb swift::Mangle::ASTMangler::mangleDeclAsUSR(swift::ValueDecl const*, llvm::StringRef) + 443 (ASTMangler.cpp:523)
7   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102e6c9bd swift::USRGenerationRequest::evaluate(swift::Evaluator&, swift::ValueDecl const*) const + 1101 (Error.h:477)
8   com.apple.SourceKitService.5.0.20191101101_osx	0x0000000102a49f01 swift::SimpleRequest<swift::USRGenerationRequest, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > (swift::ValueDecl const*), (swift::CacheKind)1>::evaluateRequest(swift::USRGenerationRequest const&, swift::Evaluator&) + 17 (SimpleRequest.h:219)

CodaFi · 2019-11-27T05:57:40Z

@johnno1962 This pull request should be broken down into multiple SRs if it isn't already.

johnno1962 · 2019-11-27T06:35:20Z

This is not a serious pull request, more documentation of the assertions I had to remove to run through a large corpus of code.

CodaFi · 2019-11-27T06:46:18Z

I'm grateful that you took the time to reduce the problem and provide your analysis - it was super helpful getting these things fixed on our end - but we have a policy of not taking issues, or things that look like issues, on Github and tracking work in Jira tickets. As unpleasant as it is, it is standard operating procedure.

Have you tried seeing if a recent toolchain fixes the issues you saw too? I'm wondering if we still need to keep this open after Nathan's patches.

…eKit.

johnno1962 · 2019-11-27T16:20:07Z

I've created an SR https://bugs.swift.org/browse/SR-11863, to track these crashes referring to this PR which is the easiest way to track the remaining problems. I've also rebased to master and removed from this PR the crash points that have been resolved. The most common sources of assertion failures have been fixed now (thanks!). The next most common at this stage and perhaps the easiest to fix in a superficial way is the "Attribute's TokenNodes already consumed?" assertion failure which occurs in projects https://github.com/eBardX/XestiMonitors.git at XestiMonitors/Sources/Core/UIKit/Accessibility/AccessibilityStatusMonitor.swift:10:1, https://github.com/adam-fowler/xml-encoder.git at xml-encoder/Sources/xml-encoder/XMLDecoder.swift:49:8, https://github.com/adam-fowler/dictionary-encoder.git at dictionary-encoder/Sources/DictionaryEncoder/DictionaryEncoder.swift:61:6, https://github.com/ashleymills/Reachability.swift.git at Reachability.swift/Sources/Reachability.swift:46:8.

After that the crashes are getting pretty rare and it may not be worth pursuing them. Thanks again!

johnno1962 · 2020-02-12T13:47:01Z

I’m closing this now to tidy things up. I’ve just done a test run using a toolchain from master and the script above and main problem remaining is the new "Attribute's TokenNodes already consumed?" assertion failure which may be more trouble than it’s worth but it’s not so common. Thanks 👍

nathawes · 2020-02-12T17:45:30Z

@johnno1962 I'm actually fixing that one at the moment :-)
#29769

johnno1962 · 2020-02-12T17:55:06Z

Ha, Terrific! Do you know if it’ll make the Xcode 11.4 release?

nathawes · 2020-02-12T18:15:03Z

I wasn't planning on cherry-picking it into the release branch because it's purely an assertion failure. When assertions are disabled (like they are in a release build) it doesn't crash and the only difference the fix makes otherwise is that built-in attributes on enum cases are now correctly reported as built-in (instead of non-builtin) in SourceKit's syntax map used for syntax coloring.

johnno1962 · 2020-02-12T18:26:26Z

Ah, silly me. Do you know if there are plans to include the sourcekit-lsp binary in Xcode? I was surprised not to see it in the 11.4 beta1 toolchain.

benlangmuir · 2020-02-12T18:50:01Z

Do you know if there are plans to include the sourcekit-lsp binary in Xcode? I was surprised not to see it in the 11.4 beta1 toolchain.

It should be at

Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/sourcekit-lsp

in the 11.4 beta.

johnno1962 · 2020-02-12T19:05:05Z

It is indeed there. Great!

nathawes mentioned this pull request Nov 7, 2019

[IDE] Fix CursorInfo crash on arg label when the called 'function' is actually a module decl. #28134

Merged

johnno1962 force-pushed the sourckit-lsp-crashes branch 2 times, most recently from 4ea8198 to f265b69 Compare November 19, 2019 08:01

nathawes mentioned this pull request Nov 21, 2019

[ASTMangler] Don't verify the USR mangling for invalid decls #28397

Merged

johnno1962 changed the title ~~[SourceKit] Log attempts to mangle a module instead of llvm_unreachable for SourceKit/LSP~~ [DNM][SourceKit] Log attempts to mangle a module instead of llvm_unreachable for SourceKit/LSP Nov 27, 2019

johnno1962 added 4 commits November 27, 2019 16:34

Log attempts to mangle a module instead of llvm_unreachable for Sourc…

51eaca0

…eKit.

Remove for assertion for now

8b88d6c

Remove extension assertion for now

1066f0c

Remove a few more crashing assertions

ba40a0a

johnno1962 force-pushed the sourckit-lsp-crashes branch from f265b69 to ba40a0a Compare November 27, 2019 15:35

Remove problem assertions that have been fixed.

9f1c7b7

johnno1962 closed this Feb 12, 2020

johnno1962 mentioned this pull request May 5, 2022

[SR-11863] A few SourceKit assertions firing, testing with broad corpus of code swiftlang/sourcekit-lsp#513

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DNM][SourceKit] Log attempts to mangle a module instead of llvm_unreachable for SourceKit/LSP #28051

[DNM][SourceKit] Log attempts to mangle a module instead of llvm_unreachable for SourceKit/LSP #28051

johnno1962 commented Nov 4, 2019 •

edited

Loading

nathawes commented Nov 5, 2019

johnno1962 commented Nov 5, 2019 •

edited

Loading

johnno1962 commented Nov 5, 2019 •

edited

Loading

johnno1962 commented Nov 5, 2019 •

edited

Loading

johnno1962 commented Nov 11, 2019 •

edited

Loading

benlangmuir commented Nov 11, 2019

johnno1962 commented Nov 11, 2019 •

edited

Loading

benlangmuir commented Nov 11, 2019

johnno1962 commented Nov 11, 2019 •

edited

Loading

benlangmuir commented Nov 11, 2019

johnno1962 commented Nov 11, 2019 •

edited

Loading

johnno1962 commented Nov 12, 2019 •

edited

Loading

johnno1962 commented Nov 19, 2019

nathawes commented Nov 19, 2019

johnno1962 commented Nov 19, 2019

nathawes commented Nov 19, 2019

johnno1962 commented Nov 19, 2019 •

edited

Loading

CodaFi commented Nov 27, 2019

johnno1962 commented Nov 27, 2019

CodaFi commented Nov 27, 2019

johnno1962 commented Nov 27, 2019 •

edited

Loading

johnno1962 commented Feb 12, 2020 •

edited

Loading

nathawes commented Feb 12, 2020

johnno1962 commented Feb 12, 2020

nathawes commented Feb 12, 2020

johnno1962 commented Feb 12, 2020

benlangmuir commented Feb 12, 2020

johnno1962 commented Feb 12, 2020

[DNM][SourceKit] Log attempts to mangle a module instead of llvm_unreachable for SourceKit/LSP #28051

[DNM][SourceKit] Log attempts to mangle a module instead of llvm_unreachable for SourceKit/LSP #28051

Conversation

johnno1962 commented Nov 4, 2019 • edited Loading

nathawes commented Nov 5, 2019

johnno1962 commented Nov 5, 2019 • edited Loading

johnno1962 commented Nov 5, 2019 • edited Loading

johnno1962 commented Nov 5, 2019 • edited Loading

johnno1962 commented Nov 11, 2019 • edited Loading

benlangmuir commented Nov 11, 2019

johnno1962 commented Nov 11, 2019 • edited Loading

benlangmuir commented Nov 11, 2019

johnno1962 commented Nov 11, 2019 • edited Loading

benlangmuir commented Nov 11, 2019

johnno1962 commented Nov 11, 2019 • edited Loading

johnno1962 commented Nov 12, 2019 • edited Loading

johnno1962 commented Nov 19, 2019

nathawes commented Nov 19, 2019

johnno1962 commented Nov 19, 2019

nathawes commented Nov 19, 2019

johnno1962 commented Nov 19, 2019 • edited Loading

CodaFi commented Nov 27, 2019

johnno1962 commented Nov 27, 2019

CodaFi commented Nov 27, 2019

johnno1962 commented Nov 27, 2019 • edited Loading

johnno1962 commented Feb 12, 2020 • edited Loading

nathawes commented Feb 12, 2020

johnno1962 commented Feb 12, 2020

nathawes commented Feb 12, 2020

johnno1962 commented Feb 12, 2020

benlangmuir commented Feb 12, 2020

johnno1962 commented Feb 12, 2020

johnno1962 commented Nov 4, 2019 •

edited

Loading

johnno1962 commented Nov 5, 2019 •

edited

Loading

johnno1962 commented Nov 5, 2019 •

edited

Loading

johnno1962 commented Nov 5, 2019 •

edited

Loading

johnno1962 commented Nov 11, 2019 •

edited

Loading

johnno1962 commented Nov 11, 2019 •

edited

Loading

johnno1962 commented Nov 11, 2019 •

edited

Loading

johnno1962 commented Nov 11, 2019 •

edited

Loading

johnno1962 commented Nov 12, 2019 •

edited

Loading

johnno1962 commented Nov 19, 2019 •

edited

Loading

johnno1962 commented Nov 27, 2019 •

edited

Loading

johnno1962 commented Feb 12, 2020 •

edited

Loading