Some tests failing on Swift 3.1 for Linux

[norio-nomura]

We use docker image from https://hub.docker.com/r/norionomura/sourcekit/ for running SourceKitten on Linux.
On that repository, the image built from swift-3.1-RELEASE seems to be broken.
We disable failing tests on Linux until the issue will be fixed. #352

[norio-nomura]

Error output:

$ # SOURCEKIT_LOGGING=3 swift test -s SourceKittenFrameworkTests.FileTests/testFormat
Test Suite 'Selected tests' started at 00:16:45.980
Test Suite 'FileTests' started at 00:16:45.981
Test Case 'FileTests.testFormat' started at 00:16:45.982
sourcekit: [2:sourcekitd_initialize: 0.0000] initializing
sourcekit: [2:handleRequest-before: 0.0010] {
  key.request: source.request.editor.open,
  key.name: "/Users/norio/github/swift-dev/SourceKitten/Tests/SourceKittenFrameworkTests/Fixtures/BicycleUnformatted.swift",
  key.sourcefile: "/Users/norio/github/swift-dev/SourceKitten/Tests/SourceKittenFrameworkTests/Fixtures/BicycleUnformatted.swift"
}
SOURCEKITD FATAL ERROR: sourcekitd object did not resolve to a known type

backtrace on lldb:

root@9d01a4a22f2f:/Users/norio/github/swift-dev/SourceKitten# lldb .build/debug/SourceKittenPackageTests.xctest SourceKittenFrameworkTests.FileTests/testFormat
(lldb) target create ".build/debug/SourceKittenPackageTests.xctest"
Current executable set to '.build/debug/SourceKittenPackageTests.xctest' (x86_64).
(lldb) settings set -- target.run-args  "SourceKittenFrameworkTests.FileTests/testFormat"
(lldb) process launch
Process 44 launched: '/Users/norio/github/swift-dev/SourceKitten/.build/debug/SourceKittenPackageTests.xctest' (x86_64)
Test Suite 'Selected tests' started at 00:41:32.246
Test Suite 'FileTests' started at 00:41:32.247
Test Case 'FileTests.testFormat' started at 00:41:32.247
SOURCEKITD FATAL ERROR: sourcekitd object did not resolve to a known type
Process 44 stopped
* thread #1, name = 'SourceKittenPac', stop reason = signal SIGABRT
    frame #0: 0x00007ffff5a4a428 libc.so.6`gsignal + 56
libc.so.6`gsignal:
->  0x7ffff5a4a428 <+56>: cmpq   $-0x1000, %rax            ; imm = 0xF000 
    0x7ffff5a4a42e <+62>: ja     0x7ffff5a4a450            ; <+96>
    0x7ffff5a4a430 <+64>: rep    
    0x7ffff5a4a431 <+65>: retq   
(lldb) bt
warning: (x86_64) /Users/norio/github/swift-dev/SourceKitten/.build/debug/SourceKittenPackageTests.xctest unable to load swift module 'SourceKittenPackageTests' (failed to get module 'SourceKittenPackageTests' from AST context:
/usr/lib/swift/CoreFoundation/CoreFoundation.h:25:10: note: while building module 'SwiftGlibc' imported from /usr/lib/swift/CoreFoundation/CoreFoundation.h:25:
#include <sys/types.h>
         ^

<module-includes>:3:10: note: in file included from <module-includes>:3:
#include "///usr/include/utmp.h"
         ^

///usr/include/utmp.h:23:10: note: in file included from ///usr/include/utmp.h:23:
#include <sys/types.h>
         ^

error: /usr/include/x86_64-linux-gnu/sys/types.h:146:10: error: 'stddef.h' file not found
#include <stddef.h>
         ^

<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "CoreFoundation.h"
         ^

error: /usr/lib/swift/CoreFoundation/CoreFoundation.h:25:10: error: could not build module 'SwiftGlibc'
#include <sys/types.h>
         ^

/usr/lib/swift/CoreFoundation/CFStream.h:20:10: note: while building module 'CDispatch' imported from /usr/lib/swift/CoreFoundation/CFStream.h:20:
#include <dispatch/dispatch.h>
         ^

<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "dispatch.h"
         ^

/usr/lib/swift/dispatch/dispatch.h:42:10: note: in file included from /usr/lib/swift/dispatch/dispatch.h:42:
#include <sys/cdefs.h>
         ^

error: /usr/include/x86_64-linux-gnu/sys/cdefs.h:23:11: error: could not build module 'SwiftGlibc'
# include <features.h>
          ^

error: could not build Objective-C module 'CoreFoundation'
)
* thread #1, name = 'SourceKittenPac', stop reason = signal SIGABRT
  * frame #0: 0x00007ffff5a4a428 libc.so.6`gsignal + 56
    frame #1: 0x00007ffff5a4c02a libc.so.6`abort + 362
    frame #2: 0x00007fffea809421 libsourcekitdInProc.so`fatal_error_handler(void*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) + 33
    frame #3: 0x00007fffea8148d8 libsourcekitdInProc.so`llvm::report_fatal_error(llvm::Twine const&, bool) + 280
    frame #4: 0x00007fffea8147b4 libsourcekitdInProc.so`llvm::report_fatal_error(char const*, bool) + 180
    frame #5: 0x00007fffea80d52e libsourcekitdInProc.so`(anonymous namespace)::SKDCustomData::getVariantType() const + 30
    frame #6: 0x00007fffea805afd libsourcekitdInProc.so`sourcekitd_variant_get_type + 45
    frame #7: 0x0000000000519453 SourceKittenPackageTests.xctest`fromSourceKit(sourcekitObject=<unavailable>) -> SourceKitRepresentable? at Request.swift:60
    frame #8: 0x000000000051a643 SourceKittenPackageTests.xctest`(key=<unavailable>, value=<unavailable>, context=<unavailable>) -> SourceKitRepresentable?).(closure #2).(closure #1) at Request.swift:77
    frame #9: 0x000000000051aa8a SourceKittenPackageTests.xctest`@objc (fromSourceKit(sourcekitd_variant_t) -> SourceKitRepresentable?).(closure #2).(closure #1) at Request.swift:0
    frame #10: 0x00007fffea806777 libsourcekitdInProc.so`__sourcekitd_variant_dictionary_apply_f_block_invoke + 39
    frame #11: 0x00007fffea80da8b libsourcekitdInProc.so`SKDVar_dictionary_apply(sourcekitd_variant_t, bool (sourcekitd_uid_s*, sourcekitd_variant_t) block_pointer) + 171
    frame #12: 0x00007fffea806748 libsourcekitdInProc.so`sourcekitd_variant_dictionary_apply_f + 136
    frame #13: 0x000000000051a4db SourceKittenPackageTests.xctest`(dictPtr=<unavailable>, sourcekitObject=<unavailable>) -> SourceKitRepresentable?).(closure #2) at Request.swift:82
    frame #14: 0x000000000051aaca SourceKittenPackageTests.xctest`thunk at Request.swift:0
    frame #15: 0x000000000052f199 SourceKittenPackageTests.xctest`partial apply for thunk at Request.swift:0
    frame #16: 0x00007ffff7940498 libswiftCore.so`Swift.withUnsafePointer <A, B> (to : inout A, (Swift.UnsafePointer<A>) throws -> B) throws -> B + 24
    frame #17: 0x00007ffff7a738e4 libswiftCore.so`Swift.withUnsafeMutablePointer <A, B> (to : inout A, (Swift.UnsafeMutablePointer<A>) throws -> B) throws -> B + 20
    frame #18: 0x00000000005197f6 SourceKittenPackageTests.xctest`fromSourceKit(sourcekitObject=<unavailable>) -> SourceKitRepresentable? at Request.swift:83
    frame #19: 0x0000000000525c1d SourceKittenPackageTests.xctest`Request.send(self=<unavailable>) -> [String : SourceKitRepresentable] at Request.swift:368
    frame #20: 0x00000000004f7afa SourceKittenPackageTests.xctest`File.format(trimmingTrailingWhitespace=<unavailable>, useTabs=<unavailable>, indentWidth=<unavailable>, self=<unavailable>) -> String at File.swift:63
    frame #21: 0x000000000041bb60 SourceKittenPackageTests.xctest`FileTests.testFormat(self=<unavailable>) -> () at FileTests.swift:22
    frame #22: 0x000000000041cc0a SourceKittenPackageTests.xctest`partial apply at FileTests.swift:0
    frame #23: 0x000000000041a198 SourceKittenPackageTests.xctest`thunk at CodeCompletionTests.swift:0
    frame #24: 0x000000000041ccb9 SourceKittenPackageTests.xctest`partial apply for thunk at FileTests.swift:0
    frame #25: 0x00007ffff7f63412 libXCTest.so`partial apply forwarder for reabstraction thunk helper from @callee_owned (@in ()) -> (@out (), @error @owned Swift.Error) to @callee_owned () -> (@error @owned Swift.Error) + 50
    frame #26: 0x00007ffff7f412b2 libXCTest.so`XCTest.((test in _3BE257A46ADB477C7BF2D39968B39F9D) <A where A: XCTest.XCTestCase> ((A) -> () throws -> ()) -> (XCTest.XCTestCase) throws -> ()).(closure #1) + 82
    frame #27: 0x00007ffff7f63240 libXCTest.so`partial apply forwarder for reabstraction thunk helper from @callee_owned (@owned XCTest.XCTestCase) -> (@error @owned Swift.Error) to @callee_owned (@in XCTest.XCTestCase) -> (@out (), @error @owned Swift.Error) + 64
    frame #28: 0x00007ffff7f6379e libXCTest.so`partial apply forwarder for reabstraction thunk helper from @callee_owned (@owned XCTest.XCTestCase) -> (@error @owned Swift.Error) to @callee_owned (@in XCTest.XCTestCase) -> (@out (), @error @owned Swift.Error) with unmangled suffix ".111" + 14
    frame #29: 0x00007ffff7f63480 libXCTest.so`partial apply forwarder for reabstraction thunk helper from @callee_owned (@in XCTest.XCTestCase) -> (@out (), @error @owned Swift.Error) to @callee_owned (@owned XCTest.XCTestCase) -> (@error @owned Swift.Error) + 64
    frame #30: 0x00007ffff7f40940 libXCTest.so`XCTest.XCTestCase.invokeTest () -> () + 112
    frame #31: 0x00007ffff7f56549 libXCTest.so`function signature specialization <Arg[0] = Owned To Guaranteed> of XCTest.XCTestCase.perform (XCTest.XCTestRun) -> () + 313
    frame #32: 0x00007ffff7f4080e libXCTest.so`XCTest.XCTestCase.perform (XCTest.XCTestRun) -> () + 14
    frame #33: 0x00007ffff7f3fc64 libXCTest.so`XCTest.XCTest.run () -> () + 516
    frame #34: 0x00007ffff7f56abb libXCTest.so`function signature specialization <Arg[0] = Owned To Guaranteed> of XCTest.XCTestSuite.perform (XCTest.XCTestRun) -> () + 1067
    frame #35: 0x00007ffff7f56d5b libXCTest.so`function signature specialization <Arg[0] = Owned To Guaranteed> of XCTest.XCTestSuite.perform (XCTest.XCTestRun) -> () + 1739
    frame #36: 0x00007ffff7f338e5 libXCTest.so`XCTest.XCTMain (Swift.Array<(testCaseClass : XCTest.XCTestCase.Type, allTests : Swift.Array<(Swift.String, (XCTest.XCTestCase) throws -> ())>)>) -> Swift.Never + 4069
    frame #37: 0x000000000041907a SourceKittenPackageTests.xctest`main at LinuxMain.swift:16
    frame #38: 0x00007ffff5a35830 libc.so.6`__libc_start_main + 240
    frame #39: 0x0000000000418db9 SourceKittenPackageTests.xctest`_start + 41
(lldb) 

[norio-nomura]

SourceKit on Swift 3.1 for Linux fails many tests in Swift project. norio-nomura/swift@d94c617

[AxelCB]

@norio-nomura, would the recent Swift 3.1.1 release fix any of this failing tests?

[norio-nomura]

No, on official branches.
SourceKit on Linux has been fixed by swiftlang/swift#9025 on master branch of apple/swift, but not on swift-3.1-branch branch of apple/swift.

Yes, on testing SourceKitten.
I have a branch forked from swift-3.1.1-RELEASE that has backported changes from swiftlang/swift#9025.
SourceKitten uses docker image norionomura/sourcekit:311 built from the forked branch.

[AxelCB]

Thanks for the quick and detailed answer!

[norio-nomura]

Closing this since we have fixed all crashes on filing this issue.