Issues on Windows Swift 5.9

[adam-fowler]

The vscode-swift repo has received a number of issues related to sourcekit-lsp crashing on Windows with swift 5.9. These include

swift-server/vscode-swift#612 high CPU usage at start up and then auto-complete not working
swift-server/vscode-swift#646 SourceKit-LSP stops responding after a few seconds
swift-server/vscode-swift#561 SourceKit Language Server emits errors, autocomplete doesn't work

[ahoppen]

Tracked in Apple’s issue tracker as rdar://118923500

[ahoppen]

TL;DR: It appears that some requests like code completion can sometimes produce very large responses that exceed 64KB and for some reason the transport layer cannot handle this (still to be investigated).

As a workaround you can add the following to settings.json, which limits the response size and should work around the issue that code completion hits this maximum response size limit.

"swift.sourcekit-lsp.serverArguments": ["--code-completion-max-results", "100"]

---

Let’s use swift-server/vscode-swift#646 as the basis for this issue because it is the most reduced example and I have been able to reproduce it.

While writing this, this has migrated from being a response to more of a journal of my debug process.

Reproducer

I was able to reproduce the issue by having a single Swift file (not a Swift package), that contains

import Foundation

let a = "".

My settings are as follows

{
  "swift.path": "C:\\Library\\Developer\\Toolchains\\unknown-Asserts-development.xctoolchain\\usr\\bin",
  "swift.SDK": "C:\\Library\\Developer\\Platforms\\Windows.platform\\Developer\\SDKs\\Windows.sdk",
  "swift.swiftEnvironmentVariables": {
    "SOURCEKIT_LOGGING": "3"
  }
}

Installed Swift version: 5.9.1

Open test.swift and invoke code completion. Notice that it never returns any results.

Analysis

Looking at the LSP log output, there three notable pieces:

1. sourcekitd returns a response for code completion
2. SourceKit-LSP logs that it has delivered a code completion response.
3. After SourceKit-LSP logs that it has delivered the output, VS Code still sends a cancel notification for the code completion request.

This leads me to believe that there’s something funny going on with the process communication between sourcekit-lsp and VS Code and for some reason sourcekit-lsp can’t send any more information to VS Code. All following requests fail in the same way. For example, if you change the contents of the file to

struct Foo {}
let x: Foo

And go to definition on Foo, you can see that sourcekit-lsp logs a correct response to the definition request but VS Code never reacts to it.

Interestingly, stderr continues to flow from sourcekit-lsp to VS Code as is evident from the fact that we can still see the log messages.

Possible failure reasons

AFAICT the communication channel could break in two ways:

1. The encoding of the response on the sourcekit-lsp side is broken
   a. Invalid Content-Length header or header whose length doesn’t match the actual data length. Thus VS Code is unable to parse any later message as well.
   b. The JSON is not encoded correctly (somehow JSONEncoder produces invalid JSON)
   c. Two replies are sent simultaneously and we race in JSONRPCConnection and interleave these replies
2. There is a bug in DispatchIO that causes the pipe-equivalent on Windows to fill up (or something along those lines) and we thus can’t send any additional data.

Limiting number of code completion results

When limiting the code completion results by adding the following snippet to settings.json, the issue stopped occurring.

"swift.sourcekit-lsp.serverArguments": ["--code-completion-max-results", "134"]

Inspecting the 135th result in the code completion response revealed … that it wasn’t anything special, no Unicode characters or anything.

With that insight, I was also able to reproduce the issue by setting the max code completion results to 500 and completing after Da on the top level when importing Foundation.

So it appears that we are exceeding some limit that we can send to VS Code. And indeed, it turns out that the 135 results for the string completion take 63.97KB. If we add the Content-Length header, this gets really close to 64KB, which might be some kind of limit, either imposed by Windows, by DispatchIO or by VS Code.

Checking if we are hitting some OS limit on number of bytes we can send

It doesn’t appear to be a DispatchIO problem because the following program that sends 65KB of as using essentially the same code that sourcekit-lsp is using runs fine

import Dispatch
import struct CDispatch.dispatch_fd_t
import Foundation

let realStdout = dup(STDOUT_FILENO)
if realStdout == -1 {
  fatalError("failed to dup stdout: \(strerror(errno)!)")
}
if dup2(STDERR_FILENO, STDOUT_FILENO) == -1 {
  fatalError("failed to redirect stdout -> stderr: \(strerror(errno)!)")
}
let outFD = FileHandle(fileDescriptor: realStdout, closeOnDealloc: false)

let sendQueue: DispatchQueue = DispatchQueue(label: "jsonrpc-send-queue", qos: .userInitiated)
let rawOutFD = dispatch_fd_t(bitPattern: outFD._handle)

let sendIO = DispatchIO(type: .stream, fileDescriptor: rawOutFD, queue: sendQueue) { (error: Int32) in
  print(error)
}

var dispatchData = DispatchData.empty

let str = String(repeating: "a", count: 65 * 1024)

str.utf8.map{$0}.withUnsafeBytes {
  dispatchData.append($0)
}

let group = DispatchSemaphore(value: 0)

sendIO.write(offset: 0, data: dispatchData, queue: sendQueue) { done, _, errorCode in
  group.signal()
}
group.wait()

Has this issue already been fixed?

• main develoment snapshot: Does not reproduce the issue
• 5.10 development snapshot: Not currently available for Windows
• 5.9 development snapshot (to check if there are difference between development and release toolchains): Was unable to test because launching sourcekit-lsp fails with the following error

Server initialization failed
Message: Pending response rejected since connection got disposed
Code: -32097

Next steps

1. Once we have a 5.10 toolchain for Windows, see if that reproduces the issue. If yes, figure out what exactly we need to cherry-pick from main to release/5.10 to fix this.
2. Get sourcekit-lsp from a development snapshot toolchain working so we can test if it reproduces there. If it does not, figure out if the release toolchains are built somehow differently that causes the Windows issue.

[ahoppen]

According to @compnerd this should be fixed by the following 2 PRs, which would make it a dupe of #752

• Use PIPE_NOWAIT on Windows to avoid blocking on writes swift-corelibs-libdispatch#781
• Prevent infinite attempts to write to Win32 pipes  swift-corelibs-libdispatch#796

[adam-fowler]

@compnerd is there a release with these changes?

[ahoppen]

The main development snapshot should contain the fixes. I also built a 5.9 toolchain with the libdisaptch cherry-pick here and I was able to confirm that it doesn’t have the hang issue anymore: https://ci-external.swift.org/job/swift-PR-build-toolchain-windows/1045/

[ahoppen]

This should be fixed by apple/swift-corelibs-libdispatch#802. Let’s close this issue.