Language Server Crash

[decimad]

Hello,

after updating to my local build to commit c3acda0798f9b10ac3187ad941bbd8af82fb84a1 I'm experiencing unconditional crashes of the clangd language server. I'm running on windows and building clangd with MSVC++ 16.8.3.

Here's the clangd's error information (debug build):

 #0 0x00007ff744c85b68 llvm::StringMapEntryStorage<bool>::setValue(bool const &) E:\projects\3rdparty-build\llvm-project\llvm\include\llvm\ADT\StringMapEntry.h:53:0
 #1 0x00007ff744c7415a clang::clangd::DirectoryBasedGlobalCompilationDatabase::broadcastCDB(struct clang::clangd::DirectoryBasedGlobalCompilationDatabase::CDBLookupResult) const E:\projects\3rdparty-build\llvm-project\clang-tools-extra\clangd\GlobalCompilationDatabase.cpp:518:0
 #2 0x00007ff744c73c69 clang::clangd::DirectoryBasedGlobalCompilationDatabase::lookupCDB(struct clang::clangd::DirectoryBasedGlobalCompilationDatabase::CDBLookupRequest) const E:\projects\3rdparty-build\llvm-project\clang-tools-extra\clangd\GlobalCompilationDatabase.cpp:478:0
 #3 0x00007ff744c731fc clang::clangd::DirectoryBasedGlobalCompilationDatabase::getCompileCommand(class llvm::StringRef) const E:\projects\3rdparty-build\llvm-project\clang-tools-extra\clangd\GlobalCompilationDatabase.cpp:377:0
 #4 0x00007ff744c74671 clang::clangd::OverlayCDB::getCompileCommand(class llvm::StringRef) const E:\projects\3rdparty-build\llvm-project\clang-tools-extra\clangd\GlobalCompilationDatabase.cpp:574:0
 #5 0x00007ff744b0b9b2 <lambda_79a7e5852a2b031f5703c3c9ddd8a1a8>::operator() E:\projects\3rdparty-build\llvm-project\clang-tools-extra\clangd\TUScheduler.cpp:623:0
 #6 0x00007ff744af66b2 llvm::detail::UniqueFunctionBase<void>::CallImpl<class <lambda_79a7e5852a2b031f5703c3c9ddd8a1a8>>(void *) E:\projects\3rdparty-build\llvm-project\llvm\include\llvm\ADT\FunctionExtras.h:196:0
 #7 0x00007ff74364549e llvm::unique_function<(void)>::operator()(void) E:\projects\3rdparty-build\llvm-project\llvm\include\llvm\ADT\FunctionExtras.h:355:0
 #8 0x00007ff744aeef77 clang::clangd::`anonymous namespace'::ASTWorker::run E:\projects\3rdparty-build\llvm-project\clang-tools-extra\clangd\TUScheduler.cpp:1083:0
 #9 0x00007ff744b0c3c3 <lambda_8496735dcc1912dcaf8ff2b15ebf07d0>::operator() E:\projects\3rdparty-build\llvm-project\clang-tools-extra\clangd\TUScheduler.cpp:579:0
#10 0x00007ff744af6702 llvm::detail::UniqueFunctionBase<void>::CallImpl<class <lambda_8496735dcc1912dcaf8ff2b15ebf07d0>>(void *) E:\projects\3rdparty-build\llvm-project\llvm\include\llvm\ADT\FunctionExtras.h:196:0
#11 0x00007ff74364549e llvm::unique_function<(void)>::operator()(void) E:\projects\3rdparty-build\llvm-project\llvm\include\llvm\ADT\FunctionExtras.h:355:0
#12 0x00007ff74510e2d3 <lambda_3b9785320f1cf733474d10195abe6ef8>::operator() E:\projects\3rdparty-build\llvm-project\clang-tools-extra\clangd\support\Threading.cpp:94:0
#13 0x00007ff74510d432 llvm::detail::UniqueFunctionBase<void>::CallImpl<class <lambda_3b9785320f1cf733474d10195abe6ef8>>(void *) E:\projects\3rdparty-build\llvm-project\llvm\include\llvm\ADT\FunctionExtras.h:196:0
#14 0x00007ff74364549e llvm::unique_function<(void)>::operator()(void) E:\projects\3rdparty-build\llvm-project\llvm\include\llvm\ADT\FunctionExtras.h:355:0
#15 0x00007ff74375c9df threadFuncAsync E:\projects\3rdparty-build\llvm-project\llvm\lib\Support\Windows\Threading.inc:35:0
#16 0x00007ff8118d542c (C:\WINDO[Error - 15:39:21] Connection to server got closed. Server will not be restarted.

If you can guide me how to debug this issue further, I will happily try to do so. The crashes happen so quickly that I cannot attach the debugger before it happens (I suppose there is a non-naive way to go about this).

Update bisect is underway

[njames93]

Do you get any assertions firing? Also can you disable background indexing and see if the crash still happens

[decimad]

No assertions, turning off Background-indexing did not prevent the crashes. I will report back once I'm done bisecting... This takes roughly 10 minutes per step (about 6 remaining) as for some reason I always get full rebuilds on every step... :(

[decimad]

Commit 92dd077af1ff89929f5502c6c887358f51d5afc1 introduced the crash.

[decimad]

If I change GlobalCompilationDatabase.cpp:333 to actually validate that find() did not return end() before dereferencing it, the crash is gone.

[sam-mccall]

Thanks! The entry is supposed to always be in the map, I think this is a case-sensitivity error.
All FileAncestors are necessarily in DirectoryHasCDB. However we do the lookup using Dir->path() which may not be the exact FileAncestor used to look it up, as the lookup ignores case (on windows + mac).

[sam-mccall]

Does llvm/llvm-project@74b3ace fix this?

[decimad]

Yes, the crash is gone, thank you!
Now I can really try to set up a minimal repro of my template code crashing the server, before which I wanted to update the sources to trunk :D

Btw, who is in charge of the VS-Code-Plugin? There seem to be issues with the CDB-Update-Detection and language server restarting. For one, the indexing-indicator does not seem to be disposed if the current indexing run has not yet completed, leading to an endless indicator.

[njames93]

Yes, the crash is gone, thank you!
Now I can really try to set up a minimal repro of my template code crashing the server, before which I wanted to update the sources to trunk :D

Btw, who is in charge of the VS-Code-Plugin? There seem to be issues with the CDB-Update-Detection and language server restarting. For one, the indexing-indicator does not seem to be disposed if the current indexing run has not yet completed, leading to an endless indicator.

That bug is in apparently in vscode and pertains to all language server implementations that use progress bars not being disposed when servers are unexpectedly terminated.

[njames93]

@decimad That bug has now been fixed and will land next release of the vscode plugin

[decimad]

@njames93: thanks for the good news!

[decimad]

Closing this because llvm/llvm-project@74b3ace resolved the crash from the initial report.