Startup time is slow when there are many installed databases

[aeisenberg]

When there are many of installed databases in the workspace, extension startup time is slow. This issue is about determining the root cause of this slowness and then fixing it.

[aeisenberg]

Adding the "Good first issue" label since determining what the problem is will be helpful even if the solution is quite complicated.

[nickrolfe]

I'm not sure it's related to the number of databases, necessarily. I just observed the extension taking ~50 seconds to finish loading with only a single (large) database installed.

Even with zero databases installed, it takes ~30 seconds for the extension to finish loading my query history. That may or may not be a separate problem.

[aeisenberg]

It's possible that the slowness is related to loading the source archive file system folders. Is it possible that your workspace with 0 databases still had some source archive folders?

[nickrolfe]

Ah, yes, I see almost 20 source archive folders in the Explorer pane.

[aeisenberg]

We had a discussion internally about the recently. Transcribing the interesting parts of the discussion here.

• Person A: I noticed that CodeQL extension spent a long time loading databases on startup just not. At least 30+ seconds with my 14 databases :thinking_face: is there some known slowdown for this?

• Person B: I had a look before. The problem is we add a virtual filesystem for each database and vscode decides to immediately scan it. This involves unzipping the source archive and takes time and as JS is single threaded this blocks everything else. The solution would be to either delay responding until things are loaded (not great but may be better UI, still blocks all extensions at some point) or move the work to a worker.

• Person C: That makes sense. So, it's not really about the number of databases, but the size of the archives.
  We could also delay scanning the archive until the user actually tries to open it. I wonder if we could be even lazier and only scan individual directories on request. I'd have to take a deeper look at the implementation to see if this is possible.

[aeisenberg]

@stonish I think this is something more appropriate for your team now. It may already be getting looked at with the improvements to single-rooted workspaces. I'm going to take this off of our planning board. I'm not sure how you want to track this now.

[trganda]

We had a discussion internally about the recently. Transcribing the interesting parts of the discussion here.

Looks like this extension will scan no related files on disks to find the QL pack.

I'm using the default setting, version info:

• vscode-codeql: v1.10.0
• vscode: 1.84.2 (Universal)
• os: m1max mac

I have cloned https://github.com/github/vscode-codeql-starter in a folder ~/Projects/vscode-codeql-starter, after open it with the vscode as workspace and waiting for a long time, I got this error info in OUTPU: CodeQL Language Server (And the system ask me whether to enable visual studio code to access data from other apps, I click the allowed)

Starting CodeQL language server using CodeQL CLI: /Users/trganda/.codeql/codeql execute language-server --check-errors ON_CHANGE -v --log-to-stderr
CodeQL language server started on PID: 42684
[2023-11-30 14:14:28] This is codeql execute language-server --check-errors ON_CHANGE -v --log-to-stderr
[2023-11-30 14:16:23] [WARN] IO exception occurred while scanning directory /Users/trganda/Projects/tools/jwt/hashcat-6.2.6/docs/license_libs
                             java.nio.file.AccessDeniedException: /Users/trganda/Projects/tools/jwt/hashcat-6.2.6/docs/license_libs/LZMA_SDK_LICENSE.txt
                             	at java.base/sun.nio.fs.UnixException.translateToIOException(Unknown Source)
                             	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source)
                             	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source)
                             	at java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(Unknown Source)
                             	at java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(Unknown Source)
                             	at java.base/java.nio.file.Files.readAttributes(Unknown Source)
                             	at java.base/java.nio.file.FileTreeWalker.getAttributes(Unknown Source)
                             	at java.base/java.nio.file.FileTreeWalker.visit(Unknown Source)
                             	at java.base/java.nio.file.FileTreeWalker.next(Unknown Source)
                             	at java.base/java.nio.file.Files.walkFileTree(Unknown Source)
                             	at com.semmle.frontend.packs.DiskFileSystem.directChildren(DiskFileSystem.java:56)
                             	at com.semmle.frontend.server.QLLanguageServerVFS.directChildren(QLLanguageServerVFS.java:159)
                             	at com.semmle.frontend.packs.glob.PathMatchTreeWalker.pushStackEntry(PathMatchTreeWalker.java:345)
                             	at com.semmle.frontend.packs.glob.PathMatchTreeWalker.matchDirectory(PathMatchTreeWalker.java:205)
                             	at com.semmle.frontend.packs.glob.PathMatchTreeWalker.walk(PathMatchTreeWalker.java:121)
                             	at com.semmle.frontend.packs.glob.PathMatcher.walkFileTree(PathMatcher.java:275)
                             	at com.semmle.frontend.packs.ScanByManifests.scanTree(ScanByManifests.java:354)
                             	at com.semmle.frontend.packs.ScanByManifests.scanForPacksAndWorkspaces(ScanByManifests.java:154)
                             	at com.semmle.frontend.packs.QlPackResolver.scanForPacksAndWorkspaces(QlPackResolver.java:107)
                             	at com.semmle.frontend.packs.QlPackContainer.computePacksAndWorkspaces(QlPackContainer.java:119)
                             	at com.semmle.frontend.packs.QlPackContainer.lambda$new$0(QlPackContainer.java:38)
                             	at com.semmle.api.compilation.CachedOperation.get(CachedOperation.java:91)
                             	at com.semmle.frontend.packs.QlPackContainer.collectDescendantPacks(QlPackContainer.java:104)
                             	at com.semmle.frontend.packs.QlPackContainer.getAllDescendantPacks(QlPackContainer.java:91)
                             	at com.semmle.frontend.packs.QlPackSearchContext.lambda$findAllPacks$8(QlPackSearchContext.java:195)
                             	at java.base/java.util.Optional.ifPresent(Unknown Source)
                             	at com.semmle.frontend.packs.QlPackSearchContext.findAllPacks(QlPackSearchContext.java:219)
                             	at com.semmle.frontend.packs.QlPackSearchContext.findAllPacks(QlPackSearchContext.java:175)
                             	at com.semmle.frontend.packs.QLPackLibraryPathResolver.resolveAllPacks(QLPackLibraryPathResolver.java:66)
                             	at com.semmle.frontend.project.PackBasedBindingState.reloadPackConfig(PackBasedBindingState.java:201)
                             	at com.semmle.frontend.project.PackBasedBindingState.setWorkspaceRoots(PackBasedBindingState.java:303)
                             	at com.semmle.frontend.server.QLLanguageServer.initialize(QLLanguageServer.java:213)
                             	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                             	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
                             	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
                             	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
                             	at org.eclipse.lsp4j.jsonrpc.services.GenericEndpoint.lambda$null$0(GenericEndpoint.java:65)
                             	at org.eclipse.lsp4j.jsonrpc.services.GenericEndpoint.request(GenericEndpoint.java:120)
                             	at org.eclipse.lsp4j.jsonrpc.RemoteEndpoint.handleRequest(RemoteEndpoint.java:261)
                             	at org.eclipse.lsp4j.jsonrpc.RemoteEndpoint.consume(RemoteEndpoint.java:190)
                             	at org.eclipse.lsp4j.jsonrpc.json.StreamMessageProducer.handleMessage(StreamMessageProducer.java:194)
                             	at org.eclipse.lsp4j.jsonrpc.json.StreamMessageProducer.listen(StreamMessageProducer.java:94)
                             	at org.eclipse.lsp4j.jsonrpc.json.ConcurrentMessageProcessor.run(ConcurrentMessageProcessor.java:113)
                             	at com.semmle.queryserver.client.JsonRpcLauncher.lambda$startListening$0(JsonRpcLauncher.java:49)
                             	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown Source)
                             	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
                             	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
                             	at java.base/java.lang.Thread.run(Unknown Source)
...

From the stack tree, this extension was trying to find all QL packs from my disk in others folder /Users/trganda/Projects/tools/jwt/hashcat-6.2.6 (a tools download from https://github.com/hashcat/hashcat/releases/tag/v6.2.6) and unfortunately encountered an error, it's a strange behavior.

But I have already initialized the vscode-codeql-starter and cloned the submodule, the QL packs should be in the subfolder ql.

I'm not sure why this extension will scan other folders, but this will take a long time since I have so many files in my home folder. Btw, this operation will make me feel uncomfortable.

The logging file was here, CodeQL Extension Log.txt

[aeisenberg]

@trganda I think your issue is something different. You have installed codeql in your home directory. For historical reasons, when looking for qlpacks, codeql will search sibling directories of its installed location. This means that running some codeql commands will involve traversing folders in your home directory.

Please try moving the codeql instance one level down. Eg from /Users/trganda/.codeql to something like /Users/trganda/.codeql/v2.15.2.

[trganda]

Please try moving the codeql instance one level down. Eg from /Users/trganda/.codeql to something like /Users/trganda/.codeql/v2.15.2.

It's work and the start-up being so fast. Tks :)