An AssertationError is thrown when opening/analyzing a large project

[lassemaatta]

Describe the bug

I noticed that opening a (very) large project in emacs sometimes causes clojure-lsp to throw an exception after a while.

The exception does not seem to break clojure-lsp, at least not in an obvious way. I can still invoke clojure-lsp functions from emacs and everything seems to work ok.

To Reproduce
Steps to reproduce the behavior:

1. Open a large project in emacs
2. Notice how clojure-lsp eats up all the cpu
3. Wait a few minutes
4. look at the clojure-lsp logs under /tmp and see an exception stack trace

Expected behavior
The exception should probably be handled within clojure-lsp.

Log output

2021-08-05T06:33:20.731Z archlinux DEBUG [clojure-lsp.server:?] - :documentHighlight 1ms
2021-08-05T06:33:20.816Z archlinux DEBUG [clojure-lsp.server:?] - :codeAction 86ms
2021-08-05T06:33:21.722Z archlinux DEBUG [clojure-lsp.server:?] - :codeAction 43ms
2021-08-05T06:33:27.041Z archlinux DEBUG [clojure-lsp.server:?] - :codeAction 88ms
2021-08-05T06:33:31.344Z archlinux DEBUG [clojure-lsp.server:?] - :codeAction 39ms
2021-08-05T06:33:32.387Z archlinux DEBUG [clojure-lsp.server:?] - :codeAction 89ms
2021-08-05T06:33:54.225Z archlinux DEBUG [clojure-lsp.server:?] - :didChangeWatchedFiles 0ms
2021-08-05T06:38:34.785Z archlinux ERROR [taoensso.timbre:796] - Uncaught exception on thread: async-dispatch-6
         com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine    PosixJavaThreads.java:  192
                     com.oracle.svm.core.thread.JavaThreads.threadStartRoutine         JavaThreads.java:  553
                                                          java.lang.Thread.run              Thread.java:  829
                                                                           ...                               
            clojure.core.async.impl.concurrent/counted-thread-factory/reify/fn           concurrent.clj:   29
                            java.util.concurrent.ThreadPoolExecutor$Worker.run  ThreadPoolExecutor.java:  628
                             java.util.concurrent.ThreadPoolExecutor.runWorker  ThreadPoolExecutor.java: 1128
                                                                           ...                               
                         clojure-lsp.feature.diagnostics/post-project-lint!/fn          diagnostics.clj:  131
                  clojure.core.async.impl.ioc-macros/run-state-machine-wrapped           ioc_macros.clj:  982
                          clojure.core.async.impl.ioc-macros/run-state-machine           ioc_macros.clj:  978
   clojure-lsp.feature.diagnostics/post-project-lint!/fn/state-machine--auto--          diagnostics.clj:  131
clojure-lsp.feature.diagnostics/post-project-lint!/fn/state-machine--auto--/fn          diagnostics.clj:  146
                            clojure-lsp.feature.diagnostics/lint-project-files          diagnostics.clj:  117
                                clojure-lsp.feature.diagnostics/sync-lint-file          diagnostics.clj:  101
                                                       clojure.core.async/put!                async.clj:  199
                       clojure.core.async.impl.channels.ManyToManyChannel/put!             channels.clj:  152
java.lang.AssertionError: Assert failed: No more than 1024 pending puts are allowed on a single channel. Consider using a windowed buffer.
                          (< (.size puts) impl/MAX-QUEUE-SIZE)

2021-08-05T06:45:11.574Z archlinux DEBUG [clojure-lsp.server:?] - :didOpen 129ms
2021-08-05T06:45:11.576Z archlinux DEBUG [clojure-lsp.server:?] - :codeLens 0ms
2021-08-05T06:45:11.667Z archlinux DEBUG [clojure-lsp.server:?] - :documentSymbol 0ms
2021-08-05T06:45:11.667Z archlinux DEBUG [clojure-lsp.server:?] - :documentHighlight 0ms
2021-08-05T06:45:11.679Z archlinux DEBUG [clojure-lsp.server:?] - :codeLens 0ms
2021-08-05T06:45:11.872Z archlinux DEBUG [clojure-lsp.server:?] - :codeAction 205ms
2021-08-05T06:45:15.509Z archlinux DEBUG [clojure-lsp.server:?] - :documentHighlight 0ms

User details (please complete the following information):

• OS: Linux archlinux 5.13.7-arch1-1
• Editor: emacs
• Version: clojure-lsp 2021.08.03-13.33.03 | clj-kondo 2021.08.03

[ericdallo]

Thanks for the report, It only affect linting the whole project after startup (which can be disabled with lint-project-files-after-startup?), it seems the channel clojure-lsp uses for diagnostics is reaching an overflow 🤔 I need to investigate the better way to fix it

[ericdallo]

@lassemaatta Fixed on master, would you mind try the fix before next release? you can clone clojure-lsp and run make to generate a clojure-lsp binary on the root. It'll not be compiled wtih GraalVM but it's enough to test if the issue was fixed