Separate diagnostic statistics from diagnostics needed in the current file(s)

[mdbergmann]

Is your feature request related to a problem? Please describe.
I'm using Metals/Bloop as the tool of choice for a Scala project. This project has many diagnostics, maybe because of Scapegoat compiler plugin which produces a ton of infos. After opening the project I have to wait ~5-10 minutes until all diagnostics are sent to the LSP client. I'm not fully sure what it all sends. I see many diagnostics for 'unvisited files'. Until that is finished working is unstable. Often I have to terminate java processes related to Metals/Bloop to get a stable work environment, i.e. when the LSP client doesn't get a response, resulting in timeouts or completion not working, etc.

Full diagnostics with description texts, etc. are not exactly light weight. For me (and I would be surprised if I would be the only one), I'm only interested in the diagnostics for the file I have open and/or am working in actively.
Having all the other diagnostics cached by the editor seems like a waste of resources to me, in particular when having a local connection where diagnostics can be quickly loaded on demand.

Describe the solution you'd like
I'd like to have a separation between something like diagnostic statistics and the full diagnostics needed for the open files and/or the files that are actively being worked on.

Diagnostic statistics are basically a summary containing how many info, warn and errors a file has. This information can be used in a file explorer, or some other view (depending on the LSP client) that can show information on a per file basis which is not the text editor pane as such. If something like this actually exists.
If the user actually drills down to a particular file the full diagnostics can be loaded.

Full diagnostics are only loaded for open files, or file that are actively being worked on.

Describe alternatives you've considered
I'm not sure if something like the proposed diagnostic statistics actually exist in the LSP protocol.
If not I would still vote for not loading/sending all diagnostics to the LSP client.

Additional context

Search terms:

[tgodzik]

Thanks for reporting! I think this might be useful also in cases that a project has a lot of errors, but I am curious to hear other opinions.

What we would need to do here is when sending diagnostics:

• check if a file is open
• if it is then just send diagnostics as is
• if it's not, create a summary of those diagnostics, send it and cache the diagnostics until the file is open

I can imagine though that if we don't have too many errors we might be interested in all of them, so maybe we could apply some kind of heuristic? If we have less than 100 errors/warnings/info just send all of it, otherwise apply the algorithm above.

[mdbergmann]

This sounds OK.

Depends a bit in which direction this LSP goes. Is it really fit for more IDE qualities?
When I think about IntelliJ IDEA, which I work with for ~15 years and still do. Eventually you only know if all compiles when you 'make' the project. It's possible to configure building when changes were detected. But I found that this is just too slow and CPU is permanently at a high rate.
I'd imagine Metals/Bloop is doing something similar upon saving a file, but maybe more efficient.
The files that are open and under development need all diagnostics know to the LSP client.
The rest is a fuzzy business unless you really recompile all.