Project custom lint diagnostics support

[ericdallo]

Problem

Currently, users and orgs create custom linters via clj-kondo hooks, which is a feature that you teach clj-kondo how a macro behaves, giving the option to register diagnostics during this stage, this already limits us in some ways:

• Clj-kondo hooks API only provide context about the current namespace, so we can’t have custom linters that are aware of multiple files or namespaces, since clj-kondo doesn’t have awareness of a project, only a file.
• For complex custom linters where we just want to add additional behavior to an existing function, we need to use the hooks even if the function is already understood correctly by clj-kondo, kind of messing the purpose of the hooks, especially if we want to add multiple custom linters that need to live under the same hook, like schema.core/defn.
• Some orgs like Nubank, already created > 10 custom linters and can have no metrics of how much time those add to diagnostics feature of clojure-lsp, affecting developer IDE experience time + lint task time on PRs.

clojure-lsp already have all project analysis + extra information like deps hierarchy, classpath and others which could enhance to create more complex linters.
So we can say that current linters are file only linters while we want to support project linters.

Example of custom linters

Each project or org can create their custom linters for their own needs, but to give visibility of what a project awareness can do, here are some examples:

• Ensure public logic/adapter functions have unit tests for a org codebase
• Ensure there are integration tests for a service HTTP endpoints
• Ensure use of a arch of a service (not only ns layers but functions usage across namespaces)

[ericdallo]

FYI @borkdude, this looks like the start to avoid inflating clj-kondo hooks with lint stuff and having that on clojure-lsp, leaving the hooks only for teaching really how a macro behaves

[ericdallo]

Proposed solution

Currently there are 3 linter providers inside clojure-lsp:

• clj-kondo: provide most linters since is the core of clojure-lsp providing all analysis for other features.
• clj-depend: for ns layer enforcement, used for diplomat arch for example.
• clojure-lsp: using clj-kondo custom-lint-fn limited feature to provide more linters given project analysis clojure-lsp has.
  • unused-public-var: reporting unused public functions across the project
  • different-aliases: reporting multiple aliases names of the same namespace across the project.

We would improve clojure-lsp linter capability to allow users to configure extra linters to be executed along the existing ones in clojure-lsp.

The main challenge is the need to execute custom Clojure code provided by the user in runtime when finding diagnostics for the project/file, we can leverage sci to help with that, which is the base of babashka, and the same lib used by clj-kondo hooks to provide similar functionality.

The idea is:

1. User define a custom linter config specifying the severity and any extra params:

.lsp/config.edn or any classpath config

{:linters {:custom {my.custom.linter/lint {:severity :info
                                           :foo 1 
                                           :bar 2}}}}

2. User define the custom linter anywhere in the classpath following clojure-lsp.exports/linters standard:

clojure-lsp.exports/linters/my/custom/linter.clj

(ns my.custom.linter)
(defn lint [{:keys [db params reg-diagnostic! analysis-type]}]
  (reg-diagnostic! {:message "Logic function missing unit test"
                    :source "my-org/codestyle"
                    :code "my-org/missing-unit-test"
                    :range {:start {:line 1 :character 2} :end {:line 3 :character 4}}
                    :severity (:severity params)
                    :uri "..."}))

3. When clojure-lsp analyze a project or file, it will consider that linter calling it passing its db and other important data.

We should metrify with logs how much time took to call each custom lint.

@acamargo did a really good PoC here using sci to support this custom project linters.

[borkdude]

sounds good!

[NoahTheDuke]

The linters would be forced to use sci? This might preclude someone writing a typed-clojure custom linter (example), and would keep me from writing a bridge linter for splint. Could this be relaxed with using non-native clojure-lsp?

[ericdallo]

@NoahTheDuke Hum, interesting need.
What I see is: we want to leverage clojure-lsp analysis to let users register their own diagnostics using those analysis, would that example or splint use those analysis to register diagnostics? I think I need a more refined example here to understand it better