Skip to content

Commit

Permalink
plugin: allow to use settings for plugins (#3887)
Browse files Browse the repository at this point in the history
  • Loading branch information
ldez committed Jun 15, 2023
1 parent 2dcd82f commit 25c2b07
Show file tree
Hide file tree
Showing 4 changed files with 193 additions and 122 deletions.
87 changes: 57 additions & 30 deletions docs/src/docs/contributing/new-linters.mdx
Expand Up @@ -2,10 +2,10 @@
title: New linters
---

## How to write a custom linter
## How to write a linter

Use `go/analysis` and take a look at [this tutorial](https://disaev.me/p/writing-useful-go-analysis-linter/): it shows how to write `go/analysis` linter
from scratch and integrate it into `golangci-lint`.
Use `go/analysis` and take a look at [this tutorial](https://disaev.me/p/writing-useful-go-analysis-linter/):
it shows how to write `go/analysis` linter from scratch and integrate it into `golangci-lint`.

## How to add a public linter to `golangci-lint`

Expand All @@ -16,8 +16,14 @@ After that:

1. Implement functional tests for the linter:
- Add one file into directory [`test/testdata`](https://github.com/golangci/golangci-lint/tree/master/test/testdata).
- Run `T=yourlintername.go make test_linters` to ensure that test fails.
- Run `go run ./cmd/golangci-lint/ run --no-config --disable-all --enable=yourlintername ./test/testdata/yourlintername.go`
- Run the test to ensure that test fails:
```bash
T=yourlintername.go make test_linters
```
- Run:
```bash
go run ./cmd/golangci-lint/ run --no-config --disable-all --enable=yourlintername ./test/testdata/yourlintername.go
```
2. Add a new file `pkg/golinters/{yourlintername}.go`.
Look at other linters in this directory.
Implement linter integration and check that test passes.
Expand All @@ -33,58 +39,79 @@ After that:
if you think that this project needs not default values.
- [config struct](https://github.com/golangci/golangci-lint/blob/master/pkg/config/config.go) -
don't forget about `mapstructure` tag for proper configuration files parsing by [pflag](https://github.com/spf13/pflag).
5. Take a look at the example of [Pull Request with new linter support](https://github.com/golangci/golangci-lint/pulls?q=is%3Apr+is%3Amerged+label%3A%22linter%3A+new%22).
5. Take a look at the example of [pull requests with new linter support](https://github.com/golangci/golangci-lint/pulls?q=is%3Apr+is%3Amerged+label%3A%22linter%3A+new%22).

## How to add a private linter to `golangci-lint`

Some people and organizations may choose to have custom-made linters run as a part of `golangci-lint`.
Typically, these linters can't be open-sourced or too specific.

Such linters can be added through Go's plugin library.

For a private linter (which acts as a plugin) to work properly,
the plugin as well as the golangci-lint binary needs to be built for the same environment. `CGO_ENABLED` is another requirement.
the plugin as well as the golangci-lint binary **needs to be built for the same environment**.

`CGO_ENABLED` is another requirement.

This means that `golangci-lint` needs to be built for whatever machine you intend to run it on
(cloning the golangci-lint repository and running a `CGO_ENABLED=1 make build` should do the trick for your machine).

### Configure a Plugin

If you already have a linter plugin available, you can follow these steps to define it's usage in a projects
`.golangci.yml` file. An example linter can be found at [here](https://github.com/golangci/example-plugin-linter). If you're looking for
instructions on how to configure your own custom linter, they can be found further down.
If you already have a linter plugin available, you can follow these steps to define its usage in a projects `.golangci.yml` file.

An example linter can be found at [here](https://github.com/golangci/example-plugin-linter).

If you're looking for instructions on how to configure your own custom linter, they can be found further down.

1. If the project you want to lint does not have one already, copy the [.golangci.yml](https://github.com/golangci/golangci-lint/blob/master/.golangci.yml) to the root directory.
2. Adjust the yaml to appropriate `linters-settings:custom` entries as so:

```yaml
linters-settings:
custom:
example:
path: /example.so
description: The description of the linter
original-url: github.com/golangci/example-linter
```
```yaml
linters-settings:
custom:
example:
path: /example.so
description: The description of the linter
original-url: github.com/golangci/example-linter
settings: # Settings are optional.
one: Foo
two:
- name: Bar
three:
name: Bar
```

That is all the configuration that is required to run a custom linter in your project.

Custom linters are disabled by default, and are not enabled when `linters.enable-all` is specified.
They can be enabled by adding them the `linters.enable` list, or providing the enabled option on the command line (`golangci-lint run -Eexample`).

The configuration inside the `settings` field of linter have some limitations (there are NOT related to the plugin system itself):
we use Viper to handle the configuration but Viper put all the keys in lowercase, and `.` cannot be used inside a key.

### Create a Plugin

Your linter must implement one or more `golang.org/x/tools/go/analysis.Analyzer` structs.
Your project should also use `go.mod`. All versions of libraries that overlap `golangci-lint` (including replaced
libraries) MUST be set to the same version as `golangci-lint`. You can see the versions by running `go version -m golangci-lint`.
Your linter must provide one or more `golang.org/x/tools/go/analysis.Analyzer` structs.

Your project should also use `go.mod`.

You'll also need to create a go file like `plugin/example.go`. This MUST be in the package `main`, and define a
variable of name `AnalyzerPlugin`. The `AnalyzerPlugin` instance MUST implement the following interface:
All versions of libraries that overlap `golangci-lint` (including replaced libraries) MUST be set to the same version as `golangci-lint`.
You can see the versions by running `go version -m golangci-lint`.

You'll also need to create a Go file like `plugin/example.go`.

This file MUST be in the package `main`, and MUST define an exposed function called `New` with the following signature:
```go
type AnalyzerPlugin interface {
GetAnalyzers() []*analysis.Analyzer
func New(conf any) ([]*analysis.Analyzer, error) {
// ...
}
```

The type of `AnalyzerPlugin` is not important, but is by convention `type analyzerPlugin struct {}`. See
[plugin/example.go](https://github.com/golangci/example-plugin-linter/blob/master/plugin/example.go) for more info.
See [plugin/example.go](https://github.com/golangci/example-plugin-linter/blob/master/plugin/example.go) for more info.

To build the plugin, from the root project directory, run:
```bash
go build -buildmode=plugin plugin/example.go
```

To build the plugin, from the root project directory, run `go build -buildmode=plugin plugin/example.go`. This will create a plugin `*.so`
file that can be copied into your project or another well known location for usage in golangci-lint.
This will create a plugin `*.so` file that can be copied into your project or another well known location for usage in `golangci-lint`.
5 changes: 4 additions & 1 deletion pkg/config/linters_settings.go
Expand Up @@ -828,6 +828,9 @@ type CustomLinterSettings struct {
Path string
// Description describes the purpose of the private linter.
Description string
// The URL containing the source code for the private linter.
// OriginalURL The URL containing the source code for the private linter.
OriginalURL string `mapstructure:"original-url"`

// Settings plugin settings only work with linterdb.PluginConstructor symbol.
Settings any
}
132 changes: 132 additions & 0 deletions pkg/lint/lintersdb/custom_linters.go
@@ -0,0 +1,132 @@
package lintersdb

import (
"fmt"
"path/filepath"
"plugin"

"github.com/hashicorp/go-multierror"
"github.com/spf13/viper"
"golang.org/x/tools/go/analysis"

"github.com/golangci/golangci-lint/pkg/config"
"github.com/golangci/golangci-lint/pkg/golinters/goanalysis"
"github.com/golangci/golangci-lint/pkg/lint/linter"
"github.com/golangci/golangci-lint/pkg/logutils"
"github.com/golangci/golangci-lint/pkg/report"
)

type AnalyzerPlugin interface {
GetAnalyzers() []*analysis.Analyzer
}

// WithCustomLinters loads private linters that are specified in the golangci config file.
func (m *Manager) WithCustomLinters() *Manager {
if m.log == nil {
m.log = report.NewLogWrapper(logutils.NewStderrLog(logutils.DebugKeyEmpty), &report.Data{})
}

if m.cfg == nil {
return m
}

for name, settings := range m.cfg.LintersSettings.Custom {
lc, err := m.loadCustomLinterConfig(name, settings)

if err != nil {
m.log.Errorf("Unable to load custom analyzer %s:%s, %v", name, settings.Path, err)
} else {
m.nameToLCs[name] = append(m.nameToLCs[name], lc)
}
}

return m
}

// loadCustomLinterConfig loads the configuration of private linters.
// Private linters are dynamically loaded from .so plugin files.
func (m *Manager) loadCustomLinterConfig(name string, settings config.CustomLinterSettings) (*linter.Config, error) {
analyzers, err := m.getAnalyzerPlugin(settings.Path, settings.Settings)
if err != nil {
return nil, err
}

m.log.Infof("Loaded %s: %s", settings.Path, name)

customLinter := goanalysis.NewLinter(name, settings.Description, analyzers, nil).
WithLoadMode(goanalysis.LoadModeTypesInfo)

linterConfig := linter.NewConfig(customLinter).
WithEnabledByDefault().
WithLoadForGoAnalysis().
WithURL(settings.OriginalURL)

return linterConfig, nil
}

// getAnalyzerPlugin loads a private linter as specified in the config file,
// loads the plugin from a .so file,
// and returns the 'AnalyzerPlugin' interface implemented by the private plugin.
// An error is returned if the private linter cannot be loaded
// or the linter does not implement the AnalyzerPlugin interface.
func (m *Manager) getAnalyzerPlugin(path string, settings any) ([]*analysis.Analyzer, error) {
if !filepath.IsAbs(path) {
// resolve non-absolute paths relative to config file's directory
configFilePath := viper.ConfigFileUsed()
absConfigFilePath, err := filepath.Abs(configFilePath)
if err != nil {
return nil, fmt.Errorf("could not get absolute representation of config file path %q: %v", configFilePath, err)
}
path = filepath.Join(filepath.Dir(absConfigFilePath), path)
}

plug, err := plugin.Open(path)
if err != nil {
return nil, err
}

analyzers, err := m.lookupPlugin(plug, settings)
if err != nil {
return nil, fmt.Errorf("lookup plugin %s: %w", path, err)
}

return analyzers, nil
}

func (m *Manager) lookupPlugin(plug *plugin.Plugin, settings any) ([]*analysis.Analyzer, error) {
symbol, err := plug.Lookup("New")
if err != nil {
analyzers, errP := m.lookupAnalyzerPlugin(plug)
if errP != nil {
// TODO(ldez): use `errors.Join` when we will upgrade to go1.20.
return nil, multierror.Append(err, errP)
}

return analyzers, nil
}

// The type func cannot be used here, must be the explicit signature.
constructor, ok := symbol.(func(any) ([]*analysis.Analyzer, error))
if !ok {
return nil, fmt.Errorf("plugin does not abide by 'New' function: %T", symbol)
}

return constructor(settings)
}

func (m *Manager) lookupAnalyzerPlugin(plug *plugin.Plugin) ([]*analysis.Analyzer, error) {
symbol, err := plug.Lookup("AnalyzerPlugin")
if err != nil {
return nil, err
}

m.log.Warnf("plugin: 'AnalyzerPlugin' plugins are deprecated, please use the new plugin signature: " +
"https://golangci-lint.run/contributing/new-linters/#create-a-plugin")

analyzerPlugin, ok := symbol.(AnalyzerPlugin)
if !ok {
return nil, fmt.Errorf("plugin does not abide by 'AnalyzerPlugin' interface: %T", symbol)
}

return analyzerPlugin.GetAnalyzers(), nil
}
91 changes: 0 additions & 91 deletions pkg/lint/lintersdb/manager.go
@@ -1,19 +1,10 @@
package lintersdb

import (
"fmt"
"path/filepath"
"plugin"

"github.com/spf13/viper"
"golang.org/x/tools/go/analysis"

"github.com/golangci/golangci-lint/pkg/config"
"github.com/golangci/golangci-lint/pkg/golinters"
"github.com/golangci/golangci-lint/pkg/golinters/goanalysis"
"github.com/golangci/golangci-lint/pkg/lint/linter"
"github.com/golangci/golangci-lint/pkg/logutils"
"github.com/golangci/golangci-lint/pkg/report"
)

type Manager struct {
Expand All @@ -37,28 +28,6 @@ func NewManager(cfg *config.Config, log logutils.Log) *Manager {
return m
}

// WithCustomLinters loads private linters that are specified in the golangci config file.
func (m *Manager) WithCustomLinters() *Manager {
if m.log == nil {
m.log = report.NewLogWrapper(logutils.NewStderrLog(logutils.DebugKeyEmpty), &report.Data{})
}
if m.cfg != nil {
for name, settings := range m.cfg.LintersSettings.Custom {
lc, err := m.loadCustomLinterConfig(name, settings)

if err != nil {
m.log.Errorf("Unable to load custom analyzer %s:%s, %v",
name,
settings.Path,
err)
} else {
m.nameToLCs[name] = append(m.nameToLCs[name], lc)
}
}
}
return m
}

func (Manager) AllPresets() []string {
return []string{
linter.PresetBugs,
Expand Down Expand Up @@ -950,63 +919,3 @@ func (m Manager) GetAllLinterConfigsForPreset(p string) []*linter.Config {

return ret
}

// loadCustomLinterConfig loads the configuration of private linters.
// Private linters are dynamically loaded from .so plugin files.
func (m Manager) loadCustomLinterConfig(name string, settings config.CustomLinterSettings) (*linter.Config, error) {
analyzer, err := m.getAnalyzerPlugin(settings.Path)
if err != nil {
return nil, err
}
m.log.Infof("Loaded %s: %s", settings.Path, name)
customLinter := goanalysis.NewLinter(
name,
settings.Description,
analyzer.GetAnalyzers(),
nil).WithLoadMode(goanalysis.LoadModeTypesInfo)

linterConfig := linter.NewConfig(customLinter).
WithEnabledByDefault().
WithLoadForGoAnalysis().
WithURL(settings.OriginalURL)

return linterConfig, nil
}

type AnalyzerPlugin interface {
GetAnalyzers() []*analysis.Analyzer
}

// getAnalyzerPlugin loads a private linter as specified in the config file,
// loads the plugin from a .so file, and returns the 'AnalyzerPlugin' interface
// implemented by the private plugin.
// An error is returned if the private linter cannot be loaded or the linter
// does not implement the AnalyzerPlugin interface.
func (m Manager) getAnalyzerPlugin(path string) (AnalyzerPlugin, error) {
if !filepath.IsAbs(path) {
// resolve non-absolute paths relative to config file's directory
configFilePath := viper.ConfigFileUsed()
absConfigFilePath, err := filepath.Abs(configFilePath)
if err != nil {
return nil, fmt.Errorf("could not get absolute representation of config file path %q: %v", configFilePath, err)
}
path = filepath.Join(filepath.Dir(absConfigFilePath), path)
}

plug, err := plugin.Open(path)
if err != nil {
return nil, err
}

symbol, err := plug.Lookup("AnalyzerPlugin")
if err != nil {
return nil, err
}

analyzerPlugin, ok := symbol.(AnalyzerPlugin)
if !ok {
return nil, fmt.Errorf("plugin %s does not abide by 'AnalyzerPlugin' interface", path)
}

return analyzerPlugin, nil
}

0 comments on commit 25c2b07

Please sign in to comment.