This project demonstrates the recommended way to build your own Benthos component plugins and run them in a custom distribution.
Start by writing your plugins where ever you like, there are examples in this repo for bloblang functions and methods, inputs, processors and outputs to copy from.
Next, author a main file that calls service.Run()
and imports your plugins as shown in this example:
package main
import (
"github.com/Jeffail/benthos/v3/public/service"
// Add your plugin packages here
_ "github.com/benthosdev/benthos-plugin-example/bloblang"
_ "github.com/benthosdev/benthos-plugin-example/input"
_ "github.com/benthosdev/benthos-plugin-example/output"
_ "github.com/benthosdev/benthos-plugin-example/processor"
)
func main() {
service.RunCLI(context.Background())
}
Finally, build your custom main func:
go build ./cmd/benthos-plugin-example
Alternatively build it as a Docker image with:
go mod vendor
docker build . -t benthos-plugin-example
The new service you've built will come with all of the usual Benthos components plus all of your custom plugins, which you can use like any other type. The only difference between your plugins and original Benthos components is that the config field for plugin specific fields is always plugin
.
For example, to use the example plugin components gibberish
, reverse
and blue_stdout
, and our new Bloblang function crazy_object
and method into_object
, our config might look like this:
input:
gibberish:
length: 80
pipeline:
processors:
- throttle:
period: 1s
- reverse: {}
- bloblang: |
root.gibberish = content()
root.more_stuff = crazy_object(10).into_object("foo")
output:
blue_stdout: {}
And you can run it like this:
./benthos-plugin-example -c ./yourconfig.yaml
For more examples on how to configure your plugins check out ./config
.