Skip to content

Commit

Permalink
new prometheus middleware (#94)
Browse files Browse the repository at this point in the history
new prometheus middleware
  • Loading branch information
aldas committed May 22, 2023
1 parent b6855c2 commit c764849
Show file tree
Hide file tree
Showing 7 changed files with 955 additions and 50 deletions.
70 changes: 28 additions & 42 deletions .github/workflows/echo-contrib.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,73 +4,54 @@ on:
push:
branches:
- master
paths:
- '**.go'
- 'go.*'
- '_fixture/**'
- '.github/**'
- 'codecov.yml'
pull_request:
branches:
- master
paths:
- '**.go'
- 'go.*'
- '_fixture/**'
- '.github/**'
- 'codecov.yml'
workflow_dispatch:

permissions:
contents: read # to fetch code (actions/checkout)

env:
# run coverage and benchmarks only with the latest Go version
LATEST_GO_VERSION: "1.20"

jobs:
test:
env:
latest: '1.19'
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
go: [ '1.17', '1.18', '1.19']
# Each major Go release is supported until there are two newer major releases. https://golang.org/doc/devel/release.html#policy
# Echo tests with last four major releases (unless there are pressing vulnerabilities)
# As we depend on `golang.org/x/` libraries which only support last 2 Go releases we could have situations when
# we derive from last four major releases promise.
go: ["1.18", "1.19", "1.20"]
name: ${{ matrix.os }} @ Go ${{ matrix.go }}
runs-on: ${{ matrix.os }}
steps:
- name: Set up Go ${{ matrix.go }}
uses: actions/setup-go@v3
with:
go-version: ${{ matrix.go }}

- name: Checkout Code
uses: actions/checkout@v3
with:
ref: ${{ github.ref }}

- name: Run static checks
if: matrix.go == env.latest && matrix.os == 'ubuntu-latest'
run: |
go install honnef.co/go/tools/cmd/staticcheck@latest
staticcheck -tests=false ./...
- name: Set up Go ${{ matrix.go }}
uses: actions/setup-go@v4
with:
go-version: ${{ matrix.go }}

- name: Run Tests
run: |
go test -race --coverprofile=coverage.coverprofile --covermode=atomic ./...
run: go test -race --coverprofile=coverage.coverprofile --covermode=atomic ./...

- name: Upload coverage to Codecov
if: success() && matrix.go == env.latest && matrix.os == 'ubuntu-latest'
if: success() && matrix.go == env.LATEST_GO_VERSION && matrix.os == 'ubuntu-latest'
uses: codecov/codecov-action@v3
with:
token:
fail_ci_if_error: false

benchmark:
needs: test
strategy:
matrix:
os: [ubuntu-latest]
go: [1.19]
name: Benchmark comparison ${{ matrix.os }} @ Go ${{ matrix.go }}
runs-on: ${{ matrix.os }}
name: Benchmark comparison
runs-on: ubuntu-latest
steps:
- name: Set up Go ${{ matrix.go }}
uses: actions/setup-go@v3
with:
go-version: ${{ matrix.go }}

- name: Checkout Code (Previous)
uses: actions/checkout@v3
with:
Expand All @@ -82,6 +63,11 @@ jobs:
with:
path: new

- name: Set up Go ${{ matrix.go }}
uses: actions/setup-go@v4
with:
go-version: ${{ env.LATEST_GO_VERSION }}

- name: Install Dependencies
run: go install golang.org/x/perf/cmd/benchstat@latest

Expand All @@ -97,4 +83,4 @@ jobs:
- name: Run Benchstat
run: |
benchstat previous/benchmark.txt new/benchmark.txt
benchstat previous/benchmark.txt new/benchmark.txt
10 changes: 3 additions & 7 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,15 +1,11 @@
PKG := "github.com/labstack/echo-contrib"
PKG_LIST := $(shell go list ${PKG}/...)

tag:
@git tag `grep -P '^\tversion = ' echo.go|cut -f2 -d'"'`
@git tag|grep -v ^v

.DEFAULT_GOAL := check
check: lint vet race ## Check project

init:
@go get -u honnef.co/go/tools/cmd/staticcheck@latest
@go install honnef.co/go/tools/cmd/staticcheck@latest

format: ## Format the source code
@find ./ -type f -name "*.go" -exec gofmt -w {} \;
Expand All @@ -32,6 +28,6 @@ benchmark: ## Run benchmarks
help: ## Display this help screen
@grep -h -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'

goversion ?= "1.16"
test_version: ## Run tests inside Docker with given version (defaults to 1.15 oldest supported). Example: make test_version goversion=1.15
goversion ?= "1.18"
test_version: ## Run tests inside Docker with given version (defaults to 1.18 oldest supported). Example: make test_version goversion=1.18
@docker run --rm -it -v $(shell pwd):/project golang:$(goversion) /bin/sh -c "cd /project && make race"
154 changes: 154 additions & 0 deletions echoprometheus/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
# Usage

```
package main
import (
"github.com/labstack/echo/v4"
"github.com/labstack/echo-contrib/prometheus/echoprometheus"
)
func main() {
e := echo.New()
// Enable metrics middleware
e.Use(echoprometheus.NewMiddleware("myapp"))
e.GET("/metrics", echoprometheus.NewHandler())
e.Logger.Fatal(e.Start(":1323"))
}
```


# How to migrate

## Creating and adding middleware to the application

Older `prometheus` middleware
```go
e := echo.New()
p := prometheus.NewPrometheus("echo", nil)
p.Use(e)
```

With the new `echoprometheus` middleware
```go
e := echo.New()
e.Use(echoprometheus.NewMiddleware("myapp")) // register middleware to gather metrics from requests
e.GET("/metrics", echoprometheus.NewHandler()) // register route to serve gathered metrics in Prometheus format
```

## Replacement for `Prometheus.MetricsList` field, `NewMetric(m *Metric, subsystem string)` function and `prometheus.Metric` struct

The `NewMetric` function allowed to create custom metrics with the old `prometheus` middleware. This helper is no longer available
to avoid the added complexity. It is recommended to use native Prometheus metrics and register those yourself.

This can be done now as follows:
```go
e := echo.New()

customRegistry := prometheus.NewRegistry() // create custom registry for your custom metrics
customCounter := prometheus.NewCounter( // create new counter metric. This is replacement for `prometheus.Metric` struct
prometheus.CounterOpts{
Name: "custom_requests_total",
Help: "How many HTTP requests processed, partitioned by status code and HTTP method.",
},
)
if err := customRegistry.Register(customCounter); err != nil { // register your new counter metric with metrics registry
log.Fatal(err)
}

e.Use(NewMiddlewareWithConfig(MiddlewareConfig{
AfterNext: func(c echo.Context, err error) {
customCounter.Inc() // use our custom metric in middleware. after every request increment the counter
},
Registerer: customRegistry, // use our custom registry instead of default Prometheus registry
}))
e.GET("/metrics", NewHandlerWithConfig(HandlerConfig{Gatherer: customRegistry})) // register route for getting gathered metrics data from our custom Registry
```

## Replacement for `Prometheus.MetricsPath`

`MetricsPath` was used to skip metrics own route from Prometheus metrics. Skipping is no longer done and requests to Prometheus
route will be included in gathered metrics.

To restore the old behaviour the `/metrics` path needs to be excluded from counting using the Skipper function:
```go
conf := echoprometheus.MiddlewareConfig{
Skipper: func(c echo.Context) bool {
return c.Path() == "/metrics"
},
}
e.Use(echoprometheus.NewMiddlewareWithConfig(conf))
```

## Replacement for `Prometheus.RequestCounterURLLabelMappingFunc` and `Prometheus.RequestCounterHostLabelMappingFunc`

These function fields were used to define how "URL" or "Host" attribute in Prometheus metric lines are created.

These can now be substituted by using `LabelFuncs`:
```go
e.Use(echoprometheus.NewMiddlewareWithConfig(echoprometheus.MiddlewareConfig{
LabelFuncs: map[string]echoprometheus.LabelValueFunc{
"scheme": func(c echo.Context, err error) string { // additional custom label
return c.Scheme()
},
"url": func(c echo.Context, err error) string { // overrides default 'url' label value
return "x_" + c.Request().URL.Path
},
"host": func(c echo.Context, err error) string { // overrides default 'host' label value
return "y_" + c.Request().Host
},
},
}))
```

Will produce Prometheus line as
`echo_request_duration_seconds_count{code="200",host="y_example.com",method="GET",scheme="http",url="x_/ok",scheme="http"} 1`


## Replacement for `Metric.Buckets` and modifying default metrics

The `echoprometheus` middleware registers the following metrics by default:

* Counter `requests_total`
* Histogram `request_duration_seconds`
* Histogram `response_size_bytes`
* Histogram `request_size_bytes`

You can modify their definition before these metrics are registed with `CounterOptsFunc` and `HistogramOptsFunc` callbacks

Example:
```go
e.Use(NewMiddlewareWithConfig(MiddlewareConfig{
HistogramOptsFunc: func(opts prometheus.HistogramOpts) prometheus.HistogramOpts {
if opts.Name == "request_duration_seconds" {
opts.Buckets = []float64{1.0 * bKB, 2.0 * bKB, 5.0 * bKB, 10.0 * bKB, 100 * bKB, 500 * bKB, 1.0 * bMB, 2.5 * bMB, 5.0 * bMB, 10.0 * bMB}
}
return opts
},
CounterOptsFunc: func(opts prometheus.CounterOpts) prometheus.CounterOpts {
if opts.Name == "requests_total" {
opts.ConstLabels = prometheus.Labels{"my_const": "123"}
}
return opts
},
}))
```

## Replacement for `PushGateway` struct and related methods

Function `RunPushGatewayGatherer` starts pushing collected metrics and block until context completes or ErrorHandler returns an error.
This function should be run in separate goroutine.

Example:
```go
go func() {
config := echoprometheus.PushGatewayConfig{
PushGatewayURL: "https://host:9080",
PushInterval: 10 * time.Millisecond,
}
if err := echoprometheus.RunPushGatewayGatherer(context.Background(), config); !errors.Is(err, context.Canceled) {
log.Fatal(err)
}
}()
```

0 comments on commit c764849

Please sign in to comment.