Custom Metrics

Custom metrics

The primitives are implemented in this module — no Prometheus client library is involved.

Getting the registry

The extension registers it in the DI container during OnInitialize:

var reg metric.Registry
if err := app.Container().Resolve(&reg); err != nil {
	return err
}

Resolve it from your own extension's OnInitialize or later — before that, the metrics extension has not run.

Registering

orders := metric.NewCounterVec([]string{"status", "channel"})
orders.SetHelp("Orders processed, by outcome and channel")

if err := reg.Register("orders_processed_total", orders); err != nil {
	return fmt.Errorf("metrics: %w", err)
}

Check the error. A duplicate name returns metric.ErrDuplicateMetric and is refused.

Registering a name twice used to append both, so the exposition carried two HELP and two TYPE lines for one name — which Prometheus rejects, discarding the entire scrape. One duplicate therefore blinded every metric the process exported, and nothing reported it.

reg.Get(name) retrieves a registered metric, if you would rather look one up than thread it through.

The primitives

Counter — monotonic

c := metric.NewCounter()
c.SetHelp("Widgets produced")
c.Inc()
c.Add(5)
v := c.Value()

Gauge — up and down

g := metric.NewGauge()
g.Set(42)
g.Inc(); g.Dec(); g.Add(-3)

Histogram — distributions

h := metric.NewHistogram()                                  // default buckets
h = metric.NewHistogramWithBuckets([]float64{.005, .01, .025, .05, .1, .25, .5, 1, 2.5, 5, 10})
h.Observe(0.137)

Buckets are cumulative upper bounds in the unit you observe. Choose them around the latencies you actually care about — a bucket set that puts 99% of observations in one bucket tells you nothing.

Labelled variants

cv := metric.NewCounterVec([]string{"status"})
cv.With(map[string]string{"status": "settled"}).Inc()

gv := metric.NewGaugeVec([]string{"queue"})
gv.With(map[string]string{"queue": "outbound"}).Set(17)

hv := metric.NewHistogramVecWithBuckets([]string{"backend"}, buckets)
hv.With(map[string]string{"backend": "payments"}).Observe(0.42)

With creates the series if it does not exist, so calling it with no operation is how you pre-create one:

// Appears in the exposition as 0 rather than being absent until first use.
gv.InitWith(map[string]string{"queue": "outbound"})

Pre-creating matters more than it looks: a counter that is absent and a counter that is zero look different to rate(), and an alert on "no data" fires differently from one on "zero".

Label rules

Never label with anything a client controls. User identifiers, raw paths, query parameters, arbitrary header values — each distinct value is a series retained for the lifetime of the process. See Cardinality.

Keep label sets small and fixed. The number of series is the product of the cardinalities of every label. Three labels with ten values each is a thousand series for one metric.

Use the same label names as the built-ins — router, method, path, status — so queries can join across them.

Naming

Follow Prometheus conventions:

• _total for counters, _seconds / _bytes for units
• base unit, not milliseconds or kilobytes
• prefix by subsystem: orders_processed_total, not processed
• SetHelp on everything — an unexplained metric gets misread

Bounding your own vectors

if b, ok := orders.(metric.BoundedVec); ok {
	b.SetMaxSeries(200)  // 0 takes the default of 1000; negative disables
	_ = b.MaxSeries()
	_ = b.DroppedSeries()
}

Every labelled vector in this module implements BoundedVec.

A worked example

type OrderMetrics struct {
	Processed metric.CounterVec
	Duration  metric.HistogramVec
	Pending   metric.Gauge
}

func NewOrderMetrics(reg metric.Registry) (*OrderMetrics, error) {
	m := &OrderMetrics{
		Processed: metric.NewCounterVec([]string{"outcome"}),
		Duration: metric.NewHistogramVecWithBuckets([]string{"outcome"},
			[]float64{.01, .05, .1, .5, 1, 5}),
		Pending: metric.NewGauge(),
	}
	m.Processed.SetHelp("Orders processed, by outcome")
	m.Duration.SetHelp("Order processing duration in seconds")
	m.Pending.SetHelp("Orders awaiting processing")

	return m, errors.Join(
		reg.Register("orders_processed_total", m.Processed),
		reg.Register("order_processing_duration_seconds", m.Duration),
		reg.Register("orders_pending", m.Pending),
	)
}

func (m *OrderMetrics) Observe(outcome string, d time.Duration) {
	l := map[string]string{"outcome": outcome}
	m.Processed.With(l).Inc()
	m.Duration.With(l).Observe(d.Seconds())
}

Register it as a singleton so handlers can resolve it:

app.Container().Singleton(func(r rextension.Resolver) *OrderMetrics { … })