Sampling core doesn't seem to have any effect

[mholt]

I'm trying to limit the excessive output of some very active logs, the same two messages repeating (same logger) thousands of times per second.

I figured that wrapping my custom core in a NewSamplerWithOptions() would do the trick:

// ...

core := zapcore.NewTee(
	zapcore.NewCore(consoleEncoder, consoleOut, zap.DebugLevel),
	zapcore.NewCore(jsonEncoder, webOut, zap.InfoLevel),
)

core = zapcore.NewSamplerWithOptions(core, 5*time.Second, 1, 1)

logger := zap.New(core)

// ...

I thought I could expect about 2 log messages per 5 seconds with this configuration, but I'm still seeing a firehose of thousands of the same logs every second, completely obliterating the performance of the app (writing the logs to the outputs is slowing it down).

Am I using it wrong? 😅

[abhinav]

Hey there! Yes, your configuration of the Zap sampler is slightly incorrect.

If you do the following,

core := zapcore.NewSamplerWithOptions(core, interval, N, M)

Zap will log the first N messages in interval without sampling.
Following that, it will log every M-th message.

The code above sets M to 1: log every message, so it effectively disables sampling.
You can set M to a higher value to get it to sample messages.

// Allow every 100th duplicate message in a 5 second interval.
core := zapcore.NewSamplerWithOptons(core, 5*time.Second, 1, 100)

Unfortunately, the sampler does not yet support setting M to zero.
We can add support for that.

[abhinav]

@mholt We've added support for a zero value of M. It will let you do the following to log at most 2 messages per 5 second interval.

core = zapcore.NewSamplerWithOptions(core, 5*time.Second, 2, 0)

We won't release this until January, though.
You can temporarily pin to the current master branch with:

go mod edit -replace go.uber.org/zap=github.com/uber-go/zap@master

[mholt]

Ohh, thank you -- I was wondering what N and M were referring to, as I couldn't find them in the method signature. And I tiotally misunderstood M, I realize now it's part of a ratio. Really appreciate the update too, that'll be useful going forward. Thanks again!