Description
Opening this issue on behalf of @Gandem
runtime/metrics is supposed to be the new (and preferred) way of of exporting metrics from the Go runtime (added in Go 1.16). A couple of advantages:
- It's more performant than alternatives: it does not involve a stop-the-world to be able to collect those metrics. Also, the metrics used by runtime/metrics are always collected in the go runtime whether you export them or not. So the overhead of actually exporting them is minimal (i.e. if we use runtime/metrics as a source, it could be reasonable to enable their collection by default, unless we're concerned about the number of metrics generated).
- It contains metrics which are not available through other means such as the amount of GC scannable space, live heap/heap goal (rationale), distribution of goroutine latency (time goroutines have spent in the scheduler in a runnable state before actually running). And we can expect more metrics to be added here in the future.
One caveat is that runtime/metrics contains most of the metrics from ReadMemStats and ReadGcStats but not strictly all of the metrics (e.g. memory Sys, Lookups, LastGC).
@felixge's suggestion
we could add a new WithRuntimeMetricsV2 option for adding this feature. That would allow us to give our users the option to use the new runtime/metrics package exclusively without having to worry about backwards compatibility. This would also have the advantage that dd-go could consume this right away.
And when we move to v2 for dd-trace-go, we just make this the default (we could still optionally offer the old version).
Bonus: We should explore enabling runtime metrics by default in dd-trace-go v2. Note that JVM metrics are enabled by default in the java tracer.