Consider reducing label cardinality for persist error exporting

[juliusv]

Currently we export the label error="<entire error string>" for persist errors on the metric prometheus_local_storage_persist_errors_total:

prometheus/storage/local/storage.go

Line 539 in f541390

s.persistErrors.WithLabelValues(err.Error()).Inc()

In a Prometheus whose disk just ran full, this already produced two different error strings:

offset 262144 is not a multiple of on-disk chunk length 1041
offset 131072 is not a multiple of on-disk chunk length 1041

Conceivably there could be many more different offsets affected, so we should probably clamp down the label cardinality on this one.

/cc @beorn7

[brian-brazil]

From quick look this should be a no-label counter, it's already in the logs.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.