Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Internal request metric histograms labeled as nanoseconds are actually seconds #7372

Closed
jszwedko opened this issue May 6, 2021 · 0 comments · Fixed by #7373
Closed

Internal request metric histograms labeled as nanoseconds are actually seconds #7372

jszwedko opened this issue May 6, 2021 · 0 comments · Fixed by #7373
Assignees
Labels
domain: metrics Anything related to Vector's metrics events domain: networking Anything related to Vector's networking domain: observability Anything related to monitoring/observing Vector source: internal_metrics Anything `internal_metrics` source related type: bug A code related bug.
Milestone

Comments

@jszwedko
Copy link
Member

jszwedko commented May 6, 2021

While debugging #7371

I found these metrics:

vector_http_client_response_rtt_nanoseconds_bucket{le="-inf",status="200 OK"} 0 1620332874288
vector_http_client_response_rtt_nanoseconds_bucket{le="0.015625",status="200 OK"} 10 1620332874288
vector_http_client_response_rtt_nanoseconds_bucket{le="0.03125",status="200 OK"} 75 1620332874288
vector_http_client_response_rtt_nanoseconds_bucket{le="0.0625",status="200 OK"} 636 1620332874288
vector_http_client_response_rtt_nanoseconds_bucket{le="0.125",status="200 OK"} 4130 1620332874288
vector_http_client_response_rtt_nanoseconds_bucket{le="0.25",status="200 OK"} 4138 1620332874288
vector_http_client_response_rtt_nanoseconds_bucket{le="0.5",status="200 OK"} 4140 1620332874288
vector_http_client_response_rtt_nanoseconds_bucket{le="0",status="200 OK"} 4140 1620332874288
vector_http_client_response_rtt_nanoseconds_bucket{le="1",status="200 OK"} 4144 1620332874288
vector_http_client_response_rtt_nanoseconds_bucket{le="2",status="200 OK"} 4151 1620332874288
vector_http_client_response_rtt_nanoseconds_bucket{le="4",status="200 OK"} 4151 1620332874288
vector_http_client_response_rtt_nanoseconds_bucket{le="8",status="200 OK"} 4151 1620332874288
vector_http_client_response_rtt_nanoseconds_bucket{le="16",status="200 OK"} 4151 1620332874288
vector_http_client_response_rtt_nanoseconds_bucket{le="32",status="200 OK"} 4151 1620332874288
vector_http_client_response_rtt_nanoseconds_bucket{le="32",status="200 OK"} 4151 1620332874288
vector_http_client_response_rtt_nanoseconds_bucket{le="128",status="200 OK"} 4151 1620332874288
vector_http_client_response_rtt_nanoseconds_bucket{le="256",status="200 OK"} 4151 1620332874288
vector_http_client_response_rtt_nanoseconds_bucket{le="512",status="200 OK"} 4151 1620332874288
vector_http_client_response_rtt_nanoseconds_bucket{le="1024",status="200 OK"} 4151 1620332874288
vector_http_client_response_rtt_nanoseconds_bucket{le="2048",status="200 OK"} 4151 1620332874288
vector_http_client_response_rtt_nanoseconds_bucket{le="4096",status="200 OK"} 4151 1620332874288
vector_http_client_response_rtt_nanoseconds_bucket{le="inf",status="200 OK"} 4151 1620332874288
vector_http_client_response_rtt_nanoseconds_bucket{le="+Inf",status="200 OK"} 4151 1620332874288
vector_http_client_response_rtt_nanoseconds_sum{status="200 OK"} 345.1713884290003 1620332874288
vector_http_client_response_rtt_nanoseconds_count{status="200 OK"} 4151 1620332874288

These seem to me to just be seconds rather than nanoseconds which also matches what metrics-rs seems to do with Duration values: https://github.com/metrics-rs/metrics/blob/0df7fe15e4b2b3a415379d59f989adb7881fa323/metrics/src/common.rs#L253

We should audit other places we are reporting Duration values in our metrics to ensure they are properly labeled as seconds.

An alternative is multiply these by 10 * 9 to turn them into nanoseconds. The histogram buckets will need to be updated if we do that.

@jszwedko jszwedko added type: bug A code related bug. domain: metrics Anything related to Vector's metrics events labels May 6, 2021
@jszwedko jszwedko added this to the Vector 0.14 milestone May 6, 2021
@binarylogic binarylogic added domain: networking Anything related to Vector's networking domain: observability Anything related to monitoring/observing Vector labels May 6, 2021
@bruceg bruceg added the source: internal_metrics Anything `internal_metrics` source related label May 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain: metrics Anything related to Vector's metrics events domain: networking Anything related to Vector's networking domain: observability Anything related to monitoring/observing Vector source: internal_metrics Anything `internal_metrics` source related type: bug A code related bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants