Add best-effort validation for prometheus scrape interval #11376
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Alex Leong <alex@buoyant.io>
|
Prometheus exposes a config API that contains the scrape config. A more robust approach could consist on having the |
|
@alpeb oh nice, I'll check out the config API 👍 thanks |
Signed-off-by: Alex Leong <alex@buoyant.io>
Signed-off-by: Alex Leong <alex@buoyant.io>
alpeb
approved these changes
Sep 21, 2023
viz/metrics-api/grpc_server.go
Outdated
| } | ||
|
|
||
| if t < scrape_interval { | ||
| return fmt.Errorf("Time window (%s) must be at least as long as the Prometheus scrape interval (%s)", window, scrape_interval) |
There was a problem hiding this comment.
Error strings should not be capitalized
Suggested change
| return fmt.Errorf("Time window (%s) must be at least as long as the Prometheus scrape interval (%s)", window, scrape_interval) | |
| return fmt.Errorf("time window (%s) must be at least as long as the Prometheus scrape interval (%s)", window, scrape_interval) |
Signed-off-by: Alex Leong <alex@buoyant.io>
olix0r
approved these changes
Sep 26, 2023
olix0r
added a commit
that referenced
this pull request
Sep 28, 2023
This edge release makes Linkerd even better. * Added a controlPlaneVersion override to the `linkerd-control-plane` Helm chart to support including SHA256 image digests in Linkerd manifests (thanks @cromulentbanana!) ([#11406]) * Improved `linkerd viz check` to attempt to validate that the Prometheus scrape interval will work well with the CLI and Web query parameters ([#11376]) * Fixed an issue where the destination controller would not update pod metadata for profile resolutions for a pod accessed via the host network (e.g. HostPort endpoints) ([#11334]). * Added a validating webhook config for httproutes.gateway.networking.k8s.io resources (thanks @mikutas!) ([#11150]) [#11150]: #11150 [#11334]: #11334 [#11376]: #11376 [#11406]: #11406
Merged
olix0r
added a commit
that referenced
this pull request
Sep 28, 2023
This edge release makes Linkerd even better. * Added a controlPlaneVersion override to the `linkerd-control-plane` Helm chart to support including SHA256 image digests in Linkerd manifests (thanks @cromulentbanana!) ([#11406]) * Improved `linkerd viz check` to attempt to validate that the Prometheus scrape interval will work well with the CLI and Web query parameters ([#11376]) * Fixed an issue where the destination controller would not update pod metadata for profile resolutions for a pod accessed via the host network (e.g. HostPort endpoints) ([#11334]). * Added a validating webhook config for httproutes.gateway.networking.k8s.io resources (thanks @mikutas!) ([#11150]) [#11150]: #11150 [#11334]: #11334 [#11376]: #11376 [#11406]: #11406
olix0r
added a commit
that referenced
this pull request
Sep 29, 2023
This edge release makes Linkerd even better. * Added a controlPlaneVersion override to the `linkerd-control-plane` Helm chart to support including SHA256 image digests in Linkerd manifests (thanks @cromulentbanana!) ([#11406]) * Improved `linkerd viz check` to attempt to validate that the Prometheus scrape interval will work well with the CLI and Web query parameters ([#11376]) * Improved CLI error handling to print differentiated error information when versioncheck.linkerd.io cannot be resolved (thanks @dtaskai) ([#11377]) * Fixed an issue where the destination controller would not update pod metadata for profile resolutions for a pod accessed via the host network (e.g. HostPort endpoints) ([#11334]). * Added a validating webhook config for httproutes.gateway.networking.k8s.io resources (thanks @mikutas!) ([#11150]) * Introduced a new `multicluster check --timeout` flag to limit the time allowed for Kubernetes API calls (thanks @moki1202) ([#11420]) [#11150]: #11150 [#11334]: #11334 [#11376]: #11376 [#11377]: #11377 [#11406]: #11406 [#11420]: #11420
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If a Linkerd-viz metrics query has a time window which is shorter than the Prometheus scrape interval, no data will be returned. This can be confusing and unexpected.
It is difficult to validate that the time window is at least as long as the scrape interval if an external prometheus is used. Therefore, we do a best-effort validation where we validate the scrape interval if we can find it in the prometheus config in the default location.