Bug 1812412: Monitoring: Increase Prometheus query_range timeouts to 30s #7004
Conversation
The current timeout of 5 seconds is probably much too strict. Prometheus has a maximum concurrent queries limit, which is currently set to 20. This should protect against overloading Prometheus, even if we increase the timeout significantly. Also, if there are more than 20 queries, some will be queued and the time they spend in the queue will count towards the time limit, which is another reason to make the timeout less strict.
|
@kyoto: An error was encountered adding this pull request to the external tracker bugs for bug 1812412 on the Bugzilla server at https://bugzilla.redhat.com:
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
openshift-ci-robot
added
component/monitoring
|
/bugzilla refresh |
|
@kyoto: An error was encountered adding this pull request to the external tracker bugs for bug 1812412 on the Bugzilla server at https://bugzilla.redhat.com:
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
/bugzilla refresh |
|
@kyoto: An error was encountered adding this pull request to the external tracker bugs for bug 1812412 on the Bugzilla server at https://bugzilla.redhat.com:
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
/approve |
|
/bugzilla refresh |
|
@spadgett: An error was encountered adding this pull request to the external tracker bugs for bug 1812412 on the Bugzilla server at https://bugzilla.redhat.com:
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: kyoto, s-urbaniak, spadgett The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
@kyoto: All pull requests linked via external trackers have merged: Bugzilla bug 1812412 has been moved to the MODIFIED state. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
The current timeout of 5 seconds is probably much too strict.
Prometheus has a maximum concurrent queries limit, which is currently
set to 20. This should protect against overloading Prometheus, even if
we increase the timeout significantly.
Also, if there are more than 20 queries, some will be queued and the
time they spend in the queue will count towards the time limit, which is
another reason to make the timeout less strict.