Document prow alerts#483
Merged
Merged
Conversation
Signed-off-by: Michalis Kargakis <mkargaki@redhat.com>
openshift-ci-robot
added
the
size/M
Contributor
Author
|
@stevekuznetsov feel free to edit them as you see fit. We need some insight in how to debug Jenkins cc: @csrwng |
Contributor
stevekuznetsov
left a comment
stevekuznetsov
left a comment
There was a problem hiding this comment.
SGTM for what we have. For Jenkins we need to sync with Gabe and Ben re: how we can get Prometheus metrics out of the Jenkins masters.
| * Slow Jenkins operator sync | ||
| * Slow Jenkins pipeline operator sync | ||
|
|
||
| These should not be a problem in general but if any of them persists for more than a couple of hours, [`max_goroutines`](https://github.com/openshift/release/blob/ff18182aa0eb849b89e7abd1bc7765ad6d27142f/cluster/ci/config/prow/config.yaml#L7) can be incremented to allow more parallelism in the operators (note that the same option dictates both operators). |
Contributor
There was a problem hiding this comment.
Could also be Jenkins master being slow to respond, overloaded, etc?
Contributor
Author
There was a problem hiding this comment.
Yeah, I need to craft a query that uses jenkins_requests_latency so we can figure out if Jenkins is the culprit.
Contributor
Author
There was a problem hiding this comment.
Opened #485
Note though that it's unlike we are going to have slow syncs ever since kubernetes/test-infra#5774
derekhiggins
pushed a commit
to derekhiggins/release
that referenced
this pull request
Oct 24, 2023
…_found Fix/rook resources not found
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
@stevekuznetsov more docs about our alerts
cc @paradigm