-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add scrapeTimeout as global configurable parameter #3250
Conversation
lgtm on green |
Tests seem flaky. Fails seems not related to the change as I looked into the PR test runs and lots fail around |
hmm I kicked the jobs a couple of times, I think there is actually something not working |
It seems be consistently failing on |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR! One comment, otherwise can you adjust the test as Paul said, thanks!
pkg/prometheus/promcfg.go
Outdated
@@ -177,6 +177,11 @@ func (cg *configGenerator) generateConfig( | |||
scrapeInterval = p.Spec.ScrapeInterval | |||
} | |||
|
|||
scrapeTimeout := "10s" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Whats the reason for the 10seconds here, might have missed something?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's the default timeout. Even changing to 30s the same test fails. I don't think it is related to the change.
I've reverted the timeout to 10s (the default Prometheus scrape_timeout) and added a 60s timeout for the failing test. |
Add global configurable scrapeTimeout parameter to allow monitoring targets on clusters consisting of slower hosts like Raspberry Pi and many ARM boards used for labs. Signed-off-by: Carlos de Paula <me@carlosedp.com>
Now we're good to go. Better and cleaner implementation, also Travis is green! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
Add global configurable scrapeTimeout parameter to allow monitoring
targets on clusters consisting of slower hosts like Raspberry Pi and
many ARM boards used for labs.
Many times the 10s default timeout is not enough for targets like kube-controller or kube-scheduler so some targets get a "Context deadline exceeded" error.
Signed-off-by: Carlos de Paula me@carlosedp.com