-
-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci.jenkins.io jobs on Windows agents are much slower than 21 days ago #3904
Comments
Pagerduty is notifying me of performance issues that may be on the publick8s cluster. |
Same here. Most of my jobs are timeout after 1h Example here : https://ci.jenkins.io/job/Plugins/job/hidden-parameter-plugin/job/PR-97/2/console Takes 15mn to execute some very simple maven tasks |
Another timeout failure at https://ci.jenkins.io/job/Core/job/jenkins-test-harness/job/PR-713/1/ |
Here also: jenkinsci/coverage-model#64 Was better some hours ago but timeout came back |
Update:
=> Can you let us know if you see a difference in performances? Something to note: it seems that all "slow" builds were handled by Azure Container Instance (e.g. Windows builds with the |
I've started the following jobs to compare performance:
Each of them had at least one build during the last two weeks that took more than 50 minutes to complete. Builds are definitely faster. No timeouts. Windows machines are within a reasonable range of past performance. |
As far as I can tell from my comparisons, this can be closed. They are likely more things to improve, but the most crucial items that I was monitoring have all been resolved. @jonesbusy are the results looking good enough for you that we can close this issue? |
Last timeout I saw was 3 days ago. I didn't saw any more more recently |
Service(s)
ci.jenkins.io
Summary
Jobs run on ci.jenkins.io Windows agents are significantly slower today than they were 21 days ago
Reproduction steps
Compare job execution times for the Windows portions of several plugins that I maintain. Dec 23 compared to today shows a large enough increase to cause most of the plugin BOM upgrade pull requests to those repositories to fail with a timeout error. The jobs that failed with a timeout error include:
There were a few pull requests at that same time that passed, but their build was dramatically longer than the typical build for their repository. The passing builds pull requests included:
Usually the update of the plugin bill of materials in those plugins is simple and fast. Current agents seem to be running much slower than agents were running previously.
The text was updated successfully, but these errors were encountered: