[CI] Pipeline with parallel ciGroups #44925
Comments
|
Pinging @elastic/kibana-operations |
|
Example run:
Blue Ocean screenshots: At the start (looks good):
After the OSS build completes and the ciGroups start, all of the other stages disappear from this view, even though they are still executing.
After all of the oss ciGroups finish, a few more stages appear, but all of the x-pack groups are missing, even though they are still executing.
Once the job completes, all of the stages are visible. The Pipeline Steps view always seems to be accurate and up-to-date. It's the main page I use, but it's hard to read. Most statements in your Jenkinsfile generate a step in this view, so there's a lot of noise. There are various APIs that various parts of Jenkins uses to display this information, but none of them have all of the information accurately. The Blue Ocean and Workflow APIs have the most information, but they are horribly inaccurate (which is why the Blue Ocean UI is so inaccurate). The two APIs that mirror the Pipeline Steps page are very accurate, but don't have a ton of information. Get all steps: https://kibana-ci.elastic.co/job/kibana-pipeline-pull-request/8/api/json?tree=actions%5Bnodes%5BdisplayName,id,parents%5D%5D The Blue Ocean UI is not too bad, once the job has finished. During the job, however, it's awful. I'd really like to get this work out without needing to also implement a separate UI for better tracking the jobs. Splitting the pipeline into 4 pipelines (basically one for each instance) would probably make the experience a lot better, but we lose control and simplicity, and it will bite us again as soon as a little more complexity is added to any of those pipelines. Thoughts? |
|
This has been rolled out for tracked branches. It will also be rolled out for PRs and is being tracked in #45954. Closing this issue. |
Create a new version of our CI process that uses Jenkins Pipelines (i.e. Jenkinsfile), rather than matrix jobs, to give us greater control over the job.
As part of this initial implementation, we can easily run all of the ciGroups in parallel on two large machines (one for oss, one for x-pack) and intake on two small machines. This would bring our required immutable worker count down to 4 (from 28).
The biggest downside to this approach is that there is not a good UI or API in Jenkins that provides all of the information for all of the individual parts of the jobs both accurately and in an easy-to-read format. I am still experimenting, and will provide examples of what we have, and we can determine from there whether or not we need a more robust solution before moving forward with pipelines.
The text was updated successfully, but these errors were encountered: