[Feature][Jenkins-Data] Jenkins plugin option to add all the multibranch job to be monitored

[KrPetrovDev]

Search before asking

• I had searched in the issues and found no similar feature requirement.

Use case

The multibranch job is used for github pull requests which will lead for every pull request created will be created on the fly a new pipeline in the multibranch job which will not be included in the data scopes of the related jenkins connection.

Description

No response

Related issues

New pipelines created in the multibranch pipeline will not be added as a data scope in the jenkins connection automatically, they will be required to be inserted manually from the data scope UI of the jenkins connection

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[klesh]

Thanks for the report, however, I'm having trouble understanding the problem since I never used it before, do you have any concrete example of how it works?

[KrPetrovDev]

Yes, as an example I have a multi branch job in jenkins, which inside It executes multiple separated pipelines, these pipelines are created on demand as we create pull requests in github but the issue related to devlake is that the pipelines that are generated on demand are not triggered automatically by the jenkins connection in devlake, so every time I have to go to the jenkins connection, add the new pipelines and run again the process of recollecting the data from jenkins.

As all of these pipelines pertains to the same multi branch job I think It would be possible to develop a feature that will
recollect data from all the existing pipelines, inclusive the new pipelines generated after the previous recollection of metrics from this multi branch job instead of only recollecting data only from the pipelines that were initially selected when the jenkins connection was defined in the devlake project.

I hope that this explanation solves your doubt about how It works.

[klesh]

@KrPetrovDev Thanks for the comprehensive explanation, I get the idea of it.
However, to evaluate the possible support, we will need a working jenkins server with proper fake data so we can observe how and what the jenkins API offers, would you like to set up one?

[KrPetrovDev]

@klesh I cannot provide a server but I can provide you with a minimum installation guide which will help you to simulate part of my jenkins environment for PRs and to configure a github repository so Jenkins can communicate with It.

This is the guide where explains how to setup an existing jenkins server:

• https://www.youtube.com/watch?v=aDmeeVDrp0o&ab_channel=CloudBeesTV

[github-actions]

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

[klesh]

@Startrekzky Please take a look at this issue, thanks.

[klesh]

Collecting votes

[Startrekzky]

Yes, as an example I have a multi branch job in jenkins, which inside It executes multiple separated pipelines, these pipelines are created on demand as we create pull requests in github but the issue related to devlake is that the pipelines that are generated on demand are not triggered automatically by the jenkins connection in devlake

Hi @KrPetrovDev , may I ask if there are some rules for the naming, folder structure, or other patterns of the auto-created Jenkins jobs in the pipelines?

Currently, DevLake needs users to define the data scope with the absolute data scope id before collecting data, which is the Jenkins job path in this case.

@klesh If the on-premise-created Jenkins jobs follow certain rules, I guess the possible solution is to allow users to input a pattern (regex) to define the relative data scope pattern to collect.

Then this issue would be similar to #5002

[haozileung]

The jenkins mulitple branch pipeline is a job that contains multiple subjob and the subjob name is default to branch name or pull request or tags.

see this https://www.jenkins.io/zh/doc/tutorials/build-a-multibranch-pipeline-project/

[sstojak1]

Hi team, do we have any updates regarding this? Is it planned to be implemented?

[Startrekzky]

@sstojak1 I'm sorry that it's still in the backlog, as we haven't come up with a proper solution for this.

Adding Jenkins jobs by this multi-branch pipeline seems like adding a filter to get Jenkins jobs instead of specifying concrete jobs.