Enable CD on `plugin-compat-tester` & clean up obsolete modules #2103

jglick · 2021-09-28T17:59:30Z

Description

While working on jenkinsci/bom#659 I realized we are still waiting for some person to mvn -f plugin-compat-tester release:{prepare,perform} on occasion, which makes little sense for a developer tool—we should just be able to use whatever is in master.

Also I am removing modules unused as of jenkinsci/plugin-compat-tester#193.

CC @imonteroperez as apparent primary maintainer, though only @raul-arabaolaza is listed in RPU for now.

Reviewer checklist (not for requesters!)

Check this if newly added person also needs to be given merge permission to the GitHub repo (please @ the people/person with their GitHub username in this issue as well). If needed, it can be done using an IRC Bot command
Check that the $pluginId Developers team has Admin permissions while granting the access.
In the case of plugin adoption, ensure that the Jenkins Jira default assignee is either removed or changed to the new maintainer.
If security contacts are changed (this includes add/remove), ping the security officer (currently @daniel-beck) in this pull request. If an email contact is changed, wait for approval from the security officer.

There are IRC Bot commands for it

jglick · 2021-09-28T18:01:34Z

Also CC @bmunozm, active in last year (but again not currently authorized in RPU).

timja · 2021-09-28T19:05:00Z

permissions/component-plugins-compat-tester-aggregator.yml

@@ -3,6 +3,8 @@ name: "plugins-compat-tester-aggregator"
 github: "jenkinsci/plugin-compat-tester"
 paths:
 - "org/jenkins-ci/tests/plugins-compat-tester-aggregator"
+cd:


needs to be combined in one file for multi module to work iirc

Mm, maybe. #1747. @daniel-beck any idea? Would it suffice as a workaround to just add a cd section to one component file?

https://ci.jenkins.io/job/Infra/job/repository-permissions-updater/job/PR-2103/1/artifact/json/cd.index.json looks OK.

While

repository-permissions-updater/src/main/groovy/io/jenkins/infra/repository_permissions_updater/ArtifactoryPermissionsUpdater.groovy

Lines 89 to 95 in fe278d5

List<Definition> definitions = cdEnabledComponentsByGitHub[definition.github]

if (!definitions) {

definitions = new ArrayList<>()

cdEnabledComponentsByGitHub[definition.github] = definitions

}

LOGGER.log(Level.INFO, "CD-enabled component '${definition.name}' in repository '${definition.github}'")

definitions.add(definition)

captures each YAML file associated with a given repo and cd.enabled, it seems from

repository-permissions-updater/src/main/groovy/io/jenkins/infra/repository_permissions_updater/ArtifactoryPermissionsUpdater.groovy

Lines 222 to 236 in fe278d5

cdEnabledComponentsByGitHub.each { githubRepo, components ->

def groupName = ArtifactoryAPI.getInstance().toGeneratedGroupName(githubRepo)

File outputFile = new File(new File(apiOutputDir, 'groups'), groupName + '.json')

JsonBuilder json = new JsonBuilder()

json {

name groupName

description "CD group with permissions to deploy from ${githubRepo}"

}

String pretty = json.toPrettyString()

outputFile.parentFile.mkdirs()

outputFile.text = pretty

}

and

repository-permissions-updater/src/main/groovy/io/jenkins/infra/repository_permissions_updater/ArtifactoryPermissionsUpdater.groovy

Line 247 in fe278d5

cdRepos(cdEnabledComponentsByGitHub.keySet().toList())

that the values are actually unused. IOW it should not even matter—so long as at least one file related to this repo enables CD, we should get the right user and secrets created—though it would have been clearer for

repository-permissions-updater/src/main/groovy/io/jenkins/infra/repository_permissions_updater/ArtifactoryPermissionsUpdater.groovy

Line 61 in fe278d5

Map<String, List<Definition>> cdEnabledComponentsByGitHub = new TreeMap<>()

to be a TreeSet<String>.

timja · 2021-09-28T19:36:17Z

Looking at the API:
https://www.jfrog.com/confluence/display/JFROG/Security+Configuration+JSON#SecurityConfigurationJSON-application/vnd.org.jfrog.artifactory.security.PermissionTarget+json

We're assigning the group to the special user that gets created which has an includes pattern.

As far as I can tell that include pattern will not be applied to the right paths unless I'm understanding wrong

timja · 2021-09-28T19:43:19Z

I think it's a last one wins sort of thing from reading it, the group name for each will be normalised to the same group as it's keyed off the GitHub repo, but the paths that are used will likely be the last one

repository-permissions-updater/src/main/groovy/io/jenkins/infra/repository_permissions_updater/ArtifactoryPermissionsUpdater.groovy

Line 155 in fe278d5

path + '/*/' + definition.name + '-*',

jglick · 2021-09-28T19:53:18Z

My reading of

repository-permissions-updater/src/main/groovy/io/jenkins/infra/repository_permissions_updater/ArtifactoryPermissionsUpdater.groovy

Lines 208 to 209 in 5824223

    
           if (definition.cd?.enabled) { 
        
               groups([(ArtifactoryAPI.getInstance().toGeneratedGroupName(definition.github)): ["w", "n"]])

is that each repo will get one generated group name, but this group might be added as a principal to multiple paths, one per source file and thus permissions/*.json output.

timja · 2021-09-28T19:56:59Z

ah possibly, we can try it anyway.

jglick · 2021-09-28T19:58:43Z

$ (mvn -DskipTests package && java -DdryRun=true -DdefinitionsDir=$PWD/permissions -DartifactoryApiTempDir=$PWD/json -DartifactoryUserNamesJsonListUrl=https://reports.jenkins.io/artifactory-ldap-users-report.json -Djava.util.logging.SimpleFormatter.format="%1$tY-%1$tm-%1$td %1$tH:%1$tM:%1$tS %4$s: %5$s%6$s%n" -jar target/repository-permissions-updater-*-bin/repository-permissions-updater-*.jar) 2>&1 | fgrep plugin-compat-tester
2021-09-28 19:58:04.293+0000 [id=1]	INFO	o.c.g.r.c.PojoMetaMethodSite$PojoCachedMethodSiteNoUnwrap#invoke: CD-enabled component 'plugins-compat-tester-cli' in repository 'jenkinsci/plugin-compat-tester'
2021-09-28 19:58:04.435+0000 [id=1]	INFO	o.c.g.r.c.PojoMetaMethodSite$PojoCachedMethodSiteNoUnwrap#invoke: CD-enabled component 'plugins-compat-tester' in repository 'jenkinsci/plugin-compat-tester'
2021-09-28 19:58:04.658+0000 [id=1]	INFO	o.c.g.r.c.PojoMetaMethodSite$PojoCachedMethodSiteNoUnwrap#invoke: CD-enabled component 'plugins-compat-tester-aggregator' in repository 'jenkinsci/plugin-compat-tester'
2021-09-28 19:58:04.826+0000 [id=1]	INFO	o.c.g.r.c.PojoMetaMethodSite$PojoCachedMethodSiteNoUnwrap#invoke: CD-enabled component 'plugins-compat-tester-model' in repository 'jenkinsci/plugin-compat-tester'
2021-09-28 19:58:05.155+0000 [id=1]	INFO	o.c.g.r.c.PojoMetaMethodSite$PojoCachedMethodSiteNoUnwrap#invoke: Dry-run mode: Skipping PUT call to https://repo.jenkins-ci.org/api/security/groups/generatedv2-cd-jenkinsci_plugin-compat-tester
2021-09-28 19:58:05.288+0000 [id=1]	INFO	o.c.g.r.c.PojoMetaMethodSite$PojoCachedMethodSiteNoUnwrap#invoke: Processing repository jenkinsci/plugin-compat-tester for CD
2021-09-28 19:58:05.288+0000 [id=1]	INFO	o.c.g.r.c.PojoMetaMethodSite$PojoCachedMethodSiteNoUnwrap#invoke: Skipped creation of token for GitHub repo: 'jenkinsci/plugin-compat-tester', Artifactory user: 'CD-for-jenkinsci__plugin-compat-tester', group name: 'generatedv2-cd-jenkinsci_plugin-compat-tester', valid for 14400 seconds

timja

Want this merged since you’re listed here or wait for someone else?

jglick · 2021-09-28T20:04:37Z

In json/, we have github.index.json with

    "jenkinsci/plugin-compat-tester": [
        "org/jenkins-ci/tests/plugins-compat-tester",
        "org/jenkins-ci/tests/plugins-compat-tester-aggregator",
        "org/jenkins-ci/tests/plugins-compat-tester-cli",
        "org/jenkins-ci/tests/plugins-compat-tester-model"
    ],

and cd.index.json with

    "jenkinsci/plugin-compat-tester",

and groups/generatedv2-cd-jenkinsci_plugin-compat-tester.json with

{
    "name": "generatedv2-cd-jenkinsci_plugin-compat-tester",
    "description": "CD group with permissions to deploy from jenkinsci/plugin-compat-tester"
}

and then permissions/generatedv2-component-plugins-compat-tester{,-aggregator,-cli,-model}.json with e.g.

{
    "name": "generatedv2-component-plugins-compat-tester-cli",
    "includesPattern": "org/jenkins-ci/tests/plugins-compat-tester-cli/*/plugins-compat-tester-cli-*,org/jenkins-ci/tests/plugins-compat-tester-cli/*/maven-metadata.xml,org/jenkins-ci/tests/plugins-compat-tester-cli/*/maven-metadata.xml.*,org/jenkins-ci/tests/plugins-compat-tester-cli/maven-metadata.xml,org/jenkins-ci/tests/plugins-compat-tester-cli/maven-metadata.xml.*",
    "excludesPattern": "",
    "repositories": [
        "snapshots",
        "releases"
    ],
    "principals": {
        "users": {
            "kwhetstone": [
                "w",
                "n"
            ],
            "jglick": [
                "w",
                "n"
            ],
            "andresrc": [
                "w",
                "n"
            ],
            "rarabaolaza": [
                "w",
                "n"
            ],
            "oleg_nenashev": [
                "w",
                "n"
            ]
        },
        "groups": {
            "generatedv2-cd-jenkinsci_plugin-compat-tester": [
                "w",
                "n"
            ]
        }
    }
}

which seems like it should work.

I am also tempted to just deploy only the plugins-compat-tester-cli, which is the thing you actually need. The other things are just intermediate build products.

jglick · 2021-09-28T20:06:22Z

Want this merged since you’re listed here or wait for someone else?

I guess it can be merged, provided that you are able to watch the deployment on trusted CI in case something blows up.

oleg-nenashev

+1 from me as a maintainer. CD is helpful there

jglick · 2021-10-18T12:56:43Z

I will try to prep this as soon as I get a moment.

jglick · 2021-11-23T16:26:06Z

permissions/component-plugins-compat-tester-cli.yml

@@ -3,6 +3,8 @@ name: "plugins-compat-tester-cli"
 github: "jenkinsci/plugin-compat-tester"
 paths:
 - "org/jenkins-ci/tests/plugins-compat-tester-cli"
+cd:
+  enabled: true


Note jenkinsci/plugin-compat-tester#320 (comment): could perhaps omit CD on the other modules, or if jenkinsci/plugin-compat-tester#320 is merged, simply remove the other modules from RPU altogether (would only ever be installed locally).

Enable CD on plugin-compat-tester & clean up obsolete modules

5824223

jglick requested a review from a team as a code owner September 28, 2021 17:59

timja reviewed Sep 28, 2021

View reviewed changes

timja approved these changes Sep 28, 2021

View reviewed changes

oleg-nenashev approved these changes Oct 17, 2021

View reviewed changes

oleg-nenashev merged commit 9501ef6 into jenkins-infra:master Oct 17, 2021

jglick deleted the plugin-compat-tester branch October 18, 2021 12:55

jglick mentioned this pull request Nov 23, 2021

Enable CD jenkinsci/plugin-compat-tester#320

Merged

jglick commented Nov 23, 2021

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable CD on `plugin-compat-tester` & clean up obsolete modules #2103

Enable CD on `plugin-compat-tester` & clean up obsolete modules #2103

jglick commented Sep 28, 2021

jglick commented Sep 28, 2021

timja Sep 28, 2021

jglick Sep 28, 2021

jglick Sep 28, 2021

jglick Sep 28, 2021

timja commented Sep 28, 2021

timja commented Sep 28, 2021 •

edited

jglick commented Sep 28, 2021

timja commented Sep 28, 2021

jglick commented Sep 28, 2021

timja left a comment

jglick commented Sep 28, 2021

jglick commented Sep 28, 2021

oleg-nenashev left a comment

jglick commented Oct 18, 2021

jglick Nov 23, 2021

	List<Definition> definitions = cdEnabledComponentsByGitHub[definition.github]
	if (!definitions) {
	definitions = new ArrayList<>()
	cdEnabledComponentsByGitHub[definition.github] = definitions
	}
	LOGGER.log(Level.INFO, "CD-enabled component '${definition.name}' in repository '${definition.github}'")
	definitions.add(definition)

	cdEnabledComponentsByGitHub.each { githubRepo, components ->
	def groupName = ArtifactoryAPI.getInstance().toGeneratedGroupName(githubRepo)
	File outputFile = new File(new File(apiOutputDir, 'groups'), groupName + '.json')
	JsonBuilder json = new JsonBuilder()

	json {
	name groupName
	description "CD group with permissions to deploy from ${githubRepo}"
	}

	String pretty = json.toPrettyString()

	outputFile.parentFile.mkdirs()
	outputFile.text = pretty
	}

Enable CD on plugin-compat-tester & clean up obsolete modules #2103

Enable CD on plugin-compat-tester & clean up obsolete modules #2103

Conversation

jglick commented Sep 28, 2021

Description

Reviewer checklist (not for requesters!)

jglick commented Sep 28, 2021

timja Sep 28, 2021

Choose a reason for hiding this comment

jglick Sep 28, 2021

Choose a reason for hiding this comment

jglick Sep 28, 2021

Choose a reason for hiding this comment

jglick Sep 28, 2021

Choose a reason for hiding this comment

timja commented Sep 28, 2021

timja commented Sep 28, 2021 • edited

jglick commented Sep 28, 2021

timja commented Sep 28, 2021

jglick commented Sep 28, 2021

timja left a comment

Choose a reason for hiding this comment

jglick commented Sep 28, 2021

jglick commented Sep 28, 2021

oleg-nenashev left a comment

Choose a reason for hiding this comment

jglick commented Oct 18, 2021

jglick Nov 23, 2021

Choose a reason for hiding this comment

Enable CD on `plugin-compat-tester` & clean up obsolete modules #2103

Enable CD on `plugin-compat-tester` & clean up obsolete modules #2103

timja commented Sep 28, 2021 •

edited