Switch to Renovate from Dependabot for remaining dependencies

[timja]

See jenkins-infra/helpdesk#4173

We were already using renovate for JavaScript dependencies, this switches us to using it for the remaining ones.
Primary motivator is buggy handling of SNAPSHOTS in dependabot.

Testing done

Tested on my org fork (my personal fork was getting auth failures for some reason):
https://github.com/timja-org/jenkins/pulls?q=sort%3Aupdated-desc+is%3Apr+is%3Aopen

I downgraded remoting on my fork and verified it got upgraded correctly: timja-org#6

Full result of detected packages / package managers:
https://developer.mend.io/github/timja-org/jenkins

Proposed changelog entries

N/A

Proposed upgrade guidelines

N/A

Submitter checklist

Beta Give feedback

1. The Jira issue, if it exists, is well-described.
2. The changelog entries and upgrade guidelines are appropriate for the audience affected by the change (users or developers, depending on the change) and are in the imperative mood (see examples). Fill in the Proposed upgrade guidelines section only if there are breaking changes or changes that may require extra steps from users during upgrade.
3. There is automated testing or an explanation as to why this change has no tests.
4. New public classes, fields, and methods are annotated with @Restricted or have @since TODO Javadocs, as appropriate.
5. New deprecations are annotated with @Deprecated(since = "TODO") or @Deprecated(forRemoval = true, since = "TODO"), if applicable.
6. New or substantially changed JavaScript is not defined inline and does not call eval to ease future introduction of Content Security Policy (CSP) directives (see documentation).
7. For dependency updates, there are links to external changelogs and, if possible, full differentials.
8. For new APIs and extension points, there is a link to at least one consumer.

Desired reviewers

N/A

Before the changes are marked as ready-for-merge:

Maintainer checklist

Beta Give feedback

1. There are at least two (2) approvals for the pull request and no outstanding requests for change.
2. Conversations in the pull request are over, or it is explicit that a reviewer is not blocking the change.
3. Changelog entries in the pull request title and/or Proposed changelog entries are accurate, human-readable, and in the imperative mood.
4. Proper changelog labels are set so that the changelog can be generated automatically.
5. If the change needs additional upgrade steps from users, the upgrade-guide-needed label is set and there is a Proposed upgrade guidelines section in the pull request title (see example).
6. If it would make sense to backport the change to LTS, a Jira issue must exist, be a Bug or Improvement, and be labeled as lts-candidate to be considered (see query).

[timja]

I don't think this ever worked, see https://github.com/jenkinsci/jenkins/pulls?q=sort%3Aupdated-desc+is%3Apr+is%3Aclosed+author%3Aapp%2Fdependabot+label%3Ainto-lts

so I haven't kept it for going forward

[basil]

I don't think this ever worked

Sad, given that it cost me so much to get that change accepted. I still think this is something that we want to automate, and we essentially do it manually for every LTS release, including recent ones.

[timja]

Restored in 74eee38

had to debug a fair but and used:

RENOVATE_CONFIG_FILE=.github/renovate.json renovate --token=$GITHUB_TOKEN --schedule=""  --require-config=ignored --dry-run=full timja-org/jenkins

[timja]

Here are the LTS PRs that would be created: https://github.com/timja-org/jenkins/pulls?q=sort%3Aupdated-desc+is%3Apr+is%3Aopen+label%3Ainto-lts

(some remoting ones would disappear by backporting this PR to LTS)

[MarkEWaite]

The proposed list of updates seems very reasonable to me.

The args4j 2.37 release that is included in the list of proposed upgrades was released in March 2024. Good that renovate detected it. I've not done any investigation to see if the upgrade is safe, since it will need evaluation in remoting as well as Jenkins core.

[basil]

Is this able to upgrade detached plugins like Dependabot?

[timja]

Is this able to upgrade detached plugins like Dependabot?

It's upgrading war/pom.xml but not test/pom.xml for some reason:

timja-org#8

https://github.com/timja-org/jenkins/blob/b673f0d8108d2d03b5bb34d536060dc6741b107b/test/pom.xml#L353

Looking in to it

---

Pom files found:

DEBUG: Matched 9 file(s) for manager maven: .mvn/extensions.xml, bom/pom.xml, cli/pom.xml, core/pom.xml, coverage/pom.xml, pom.xml, war/pom.xml, websocket/jetty10/pom.xml, websocket/spi/pom.xml

I checked all poms and only test/pom.xml is missing

---

I see this setting in logs:

DEBUG: Found repo ignorePaths
{
  "ignorePaths": [
    "**/node_modules/**",
    "**/bower_components/**",
    "**/vendor/**",
    "**/examples/**",
    "**/__tests__/**",
    "**/test/**",
    "**/tests/**",
    "**/__fixtures__/**"
  ]
}

[timja]

Is this able to upgrade detached plugins like Dependabot?

Yes, all working now that test isn't ignored: timja-org#8

[timja]

It is now trying to upgrade old-remoting-for-test and remoting.minimum.supported.version I'll see if I can stop that

[timja]

It is now trying to upgrade old-remoting-for-test and remoting.minimum.supported.version I'll see if I can stop that

@viceice is it possible to stop it updating those two intentionally old deps?

timja-org#10
timja-org#9

[viceice]

yes, with a package rule and an allowed version filter. you can also disable that package or require dashboard approval.

[timja]

yes, with a package rule and an allowed version filter. you can also disable that package or require dashboard approval.

Hmm we still want the regular dependency one to work, timja-org#6
Just not the 'minimum version' or the 'old version to test'.

Would that still work?

I couldn't see a way to filter out a dependency from certain paths

Dashboard approval could maybe work

[viceice]

yes, with a package rule and an allowed version filter. you can also disable that package or require dashboard approval.

Hmm we still want the regular dependency one to work, timja-org#6
Just not the 'minimum version' or the 'old version to test'.

Would that still work?

I couldn't see a way to filter out a dependency from certain paths

Dashboard approval could maybe work

this can do it:
https://docs.renovatebot.com/configuration-options/#matchfilenames

[timja]

This not right? https://github.com/timja-org/jenkins/blob/37f95213954136508d215a4972d701bae1689a8b/.github/renovate.json#L71

[viceice]

This not right? https://github.com/timja-org/jenkins/blob/37f95213954136508d215a4972d701bae1689a8b/.github/renovate.json#L71

no. you're missing an action. that rule matches the file and excludes that dep. but you don't say what renovate should do. so add enabled:false to disable or allowed version to restrict only by version.

[timja]

Thanks that fixed one of them.

Surprisingly it didn't close this one though: timja-org#9

(It wasn't picked up when the entire file was ignored before adjusting ignore patterns)

Any idea?

[viceice]

I'll try to have a look again tomorrow, currently on holiday 🙃

[timja]

@viceice you have a chance to check it?

[viceice]

@viceice you have a chance to check it?

not yet, sorry. feel free to open a support discussion on the renovate repo. maybe one of my co-maintainer will have more time to look at it. I'm back on track on Monday.

[timja]

Hey @viceice any thoughts?

[timja]

raised config help at renovatebot/renovate#30555

[timja]

All working now, @basil / others any concerns?

[viceice]

this rule doesn't match pom.xml, so renovate still suggests an update for org.jenkins-ci.main:remoting inside the root pom.xml

[timja]

yeah there's one org.jenkins-ci.main:remoting that we want updated, the remoting.version one which is the latest version.

We don't want the 'old version for testing', and the 'minimum supported version' updated. Which is all working with this setup now by moving it to .mvn/maven.config

[viceice]

Thanks that fixed one of them.

Surprisingly it didn't close this one though: timja-org#9

(It wasn't picked up when the entire file was ignored before adjusting ignore patterns)

Any idea?

See my code comment for the solution

https://github.com/jenkinsci/jenkins/pull/9459/files/5895e0ebe23dea21644e42d3a7c32e4f63295615#diff-f82f7d36a61d22f640c25bdb8b0435bf9cb38679d99d5f460753fa668ee7cdb3R72

[timja]

I hit a bunch of weird behaviour when I added baseBranches in, matchPackageNames wildcarding and regex didn't seem to work.

I've managed to fix almost everything although this rule isn't working right now:

    {
      "matchPackageNames": ["net.jcip:jcip-annotations"],
      "matchDatasources": ["maven"],
      "enabled": false,
      "description": "maven-metadata.xml is missing for this really old package which is required by renovate"
    }

its still complaining about not being able to resolve it and if I just drop it to master or stable branch only then its fine with that rule (not a blocker anyway it just fixes a warning).

[viceice]

looks good

[basil]

Thanks for the PR!

[timja]

/label ready-for-merge

---

This PR is now ready for merge, after ~24 hours, we will merge it if there's no negative feedback.

Thanks!

[github-actions]

Please take a moment and address the merge conflicts of your pull request. Thanks!

[basil]

There is a problem visible in #9529: Renovate is trying to update the GitHub Action to the latest commit, not the latest tagged commit. We want the same behavior as Dependabot, where updates are only offered once a commit is tagged.

[basil]

There is a problem visible in https://github.com/jenkinsci/jenkins/pulls: Renovate is trying to backport all Java library updates, not just security updates. We want the same behavior we were trying to enable in Dependabot, where only security updates are backported.

[viceice]

add a version tag comment, that is understood by dependabot and renovate. then they follow and update that tag.

eg uses: org/repo@<sha> # v1.2.3

https://docs.renovatebot.com/modules/manager/github-actions/

[basil]

There is a problem visible in https://github.com/jenkinsci/jenkins/pulls: Renovate is trying to backport all Java library updates, not just security updates. We want the same behavior we were trying to enable in Dependabot, where only security updates are backported.

@timja Can you try re-adding the security updates with osvVulnerabilityAlerts using matchBaseBranches inside packageRules (per renovatebot/renovate#20542 (reply in thread))? vulnerabilityAlerts seems to be a non-starter, as it uses the GitHub Dependabot alerts (which only work on the default branch).

[basil]

@timja https://github.com/jenkinsci/jenkins-test-harness/releases/tag/2254.vcff7a_d4969e5 was released 7 hours ago, but Renovate does not seem to be offering it to Jenkins core, even after manually triggering Renovate. Dependabot is offering the update in other repositories, such as jenkinsci/plugin-pom#984.

[timja]

@timja jenkinsci/jenkins-test-harness@2254.vcff7a_d4969e5 (release) was released 7 hours ago, but Renovate does not seem to be offering it to Jenkins core, even after manually triggering Renovate. Dependabot is offering the update in other repositories, such as jenkinsci/plugin-pom#984.

resolved

@timja Can you try re-adding the security updates with osvVulnerabilityAlerts using matchBaseBranches inside packageRules (per renovatebot/renovate#20542 (reply in thread))? vulnerabilityAlerts seems to be a non-starter, as it uses the GitHub Dependabot alerts (which only work on the default branch).

I'm trying to get it working, haven't managed yet though.