Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignore versions using regular expressions #529

Closed
hisener opened this issue Feb 13, 2020 · 13 comments
Closed

Ignore versions using regular expressions #529

hisener opened this issue Feb 13, 2020 · 13 comments

Comments

@hisener
Copy link

@hisener hisener commented Feb 13, 2020

Which Renovate are you using?

Renovate Open Source CLI (using the docker image renovate/renovate:19.124.2)

Which platform are you using?

GitHub.com

Have you checked the logs? Don't forget to include them if relevant

What would you like to do?

In Maven versions plugin you can specify various rules in version-number-rules.xml (Please refer https://www.mojohaus.org/versions-maven-plugin/version-rules.html). It is possible to ignore versions using regular expressions, e.g., <ignoreVersion type="regex">.*-RC\d+</ignoreVersion>.

Is there a way to achieve the same in renovate?

I have checked ignoreDeps but looks like it's only possible to ignore by dependency name. And there is also allowedVersions in packageRules but IIUC it doesn't support regex.

@rarkins

This comment has been minimized.

Copy link
Collaborator

@rarkins rarkins commented Feb 13, 2020

You can't ignore versions using regular expressions, but it might be possible to satisfy your requirements using Maven ranges in allowedVersions. What specifically are you hoping to ignore? Note that Renovate will ignore "unstable" upgrades by default, e.g. things like RC versions

@stale stale bot added the pending-closure label Feb 16, 2020
@hisener

This comment has been minimized.

Copy link
Author

@hisener hisener commented Feb 17, 2020

Sorry for the late response.

There are some cases we want to ignore. For instance, we ignore com.google.guava versions that match .*-android or com.fasterxml and .*pr\d+. They are not "unstable" but we simply want to avoid those.

It's a bit hard to achieve using semantic version ranges, I guess.

@stale stale bot removed the pending-closure label Feb 17, 2020
@renovatebot renovatebot deleted a comment from stale bot Feb 17, 2020
@rarkins

This comment has been minimized.

Copy link
Collaborator

@rarkins rarkins commented Feb 17, 2020

For guava, I've been meaning to make a preset to fix this for everybody. In short: guava's versioning is not maven-compliant because they extend the meaning of the suffix to mean "compatibility". This rule should work for you:

{
  "packageRules": [{
    "packageNames": ["com.google.guava:guava"],
    "versionScheme": "regex:^(?<major>\\d+)(\\.(?<minor>\\d+))?(\\.(?<patch>\\d+))?(-(?<compatibility>.*))?$"
  }]
}

com.fasterxml and .*pr\d+

Can you explain more?

@hisener

This comment has been minimized.

Copy link
Author

@hisener hisener commented Feb 17, 2020

They publish some preview releases to Maven central but they are not part of official releases. But IIUC, it's already marked as unstable. So, it's not a good example 😬

A better example would be: ignore version rule .*-(jboss-|M)\d+ for org.apache.maven.plugins. e.g., renovate tries to upgrade that.

INFO: DRY-RUN: Would create PR: Upgrade org.apache.maven.plugins:maven-compiler-plugin 3.8.1 -> 3.8.1-jboss-1

Another thing I've observed (I am not sure it's a bug or wrong usage of semantic maven versioning):

INFO: DRY-RUN: Would create PR: Upgrade dnsjava:dnsjava 2.1.9 -> 3.0.0-next.1

Renovate assumes 3.0.0-next.1 is newer than 3.0.0 for dnsjava. Maybe it's because of the dot (.) after next.

@rarkins

This comment has been minimized.

Copy link
Collaborator

@rarkins rarkins commented Feb 17, 2020

I could replicate 2 of the 3 examples you've given so far here: https://github.com/renovate-tests/maven6

dnsjava could be fixed by specifying regular semver versioning instead of Maven.

For maven-compiler-plugin I couldn't get it to do the incorrect upgrade.

If you have any more examples you need fixed, can you perhaps fork that repo and update the pom.xml?

What I'm thinking about doing now is creating a community-driven set of presets that contain version scheme overrides for packages that have done versioning wrong but can be fixed by Renovate config. That way once it's fixed once, everyone gets the benefit.

I'm also open to you raising a feature request to detect maven version plugin rules and replicate them, if you think that it would be a common usage requirement.

@hisener

This comment has been minimized.

Copy link
Author

@hisener hisener commented Feb 17, 2020

dnsjava could be fixed by specifying regular semver versioning instead of Maven.

👌

For maven-compiler-plugin I couldn't get it to do the incorrect upgrade.

It's because the dependency comes from another repository, not from Maven Central. https://github.com/hisener/maven6/blob/c88b7ef1e03ddfe441ac33cfa1ea6a99bc451e80/pom.xml#L28-L33
I could re-produce after adding the JBossEA repo, hisener/maven6#2.

What I'm thinking about doing now is creating a community-driven set of presets that contain version scheme overrides for packages that have done versioning wrong but can be fixed by Renovate config. That way once it's fixed once, everyone gets the benefit.

I think it would be nice. I can share our package rules, as well.

@viceice

This comment has been minimized.

Copy link
Collaborator

@viceice viceice commented Feb 17, 2020

Here some of mine regex rules.

{
    packageRules: [
        {
            packageNames: ["gitlab/gitlab-runner"],
            versionScheme: "regex:^(?<compatibility>.*)-v(?<major>\\d+)\\.(?<minor>\\d+)\\.(?<patch>\\d+)?$",
            groupName: "gitlab docker images",
        },
        {
            packageNames: ["gitlab/gitlab-ce"],
            versionScheme: "regex:^(?<major>\\d+)\\.(?<minor>\\d+)\\.(?<patch>\\d+)-(?<compatibility>ce\\.\\d+)$",
            groupName: "gitlab docker images",
        },
    ],
}
@rarkins

This comment has been minimized.

Copy link
Collaborator

@rarkins rarkins commented Feb 17, 2020

Strangely I can't reproduce it when I copy/paste your pom.xml:

DEBUG: Url not found (repository=renovate-tests/maven6)
       "failedUrl": "https://repository.jboss.org/nexus/content/repositories/ea/org/apache/maven/plugins/maven-compiler-plugin/maven-metadata.xml"
DEBUG: org.apache.maven.plugins:maven-compiler-plugin not found in repository https://repository.jboss.org/nexus/content/repositories/ea/ (repository=renovate-tests/maven6)

image

@hisener

This comment has been minimized.

Copy link
Author

@hisener hisener commented Feb 17, 2020

@rarkins I found another edge case: hisener/maven6#4. It tries to upgrade com.google.auto.service:auto-service from 1.0-rc6 -> 1.0-SNAPSHOT even though it doesn't exist on Maven Central.

@rarkins

This comment has been minimized.

Copy link
Collaborator

@rarkins rarkins commented Feb 17, 2020

@hisener can you isolate just one into its own reproduction repo and raise it as a bug report to the main Renovate repo?

BTW I checked out https://repo.maven.apache.org/maven2/com/google/auto/service/auto-service/maven-metadata.xml and verified there's no snapshot there either, but I can guarantee you it came from somewhere..

@hisener

This comment has been minimized.

Copy link
Author

@hisener hisener commented Feb 17, 2020

Strangely I can't reproduce it when I copy/paste your pom.xml:

I am using our Nexus, maybe it was available when our Nexus repository indexed it.

BTW I checked out https://repo.maven.apache.org/maven2/com/google/auto/service/auto-service/maven-metadata.xml and verified there's no snapshot there either, but I can guarantee you it came from somewhere.

Yes, indeed, our nexus repo has a reference to 1.0-SNAPSHOT in maven-metadata.xml. Even though we don't have 1.0-SNAPSHOT jar. 🤔

<metadata modelVersion="1.1.0">
    <groupId>com.google.auto.service</groupId>
    <artifactId>auto-service</artifactId>
    <version>1.0-rc6</version>
    <versioning>
        <latest>1.0-SNAPSHOT</latest>
        <release>1.0-rc6</release>
        <versions>
            <version>HEAD-SNAPSHOT</version>
            <version>1.0-rc1</version>
            <version>1.0-rc2</version>
            <version>1.0-rc3</version>
            <version>1.0-rc4</version>
            <version>1.0-rc5</version>
            <version>1.0-rc6</version>
            <version>1.0-SNAPSHOT</version>
        </versions>
        <lastUpdated>20190717034937</lastUpdated>
    </versioning>
</metadata>
@stale stale bot added the pending-closure label Feb 20, 2020
@renovatebot renovatebot deleted a comment from stale bot Feb 20, 2020
@stale stale bot removed the pending-closure label Feb 20, 2020
@rarkins

This comment has been minimized.

Copy link
Collaborator

@rarkins rarkins commented Feb 20, 2020

This issue kind of diverged in topic. @hisener could you create two issues in the main repo?

  • feature request to support regex allowing or ignoring of versions. I’m not sure which is most appropriate? Also please note if you think it appropriate for Renovate to auto detect this from any maven files in the repo

  • bug report about handling mismatches between the XML and the directories. I have an idea for this

@hisener

This comment has been minimized.

Copy link
Author

@hisener hisener commented Feb 21, 2020

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.