[MRESOLVER-634] updatePolicy is ignored for version-range metadata requests

[jira-importer]

Bas van Erp opened MRESOLVER-634 and commented

maven-metadata.xml is refetched for every dependency which uses version-ranges if the resolver-status.properties file's "lastUpdated" property is before last midnight (local time, for some reason).

It ignores all updatePolicy configs in settings.xml

I have tried diving into the code, but as I'm not a programmer, I couldn't really wrap my head around it.

Setup

• Have a settings.xml with <updatePolicy>never</updatePolicy> for everything.
• Have a pom.xml with a dependency which uses a version-range.
• Run mvn validate to trigger the resolver and make sure all metadata and dependencies are downloaded and the Maven cache is warmed up.

Correct

• Use https://www.epochconverter.com/ to create a millisecond timestamp for today (local-time): 00:00:01
• Replace the xxx.xml.lastUpdated=### timestamp in the resolver-status.properties file of the dependency which is referenced in the pom.xml
• Run mvn validate -X to trigger the resolver. It should return something like: "Skipped remote request for x:y/maven-metadata.xml, locally cached metadata up-to-date"
• Good

Wrong

• Use https://www.epochconverter.com/ to create a millisecond timestamp for yesterday (local-time): 23:59:59
• Replace the xxx.xml.lastUpdated=### timestamp in the resolver-status.properties file of the dependency which is referenced in the pom.xml
• Run mvn validate to trigger the resolver.
• The resolver will download the maven-metadata.xml again. Every day.

Impact

We have some dependencies which use version ranges for a lot of transitive dependencies, and I hate it for multiple reasons. But to add insult to injury, every night the first build will refetch all these metadata files in order to see if version resolution needs to change. ;)

I have found no way to block this, since version-ranges seem to bypass repository updatePolicy settings.

---

Affects: 1.9.22

Issue Links:

• MNG-5997 Continuous download of maven-metadata.xml for version ranges.

[jira-importer]

Tamas Cservenak commented

I am removing this issue from Resolver 2.0.x, as I just realized that it is "not applicable" to it. Resolver 1.x has one update policy for remote repositories, while Resolver 2.x did split the update policies and supports separate policy for artifacts and separate policy for metadata. See MRESOLVER-377

[jira-importer]

Bas van Erp commented

Tamas Cservenak if there is any way we can assist with this issue, just let me know.

To give you some context: we use short-living (Kubernetes) Jenkins agents which get their own Maven dependency cache pre-filled as to reduce the load on our central artifact repository (Nexus). Since our dependencies don't change all that often, this works great.

Except for these version-range artifacts. And we're dependent on a little company that starts with an O and is owned by L.Ellison and they simply love version-ranges in their POM files.

So every single CI Maven build starts with retrieving these metadata files (hundreds of them), just to proceed as normal (because no update available/needed). Despite our <updatePolicy>never</updatePolicy> for all agents.

I fear that having this issue fixed/non-existent in Maven 4 will do little for us. Given our reliance on the aforementioned company, I don't think we'll be able to make the jump to Maven 4 any time soon after its release (if ever).

[jira-importer]

Tamas Cservenak commented

bverp After reconsidering, I still think that the issue is wrong, or in other words, your problem should be addressed differently, than making Maven metadata "never" update. Let me explain what happens, but before allow me a bit of a digression: As you know, we are maintaining two Resolver versions (1.x and 2.x) that are used in Maven 3 and Maven 4 respectively. One of the differences between Resolver 1.x and 2.x is that 1.x "implicitly" does these "daily updates" to metadata (as you discovered), while in 2.x we introduced split policies, and we have two of them: artifact policy and metadata policy. But still, IMO, you would be interested in prod solution (Maven 4 is far not prod, is not GA 4.0.0), but more importantly, making metadata updates "never" is not what you want.

Related doco: https://maven.apache.org/repositories/metadata.html

First, Maven has 3 kind/types of metadata, G level, A level and V level. G level is used to resolve Plugins (prefix -> A), A level is used to discover versions (ranges but also "latest" plugin version, see G level metadata) and V level is used to resolve SNAPSHOTs (when you write 1.0-SNAPSHOT to resolve it to latest "timestamped" snapshot). Second, update policy is applied once the content is downloaded.

Story: You start with empty local repository. You invoke mvn myplugin:goal, and as local repo is empty, Maven will download G level metadata for configured plugin group (assume plugin is in-house, company plugin, but same stands for ANY plugin in fact), then download A metadata, resolve latest version and download and invoke mojo. Then, next day, assume you in-house decide to create other "myotherplugin", so you develop it, build it, release and deploy it. And now, you want mvn myotherplugin:goal. Given G metadata is already present, and assuming your "never" policy is applied to metadata, Maven would fail here, as it did download from remote repository G metadata while myotherplugin was not yet deployed. But same stands for A metadata: once you "fix" G metadata, and prefix is found, A metadata would need "fix" to pick up latest version, like the version you just deployed today. Due policy, Maven would not retry, would consider metadata "immutable". Hence, the prefix myotherplugin would not be discovered. (and very same story stands for snapshot resolution as well, Maven would download once, and stop getting info about later daily deploys).

Can you do something like in this reproducer?
https://github.com/cstamas/MRESOLVER-634

Explanation to reproducer:

• dependency is the artifact produced by O-prefixed company and uses range
• dependant is your project, suffers from ranges as it depends on dependency
• dependant-fix prevents range requests totally