Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transitive snapshot dependency resolution flaky with coursier-sbt #1149

Open
jonaslipp opened this issue May 2, 2019 · 5 comments

Comments

@jonaslipp
Copy link

@jonaslipp jonaslipp commented May 2, 2019

Accessing timestamped snapshot artifacts often fails at first run and after multiple retries it works.

Lets assume the following module dependency tree:

      ------------
    /              \
modA ----> modB ----> modZ    
    \              /
      ---> modD --

All dependencies are snapshots with version task-1-SNAPSHOT. There are several timestamped snapshot versions for each module either on the nexus repo or in local cache.

If a new snapshot version of modZ is available on the nexus repo and I refresh the modA module, I often (not always!) get the following error:

coursier.error.ResolutionError$CantDownloadModule: Error downloading ch.scaling:modZ_2.12:task-1-SNAPSHOT
List(
not found: /Users/foobar/.m2/repository/ch/scaling/modZ_2.12/task-1-SNAPSHOT/modZ_2.12-task-1-SNAPSHOT.pom,
not found: /Users/foobar/.ivy2/local/ch.scaling/modZ_2.12/task-1-SNAPSHOT/ivys/ivy.xml
not found: https://nexus.foobar.ch/repository/maven-public/ch/scaling/modZ_2.12/task-1-SNAPSHOT/modZ_2.12-task-1-SNAPSHOT.pom 
not found: https://nexus.foobar.ch/repository/ivy/ch.scaling/modZ_2.12/task-1-SNAPSHOT/ivys/ivy.xml)
)

Some notes:

  • The issue occurs since we started using timestamped snapshot dependencies. All sbt-coursier v1.* versions are affected.
  • This is just a simplified subset of our dependency tree. There are much more modules with different transitive paths ending up at the same common module (like modZ)
  • The error disappears after refreshing several times, the latest modZ snapshot version was fetched correctly
  • IMHO The issue is located in the lookup URL to the nexus Repo https://nexus.foobar.ch/repository/maven-public/ch/scaling/modZ_2.12/task-1-SNAPSHOT/modZ_2.12-task-1-SNAPSHOT.pom which is wrong. It should first read the maven-metadata.xml to determine the correct path to the latest snapshot pom
    e.g. metadata: https://nexus.foobar.ch/repository/maven-public/ch/scaling/modZ/task-1-SNAPSHOT/maven-metadata.xml leads to pom at https://nexus.foobar.ch/repository/maven-public/ch/scaling/modZ/task-1-SNAPSHOT/modZ-task-1-20190430.054022-5.pom
  • Playing around with the cache TTL does not help
@jonaslipp

This comment has been minimized.

Copy link
Author

@jonaslipp jonaslipp commented May 2, 2019

One additional side note: Refreshing the whole tree with maven works just fine.

@alexarchambault

This comment has been minimized.

Copy link
Member

@alexarchambault alexarchambault commented May 3, 2019

@jonaslipp Some maven-metadata.xml errors get trapped when snapshot versioning is involved… I've got bitten by that very recently too. I fear we don't see the actual underlying error here.

I'm going to try to have them be printed / reported.

@jonaslipp

This comment has been minimized.

Copy link
Author

@jonaslipp jonaslipp commented May 6, 2019

@alexarchambault Good to see that the behaviour is reproducible by others. I tried to nail it down several times without success. Let me know if I can support with testing.

@danischroeter

This comment has been minimized.

Copy link

@danischroeter danischroeter commented May 28, 2019

@alexarchambault when you add those debug statements to see the metadata errors could you pls notify so we can use the adapted coursier and look for these errors and maybe help...

@mdedetrich

This comment has been minimized.

Copy link

@mdedetrich mdedetrich commented Dec 3, 2019

I also think I am getting the same problem, transitive dependencies that are snapshots have issues resolving correctly.

I have to manually fix this by explicitly defining the snapshot dependency and using force()

EDIT: This actually doesn't appear to reliably fix the issue, sometimes it resolves correctly and at other times it doesnt 🤷‍♂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.