Version ranges (dynamic revision) have issues with bintray

[jroper]

steps

build.sbt:

lazy val root = (project in file(".")).
  settings(
    libraryDependencies += "org.webjars" %% "webjars-play" % "2.1.0-3",
    resolvers += Resolver.typesafeRepo("releases")
  )

problem

root> update
[info] Updating {file:/Users/eugene/work/quick-test/sbt-2005/}root...
[info] Resolving play#play_2.10;[2.1.0,2.1.2] ...
[warn]  module not found: play#play_2.10;[2.1.0,2.1.2]
[warn] ==== local: tried
[warn]   /Users/eugene/.ivy2/local/play/play_2.10/[revision]/ivys/ivy.xml
[warn] ==== public: tried
[warn]   https://repo1.maven.org/maven2/play/play_2.10/[revision]/play_2.10-[revision].pom
[warn] ==== typesafe-releases: tried
[warn]   https://repo.typesafe.com/typesafe/releases/play/play_2.10/[revision]/play_2.10-[revision].pom
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: play#play_2.10;[2.1.0,2.1.2]: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn] 
[warn]  Note: Unresolved dependencies path:
[warn]      play:play_2.10:[2.1.0,2.1.2]
[warn]        +- org.webjars:webjars-play_2.10:2.1.0-3 (<set>#L1-1)
[warn]        +- root:root_2.10:0.1-SNAPSHOT
[trace] Stack trace suppressed: run last *:update for the full output.
[error] (*:update) sbt.ResolveException: unresolved dependency: play#play_2.10;[2.1.0,2.1.2]: not found
[error] Total time: 10 s, completed Jun 24, 2015 5:43:57 PM

original report

See https://groups.google.com/d/msg/play-framework/ES8WDB0pIA0/Q8bYH9jaRZoJ

And comment from @jsuereth:

Version ranges are generally a really bad idea in Ivy. I need to check implementation, but I think it tries to do a directory listing using nonstandard techniques.

[andersonvass]

Is there any estimate of when this problem will be resolved?

[eed3si9n]

I spent some time looking into this, and @jsuereth is right.

MavenRepository is converted to a custom IBiblioResolver by sbt, so you can override its methods to track what's going on.

1. The chain resolver calls getDependency(dd, data)
2. It calls findResourceUsingPatterns that takes patternList
3. It calls findResourceUsingPattern that takes single pattern
4. It calls listResources when the requested revision (version in our lingo) is dynamic

listResources will either use maven-metadata.xml or ApacheURLLister.
Bintray's Maven repository does not have maven-metadata.xml, so I've set resolver.setUseMavenMetadata(false) for Bintray repos. So now all we have is ApacheURLLister.

scala> import org.apache.ivy.util.url.ApacheURLLister
import org.apache.ivy.util.url.ApacheURLLister

scala> val lister = new ApacheURLLister
lister: org.apache.ivy.util.url.ApacheURLLister = org.apache.ivy.util.url.ApacheURLLister@29859ec3

scala> import java.net.URL
import java.net.URL

scala> lister.listAll(new URL("https://repo.typesafe.com/typesafe/releases/play/play_2.10/")).toArray.toList
res0: List[Object] = List()

scala> lister.listAll(new URL("https://dl.bintray.com/typesafe/maven-releases/play/play_2.10/")).toArray.toList
res1: List[Object] = List()

scala> lister.listAll(new URL("https://oss.sonatype.org/content/repositories/public/com/typesafe/play/play_2.11/")).toArray.toList
res2: List[Object] = List(https://oss.sonatype.org/content/repositories/public/com/typesafe/play/play_2.11/2.3.9/, https://oss.sonatype.org/content/repositories/public/com/typesafe/play/play_2.11/2.4-2014-09-01-6344061-SNAPSHOT/, https://oss.sonatype.org/content/repositories/public/com/typesafe/play/play_2.11/2.4-2014-09-02-3516a0a-SNAPSHOT/, https://oss.sonatype.org/content/repositories/public/com/typesafe/play/play_2.11/2.4-2014-09-03-3516a0a-SNAPSHOT/, https://oss.sonatype.org/content/repositories/public/com/typesafe/play/play_2.11/2.4-2014-09-04-3516a0a-SNAPSHOT/, https://oss.sonatype.org/content/repositories/public/com/typesafe/play/play_2.11/2.4-2014-09-05-3516a0a-SNAPSHOT/, https://oss.sonatype.org/content/repositories/public/com/typesafe/play/play_2.11/2.4-2014-09-06-08c504a-SNA...

This likely require us writing a screen scraper compat with Bintray's html page, or Bintray somehow faking the html output of Apache.

[eed3si9n]

When I download the source of https://dl.bintray.com/typesafe/maven-releases/play/play_2.10/ and remove the # characters, I can scrape it using ApacheURLLister.

<pre><a onclick="navi(event)" href="#2.1-10082012/" rel="nofollow">2.1-10082012/</a></pre>
<pre><a onclick="navi(event)" href="#2.1-10112012/" rel="nofollow">2.1-10112012/</a></pre>

[eed3si9n]

Some new information has come to light.

Bintray support confirmed that missing maven-metadata.xml was a glitch on their part, and it is now restored. When I tried again, it still did not resolve.

When I went digging into Ivy's code base I discovered that it's been checking if the repository layout pattern ends with M2_PATTERN to use maven-metadata.xml, which for sbt would return false since we customize the mattern - https://github.com/apache/ant-ivy/blob/2.3.0/src/java/org/apache/ivy/plugins/resolver/IBiblioResolver.java#L497-L499
PR coming up shortly for this fix.