Version Catalog API: Allow version variable in library coordinate string

[jnehlmeier]

The new Version Catalog API is great, but as soon as you want to extract the library version into its own section the required syntax in the libraries block is way too complex for the typical use case.

Expected Behavior

[versions]
groovy = "3.0.5"

[libraries]
# Example syntax. More concise for typical use case
groovy-core = "org.codehaus.groovy:groovy:${groovy}"

Current Behavior

[versions]
groovy = "3.0.5"

[libraries]
groovy-core = { module = "org.codehaus.groovy:groovy", version.ref = "groovy" }

[jnehlmeier]

A version variable would also allow specifying a classifier, e.g.

[libraries]
groovy-core-sources = "org.codehaus.groovy:groovy:${groovy}:sources"

See: #17169

[melix]

We ruled out this during design: our goal is not to design a DSL within the DSL: TOML has its own syntax and doesn't support string interpolation.

[jnehlmeier]

Hm but that decision might result in people not using the versions table in the toml file at all. Let me explain:

In the typical case dependencies are declared as "group:artifact:version" string with a strict version. You would only use a more complex notation if you absolutely have to.

So when migrating a project to use the new version catalog toml file, you would start with something like

[libraries]
common-lib1 = "com.example.common:lib1:1.0.0"
common-lib2 = "com.example.common:lib2:1.0.0"
common-lib3 = "com.example.common:lib3:1.0.0"
guava = "com.google.guava:guava:30-jre"

since you can relatively easy copy the GAV coordinates from the build.gradle files. Finally you would update all the build.gradle files to use the data declared in the toml file. That is basically the bare minimum you need to do.
Even with this result it is already pretty easy to change versions, even for the repeated version in common-* libraries as you likely have some ordering in the file.

However transforming the above to

[versions]
common = "1.0.0"
guava = "30-jre"

[libraries]
common-lib1 = { module = "com.example.common:lib1", version.ref = "common" }
common-lib2 = { module = "com.example.common:lib2", version.ref = "common" }
common-lib3 = { module = "com.example.common:lib3", version.ref = "common" }
guava = { module = "com.google.guava:guava", version.ref = "guava" }

requires quite some additional work, is less readable and provides only very little benefit compared to the first version. In addition all kinds of websites publishing GAV coordinates usually provide them in the "group:artifact:version" notation for Gradle, e.g. search.maven.org or all kinds of READMEs of Github projects. Basically ready to copy and paste into build files. Some of these projects even use "group:artifact:$version" in its README and tell you to replace $version with the version you want to use.

So as a consequence I would never use the second form of the toml file and always stick to the first version, as it is more readable, easier to copy/paste coordinates into it and does not have a significant overhead in updating a version number compared to the second form.

So supporting something like

[versions]
common = "1.0.0"
guava = "30-jre"

[libraries]
common-lib1 = "com.example.common:lib1:${common}"
common-lib2 = "com.example.common:lib2:${common}"
common-lib3 = "com.example.common:lib3:${common}"
guava = "com.google.guava:guava:${guava}"

would be a fair compromise.

[ljacomet]

To clarify one thing about the classifier part: this will not happen because we want the catalog to be about dependency coordinates only, not about which variant of the dependency to use, since that is usually a contextual decision.

[kelemen]

I recently ported a larger project to use the version catalog. Though it usually works fine, and I'm also not concerned with the syntax being not concise, I still had a problem:

sparkCore = { module = "org.apache.spark:spark-core_2.11", version.ref = "spark" }

And this problem is quite prevalent for scala libraries, and is quite annoying to duplicate the scala version everywhere (which was obviously not the case with having a dependencies.gradle).

Though I understand that interpolating something from the version block can be problematic (since it is not necessarily just a string), can we have some other way to interpolate some parts (even if with an additional block simply for interpolatable variables alongside the current 3)?

[augi]

Hello, just please note that interpolation in module would be great for us, as most of our dependencies are Scala dependencies, so we have to repeat _2.13 everywhere.

[iilyak]

Same problem here. Cannot understand how to have string interpolations going.

[versions]
scala = "2.13"
zio = "2.0.0-M5"

[libraries]
zio = dev.zio:zio_$scala:$zio" # << how to do it?

To clarify one thing about the classifier part: this will not happen because we want the catalog to be about dependency coordinates only, not about which variant of the dependency to use, since that is usually a contextual decision.

Can you advise what features to use to avoid hardcoding 2.13 in multiple places?

[melix]

I understand why you'd like to do this but I'd strongly recommend against adding this. It's a feature which should be part of the language (TOML), not something that Gradle builds on top of it. It also makes it very confusing, because the $scala token would refer to what? the scala version in [versions]? Why not something in an arbitrary other section? What if the token is using scala is actually refering to an object like scala = { require="2.13" }.

For advanced usages like this, using the settings API might actually be preferable.

[augi]

For our use-case, it would be sufficient to support Gradle properties interpolation. So we could have scalaVersion in build.properties, and the interpolation would be performed after loading the TOML file.

[iilyak]

As a workaround I am trying to use the following.

settings.gradle

pluginManagement {
    includeBuild('build-logic')
}

enableFeaturePreview "VERSION_CATALOGS"
enableFeaturePreview "TYPESAFE_PROJECT_ACCESSORS"

ext {
    scalaVersion = "2.13"
}

dependencyResolutionManagement {
    repositories {
        mavenCentral()
    }
    versionCatalogs {
        libs {
            version('zio', '2.0.0-M5')
            alias('zio').to('dev.zio' , "zio_$scalaVersion").versionRef('zio')
        }
    }
}

include ':experiments'

experiments/build.gradle

dependencies {
    implementation(libs.zio)
}

It seem to work.

[melix]

Yes, that's what I mean by using the settings API. Note that you don't need ext here, you should actually avoid ext altogether. A good old def scalaVersion = ... would work.

[tinder-ravitejabonagiri]

Is there a way to reference the version declared in libs.versions.toml file in build.gradle file of a project? Also, how would we define a project dependency in toml file? The documentation on these is pretty sparse.

[wingsofovnia]

Another use case is to have references in a rich versions:

dependencyResolutionManagement {
    versionCatalogs {
        libs {
            version("guava", "30.0-jre")

            alias("guava").to("com.google.guava", "guava").version {
                strictly "[${versions.groovy.get()}, )"
            }
        }
    }
}

this doesn't seem to work. Perhaps there is a better way to do it?

[FyiurAmron]

Is there a way to reference the version declared in libs.versions.toml file in build.gradle file of a project?

libs.versions.toml.get() syntax will provide you with a string directly from TOML. Simple .get() works only for props with no children. To access the parent that has some children - e.g. with both foo and foo-bar in TOML, to get the string for foo - you need to call e.g. libs.foo.asProvider().get() instead (for foo-bar you still need to use libs.foo.bar.get() which works as expected).

[Khazbs]

I find myself in the same boat as @iilyak with their Scala & Zio versions, but with Kotlin & KSP versions:

[versions]
kotlin = "1.9.0"
ksp = "1.9.0-1.0.13" # FIXME: 1.9.0 looks like magic numbers, is actually Kotlin version

Having to repeat the Kotlin version in the KSP version makes the process of upgrading Kotlin more prone to compatibility accidents.

Is there an idiomatic way to declare versions that depend on other versions?

[FyiurAmron]

@Khazbs in TOML? That's probably impossible without hacking Gradle via plugins and/or basically making a TOML generator or preprocessor. Still, you can do that somewhat easily in the buildscript itself, if that's OK for you.

[Khazbs]

@FyiurAmron, thank you for answering my question!

To clarify, I'm not saying TOML syntax needs to somehow get support for that.

In fact, since this discussion is a Gradle feature request, "hacking Gradle" is exactly what I'm suggesting! I see this as a potential feature of how Gradle (or a Gradle plugin) could work with version catalogs.

Support for dependent versions could be implemented by introducing specialized semantics, like:

kotlin = "1.9.0"
ksp = [{ref = "kotlin"}, "-", "1.0.13"]

Given the main purpose of maintaining a version catalog in the first place — consistently reusing version declarations — I think this feature is at least "nice to have".