Add support for Gradle Version Catalogs (Gradle 7.0)

[ReginFell]

Gradle 7.0 was released recently and it introduces a new Central Declaration System as well as a new supported format .toml. gradle/gradle#15352

Would be nice if Dependabot supported this.

[JavierSegoviaCordoba]

I would like that dependabot search for ALL toml files in the project (regardless of the directory or subdirectory the file is in).

[madhead]

As an expansion for this…

I was using a trick with a Versions.kt file in buildSrc hosting all the versions for my project and using it in the build. Take a look at the example in madhead/ImgMacroBot:

buildSrc/src/main/kotlin/Versions.kt

object Versions {
    const val JVM = "11"

    object Plugins {
        const val KOTLIN = "1.4.21"
        const val DETEKT = "1.15.0"
        const val LIQUIBASE = "2.0.4"
    }

    object Dependencies {
        const val KTOR = "1.5.1"
        const val KOIN = "2.2.2"
        const val KOTLINX_SERIALIZATION = "1.0.1"
        const val TGBOTAPI = "0.30.0"
        const val SKIJA = "0.89.18"
        const val MICROMETER = "1.6.4"
        const val LOG4J = "2.14.0"
        const val JACKSON = "2.12.1"
        const val JUNIT = "5.7.1"
        const val MOCKK = "1.10.6"
        const val LIQUIBASE = "4.3.1"
        const val SNAKEYAML = "1.28"
        const val OJDBC11 = "21.1.0.0"
    }
}

And then in any build.gradle.kts:

dependencies {
    implementation(platform("org.apache.logging.log4j:log4j-bom:${Versions.Dependencies.LOG4J}"))
    implementation(platform("io.micrometer:micrometer-bom:${Versions.Dependencies.MICROMETER}"))
 
    api("io.micrometer:micrometer-core")
    api("dev.inmo:tgbotapi:${Versions.Dependencies.TGBOTAPI}")
    api("org.jetbrains.skija:skija-linux:${Versions.Dependencies.SKIJA}")
    …
}

Dependabot seems not to handle that correctly, check the repo's insights: the dependencies list is empty. I don't know if Dependabot actually executes Gradle to parse the dependencies, but seems like it's abilities in parsing them are limited.

Probably, if it executed full-featured Gradle it could handle any scenarios like mine or like the new central declaration system.

[huehnerlady]

@madhead that sounds like a completely different issue. Here we ask for the support of the new gradle feature with toml files and not for the buildSrc workaround. Maybe it makes sense to open a separate issue for that?

[JavierSegoviaCordoba]

Another thing, a Version Catalog can be published, and you can add them to whatever project you want via settings.gradle files:

dependencyResolutionManagement {
    repositories {
        mavenCentral()
        google()
        gradlePluginPortal()
    }

    versionCatalogs {
        create("libs") { from("com.example.catalogs:some-catalog:1.0.0") }
    }
}

So should be great dependabot can search for updates for them.

[madhead]

Take it easier!

My point is not about that particular buildSrc example, it's just an example. I'm trying to add a comment that if only Dependabot actually evaluate the build script and extract the versions from the model, it could support all the scenarios, including this new TOML feature and aforementioned versionCatalogs.

P.S. I don't know how does Dependabot work currently, but it seems like it uses a bunch of regexps to parse the scripts. I understand it could be the only option - evaluating a build scripts needs a Java, which might be unacceptable. It's also impossible to make a PR based on the evaluated model, but it's just an idea.

[tbroyer]

@madhead There's #1164 already. It wouldn't tell Dependabot which file to update though, for automatic pull requests.

[madhead]

@tbroyer, yep, looks exactly what I was talking about! And, yeah, updates are almost impossible with this approach.

[JavierSegoviaCordoba]

Anyway, having dependencies in buildSrc is not only a bad practice, there are unlimited ways to organize them there, so it is really difficult to cover all use cases.

Knowing version catalogs will be the standard way for dependency management, I think there is no sense in searching a way to get it working in buildSrc.

[DanySK]

There's a bit of confusion. There are two different issues at hand:

1. supporting the central declaration system;
2. supporting Gradle fully.

Although 2. (see #1164 as @tbroyer suggests) would likely cover both, adding 1. first would be useful in many cases and easier.

[ReginFell]

Yeah, parsing toml should be relatively simple task since dependabot already has this for Python

[huehnerlady]

I would say the ultimate goal is to support gradle fully. if this could be done in the near future, then no need for the central dependencies. BUT as it does not seem to be near I would say it might be a very good way to migrate towards more support of gradle-kotlin-dsl :)

[huehnerlady]

Is there any update on this? Would be great to have this feature and I thought someone mentioned it should not be tooo much work?

[jurre]

It's not currently on our roadmap, I'm not sure who said that, but it seems like a significant amount of work from a quick glance

[DanySK]

@jurre, the comparison was being made with the proposal for going through the Gradle Tooling API #1164; my feeling is that the amount of work required to support the central declaration in a TOML file would be way smaller...

[jurre]

@jurre, the comparison was being made with the proposal for going through the Gradle Tooling API #1164; my feeling is that the amount of work required to support the central declaration in a TOML file would be way smaller...

Yes, I agree that it seems smaller, but still a significant amount of work that we currently don't have planned, so I can't give a timeline

[DanySK]

I see; thanks for the clarification.

[huehnerlady]

@jurre thanks for the clarification. is there anything we can do to get it on the roadmap asap? 😉

[be1ski]

@jurre we are very sad to hear that 🥲 maybe you have at least an approximate date for the appearance of this feature?

[StefanOltmann]

Sad. Dependabot does not work with Gradle in most use cases.

[jurre]

@jurre we are very sad to hear that 🥲 maybe you have at least an approximate date for the appearance of this feature?

@asciimike would be the best person to help answer that, but I don't think there is much we can commit to at this point

[DanySK]

@StefanOltmann @be1ski one way around the current level of support is to manage dependencies via refreshVersions and automatic upgrades via UpGradle. I'm currently achieving dependabot-like functionality this way, while I wait for support to Gradle to be complete.

[JavierSegoviaCordoba]

I am being forced to generate the TOML files from code inside build.gradle.kts files to get dependabot working.

[Nambers]

Is there any update on this? Would be great to have this kind of feature like supporting the toml or buildSrc in gradle.
But my dependabot look like dont get my toml file.

[asciimike]

No updates on timeline, as we don't have the gradle expertise or time at present to properly prioritize this (or #1164, which would be even better). If folks are interested in working on this, let us know.

[dmedinag]

I have to add myself to the interested parties in seeing this happen. TOML catalogs introduce a massive leap forward in dependency management for frameworks developed with gradle and since we're using my team and I have had to say goodbye to Dependabot :( We'd love to have it back in our project.

[warting]

Today is the day my dreams came true! I noticed a few new PR from dependabot in my projects today bumping various kind of libraries in my toml files! ❤️

[planetf1]

@warting so dependabot now appears to support toml files now? if so yay indeed!

[deivid-rodriguez]

As you observed, we added Gradle Version Catalog support 🎉.

Thanks for your patience! Please report any issues or feature requests over the initial MVP as separate issues :)

[svenjacobs]

The documentation doesn't seem to be updated yet. Do I need to specify directory: "/gradle" in dependabot.yml or how does Dependabot distinguish between build.gradle, build.gradle.kts and gradle/libs.version.toml?

[huehnerlady]

I am also looking for a documentation what the MVP is actually supporting in regards to toml files.
As @svenjacobs says are there special requirements that needs to be met in order to get it to work?

And are only versions supported, or also plugins and libraries for example, documented here?

[alsutton]

For those who want some docs, we've just deployed dependabot on our version catalogue for Gradle and we're getting update PRs. The only thing we needed to do was create /.github/dependabot.yml in our repo with the following contents;

version: 2
updates:
  - package-ecosystem: "gradle"
    directory: "/"
    schedule:
      interval: "daily"

which gave us auto-created PRs such as this one;

[deivid-rodriguez]

Hei, sorry for the lack of documentation, we have some updates there but we haven't been able to deploy them yet.

This initial version supports updating the "standard" version catalog file gradle/libs.versions.toml, and it supports updating versions, plugins and libraries. Other setups configuring custom catalogs at settings.gradle are not yet supported.

[huehnerlady]

@deivid-rodriguez thanks for the update. so there will be a documentation with examples? We are using "versions" and nothing more and wonder if that is not yet supported. Could you maybe verify?

our libs.versions.toml:

[versions]
[...]
springBoot = "2.7.9"

and we are implementing it in the dependencies section of our built.gradle.kts like this:

dependencies {

[...]
  implementation("org.springframework.boot:spring-boot-starter-web:${libs.versions.springBoot.get()}")
  implementation("org.springframework.boot:spring-boot-starter-actuator:${libs.versions.springBoot.get()}")
[...]
}

[deivid-rodriguez]

No problem!

No, I don't think that's supported either, sorry. We support referencing versions as version.ref references in plugins or libraries inside the lib.versions.toml file. But I don't think accessing versions like that is supported.

The underlying issue is that we don't run Gradle/Java under the hood, so anything that requires evaluating Java code is not really supported at the moment.

[deivid-rodriguez]

Docs have also been updated now to mention the scope of the current support: https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file (footnote number 3 in package-ecosystem table).

[tjuszczyk]

This initial version supports updating the "standard" version catalog file gradle/libs.versions.toml, and it supports updating versions, plugins and libraries. Other setups configuring custom catalogs at settings.gradle are not yet supported.

@deivid-rodriguez Is there any. scheduled plan to support additional gradle/*.toml files as well as a default one?

[deivid-rodriguez]

Not right now. Our long term plan is to move on to using the Gradle Tooling API (#1164), so that we don't have to parse Java/Kotlin/Groovy with Ruby code, but that work is non trivial and hasn't been prioritized yet.

[tjuszczyk]

With gradle/libs.versions.toml supported, wouldn't supporting list of gradle/*.toml be simple addition?

[jeffwidman]

As Deivid said, for the core team we aren't planning to do a lot of further feature enhancements to the ruby/regex code because the better long term play is #1164.

However, this is open source, and we certainly welcome PR's if you want to take a stab at it! If you do start on it and run into questions let us know... we're always happy to answer questions from contributors... typically easiest to start with a draft PR and add comments where you get stuck.

[deivid-rodriguez]

It shouldn't be too hard, but we still need to parse the configuration in settings.gradle, which is written if Groovy. Given the dynamic nature of it, there's probably a few different ways to configure it, so it might get tricky, but we'd gladly review a PR implementing it. We already parse settings.gradle from Ruby for other things.

[Kernald]

It shouldn't be too hard, but we still need to parse the configuration in settings.gradle, which is written if Groovy. Given the dynamic nature of it, there's probably a few different ways to configure it, so it might get tricky, but we'd gladly review a PR implementing it. We already parse settings.gradle from Ruby for other things.

Note that this file might be written in Kotlin as well - our repo has a settings.gradle.kts containing a version catalog. I guess the next step for us is to migrate that one to a toml until #1164.