Nested buildSrc dependencies exposed to subproject's buildSrc classpath

[gabizou]

After starting a migration to using Kotlin-DSL (which is incredible to work with for the most part, minus this issue currently, a real deal breaker), I've found that it's not possible to easily/sanely resolve class path dependencies of a subproject's build script class path to include the subproject's buildSrc. I understand as per the guide, only a root project's buildSrc is available for all build scripts class path resolution, so I understand that sub project's buildSrc plugins are somewhat limited to their own pre-compiled classes and not other classes exposed in the buildSrc. But this is something that I'd like to be able to take advantage of as an equivalent to what we've been able to do with groovy build scripts.

Expected Behavior

Being able to include sub projects which can have buildSrc plugins themselves, resolving the classes compiled in them, constants, plugins, etc. and allow to depend on those buildSrc classes (since they ideally should be pre-compiled before the parenting project's buildSrc class path is resolved, since the subprojects may not have visibility of the parenting project's buildSrc classes).

Current Behavior

Subproject buildSrc classes (not plugins, plugins are resolving fine) are not resolvable, and subprojects end up failing to build.

Context

In the past, our API project contained a few scripts that we'd apply to the API build script, and to our common implementation's supplied build scripts, and this worked transitively up the chain to the overall root projects.

Migrating the API to Kotlin-DSL, I've created a buildSrc plugin that would use constants supplied by a static object in the same buildSrc. This results in a nice efficient build.gradle.kts that applies the desired buildSrc plugin, and the rest is pretty much well migrated as far as gradle logic goes.

When porting the common implementation (first nesting of projects, so rootProject is common, subproject includes API), the API build script will no longer compile complaining that the various object classes (Plugins, Repos, etc.) I'd like to be able to recycle the object usage and dependencies of those buildSrc classes (since they're compiled already).

One way I've been able to "workaround" this limitation is by creating a copy task for each parent project's buildSrc that will perform a copy of the subproject's buildSrc src files into the parent's buildSrc src directories, and make the compileKotlin task depend on that copy task (yes, I know, it's forcing those files to become part of the rootProject's buildSrc class path, and therefor the build script class path of all projects), but of course, that then causes other usability issues, such as modifying those subproject plugins requires copying code between parent and subproject, adding the duplicated files to the .gitignore of the parent projects, etc, but it would allow for the sub projects to properly function in that context.

Steps to Reproduce (for bugs)

Essentially put, a subproject with a buildSrc that declares a plugin that refers to objects with `const Val's for compiler time constants, and the root project having a buildSrc as well that will make new classes/plugins, and the root project including said sub project:

Should be able to clone the SpongeForge branch of attempting to migrate (note that git submodule update --init --recursive is required) and attempt to import the build.gradle.kts into IDE. A few of the errors should look like so:

https://gist.github.com/gabizou/d48951439ad87783591b82aed0d4e6d4

Your Environment

• Build scan URL: N/A build does not compile
• gradle --version 5.2.1
• Version of IntelliJ or Android Studio: 2018.3.2 Build #IU-183.4886.37
• Version of the Kotlin Plugin used in IntelliJ or Android Studio: 1.3.21-release-IJ2018.3-1

[eskatos]

Hi @gabizou,

There is a single buildSrc per build and its content is shared by all projects build scripts. If I'm understanding correctly, you have two builds (SpongeForge and SpongeCommon) and you include projects from the latter into the former. This won't make SpongeCommon/buildSrc available for the outer build and can cause other issues if the two builds build-logic is different. The standard way to let several builds (multi-project or not) collaborate is using Gradle Composite Builds. Using composite builds you can also have a shared build that provides Gradle plugins and types to several other builds. please read https://docs.gradle.org/current/userguide/composite_builds.html

Note that this issue tracker isn't suited for support questions, please use the Gradle forums in the future https://discuss.gradle.org/

[gabizou]

There is a single buildSrc per build and its content is shared by all projects build scripts.

I understand that, that's one of the limitations that I'd like to request as a feature change for sub projects to be able to still evaluate their buildSrc for their specific build scripts.

This won't make SpongeCommon/buildSrc available for the outer build and can cause other issues if the two builds build-logic is different.

Not asking for that. I'm asking for the child buildSrc (in this case, SpongeAPI/buildSrc) to be avaiable to SpongeAPI which is a child of SpongeCommon. But, because of the limitation that buildSrc is only available from the root project, it's not possible for :SpongeCommon:SpongeAPI to resolve the classes within SpongeAPI/buildSrc (since it references several constants in applying plugins via a constants class located in the buildSrc)

The standard way to let several builds (multi-project or not) collaborate is using Gradle Composite Builds. Using composite builds you can also have a shared build that provides Gradle plugins and types to several other builds. please read https://docs.gradle.org/current/userguide/composite_builds.html

I was hoping to restructure the projects overall since the intention was to allow SpongeAPI to remain isolated within itself, but potentially exposing the ability to use the classes within SpongeAPI, and still declare various constants in the parent buildSrc without attempting to depend on SpongeAPI's buildSrc classes.

Note that this issue tracker isn't suited for support questions, please use the Gradle forums in the future https://discuss.gradle.org/

I understand support is better offered on the forums, but since this is more of a feature request, I figured this would be better suited as such as a GitHub issue.

[eskatos]

I see, sorry for the misunderstanding. So, in a few words, your feature request could be summarized as "support a buildSrc per project in a given build", correct?

What confuses me is that in your example, SpongeForge, SpongeCommon and SpongeApi are all distinct builds. I also see that you include the root project of SpongeCommon in the SpongeForge build and the root project of SpongeApi in the SpongeCommon build. Composite builds are better suited for that kind of setup.

[gabizou]

So, in a few words, your feature request could be summarized as "support a buildSrc per project in a given build", correct?

Yes, per project in the sense of nested subprojects having their own buildSrc and plugins, available to those subproject's build scripts classpath (since that's what's causing the failure in to evaluate SpongeAPI, regardless whether the project's setup migrates to a composite build or not).

What confuses me is that in your example, SpongeForge, SpongeCommon and SpongeApi are all distinct builds. I also see that you include the root project of SpongeCommon in the SpongeForge build and the root project of SpongeApi in the SpongeCommon build.

They're partially distinct, SpongeCommon is a shared subproject between two core implementation projects: SpongeForge and SpongeVanilla, and their reasoning for the complicated gradle scripts is that they both *do depend on some scripts declared in SpongeCommon since common sets up some dependencies and "default" configurations that otherwise would be duplicated across SpongeForge and SpongeCommon. By itself, SpongeCommon does nothing but does succeed in building itself and no explicit root project dependencies, but in combination of SpongeForge/SpongeVanilla having the nested subproject of SpongeCommon, they both extend things from SpongeCommon (providing the remaining ~5% of the implementation of SpongeAPI).

Composite builds are better suited for that kind of setup.

Since the original gradle setup was done way back in the days of yore with Gradle 2.x, I don't know enough about the migration of composite builds to be able to say that composite builds are better suited. One of the key factors involved in the project is that SpongeCommon is not just a gradle subproject of SpongeVanilla/SpongeForge, but also a git submodule. Likewise, SpongeCommon has the subproject of SpongeAPI that is also a git submodule. I don't know what the composite build setup would look like at that point.

[gabizou]

So, would this issue be reopen able as a feature request for "support a buildSrc per project in a given build"?