Support providing Jars with alternative Java compiler implementations

[jjohannes]

Context

Proposal to provide a solution for: #15942

The proposal is to allow the definition of a customCompilerClasspath of the JavaCompile task. If the customCompilerClasspath contains an implementation of javax.tools.JavaCompiler that can be discovered through Java's ServiceLoader it is used in place of the default JDK compiler (JavacTool) – "on top" of the selected Java toolchain.

This is how you would configure it to use the Eclipse Compiler (ECJ):

val ecj = configurations.create("ecj")
dependencies {
    "ecj"("org.eclipse.jdt:ecj:3.33.0")
}

tasks.withType(JavaCompile::class).configureEach {
    customCompilerClasspath.from(ecj)
    options.headerOutputDirectory.convention(null) // see #12904
}

Contributor Checklist

• Review Contribution Guidelines
• Make sure that all commits are signed off to indicate that you agree to the terms of Developer Certificate of Origin.
• Make sure all contributed code can be distributed under the terms of the Apache License 2.0, e.g. the code was written by yourself or the original code is licensed under a license compatible to Apache License 2.0.
• Check "Allow edit from maintainers" option in pull request so that additional changes can be pushed by Gradle team
• Provide integration tests (under <subproject>/src/integTest) to verify changes from a user perspective
• Provide unit tests (under <subproject>/src/test) to verify logic
• Update User Guide, DSL Reference, and Javadoc for public-facing changes
• Ensure that tests pass sanity check: ./gradlew sanityCheck
• Ensure that tests pass locally: ./gradlew <changed-subproject>:quickTest

Gradle Core Team Checklist

• Verify design and implementation
• Verify test coverage and CI build status
• Verify documentation

[eskatos]

@bot-gradle test RFN

[bot-gradle]

I've triggered the following builds for you:

• ReadyForNightly build

[eskatos]

@bot-gradle test RFN

[bot-gradle]

I've triggered the following builds for you:

• ReadyForNightly build

[jjohannes]

Gradle 8.4 + this PR: https://github.com/jjohannes/gradle/tree/release-8.4-ecj-patch/gradle-bin

[gitstream-cm]

Change Summary

This PR is 91.95% new code.

Platform	Added Lines	% of Total Line Changes	Deleted Lines	% of Total Line Changes	Files Changed	% of Total Files Changed
bt_ge_build_cache	0	0%	0	0%	0	0%
build_infrastructure	0	0%	0	0%	0	0%
core_configuration	0	0%	0	0%	0	0%
core_execution	0	0%	0	0%	0	0%
core_runtime	0	0%	0	0%	0	0%
documentation	0	0%	0	0%	0	0%
extensibility	0	0%	0	0%	0	0%
gradle_enterprise	0	0%	0	0%	0	0%
ide	0	0%	0	0%	0	0%
jvm	150	50.34%	23	7.72%	9	81.82%
kotlin_dsl	0	0%	0	0%	0	0%
release_coordination	0	0%	0	0%	0	0%
software	0	0%	0	0%	0	0%

[AceTheFace]

Is there any ETA when this will be merged? Will this feature make it into 8.6 release?

[tresat]

@AceTheFace - It will not be included in 8.6. Some conceptual questions arose after discussing this that need further consideration. It is marked with the team-triage label and will be considered again soon.

[big-guy]

Sorry, we don't want to model things this way. We know that supporting other Java compilers would be a good thing to have, but we don't want to expose this as another configuration with arbitrary jars that we execute.

How exactly this needs to be done is unclear, but we think it would be more closely tied to a Java toolchain that provides a different Java compiler.

We'll leave the original issue open, but close this PR.

[AceTheFace]

:-( Problem is, that this functionality was available with Gradle 6.x and has been dropped with 7 - which now forces us to use unstable internal API, which might change in every release. So any official way to support this would highly be appreciated.

[jjohannes]

forces us to use unstable internal API

Our problem is that there are no APIs (internal or public) at the moment to get the custom compiler used by the compile worker JVMs. And we need that to get acceptable compile performance in a (very) large project.

I think no matter where this topic goes next, this PR contains a valuable technical solution to get the additional compiler Jar into the worker. So no matter how a public API surface may look one day, I think something like that has to be added in any case.

@big-guy what would be the concrete next steps? There are folks from the community (@TheMrMilchmann, me and others) who are ready to invest time and resources to make this happen. What can we do? As I wrote already, an initial internal only API that allows us to get compiler Jars into the worker, would be very helpful already. Then we have a technical foundation to built upon (in custom plugins/builds). But without this addition, we cannot solve this without patching Gradle.