Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incremental compilation of java modules is broken with Gradle 7.6 #23067

Closed
breskeby opened this issue Dec 8, 2022 · 3 comments · Fixed by #23119
Closed

Incremental compilation of java modules is broken with Gradle 7.6 #23067

breskeby opened this issue Dec 8, 2022 · 3 comments · Fixed by #23119
Assignees
Labels
a:regression This used to work affects-version:7.6 in:java-plugins java-library, java, java-base, java-platform, java-test-fixtures in:modular-java
Milestone

Comments

@breskeby
Copy link
Contributor

breskeby commented Dec 8, 2022

After updating our elasticsearch build to Gradle 7.6 we are seeing weird compilation issues when using incremental compilation.
The only two workarounds are

  1. disable incremental compilation
  2. run clean task solves the problem

Expected Behavior

It should not be required to run the clean task for correct java module compilation

Current Behavior

When changing the sources slightly we see java module related error messages during compileJava like

package org.elasticsearch.core is not visible

Context

We seen this when updating the elasticsearch build to from 7.5.1 to Gradle 7.6

Steps to Reproduce

  • git clone https://github.com/breskeby/elasticsearch.git
  • git checkout repro-incremental-compile-issue
  • git checkout b0a82469130813d4f6894003f6bfab528d37e68a
  • ./gradlew :server:classes
  • git checkout 04c6269a2f74afe61774a9711315484d70fb63eb
  • ./gradlew :server:classes

Your Environment

Build scan URL: https://gradle.com/s/amjv2v4a4qmym

@ljacomet ljacomet added in:modular-java a:regression This used to work in:java-plugins java-library, java, java-base, java-platform, java-test-fixtures and removed a:bug to-triage labels Dec 9, 2022
@ljacomet ljacomet added this to the 7.6.1 milestone Dec 9, 2022
@asodja asodja self-assigned this Dec 9, 2022
@breskeby
Copy link
Contributor Author

Reproducing the expected behaviour with Gradle 7.5.1 can be done as follows:

git clone https://github.com/breskeby/elasticsearch.git
git checkout repro-incremental-compile-issue-751
git checkout 5adb055cc110d945e9221cbf452f19fffa07a16d
./gradlew clean :server:classes
git checkout f9f9f5b22e957760510c81116790c26575120095
./gradlew :server:classes```

@asodja
Copy link
Member

asodja commented Dec 14, 2022

Moved to 8.0RC1, since we have to finish 8 in next weeks. But we will backport the fix to 7.6.1.

@asodja
Copy link
Member

asodja commented Dec 20, 2022

Note: If the fix still causes some issues for you, you can use in the future Gradle 8 (and in the future Gradle 7.6.1):

tasks.withType(JavaCompile).configureEach {
  options.incrementalAfterFailure = false
}

that should make incremental compilation behavior the same as with Gradle 7.5.

bot-gradle added a commit that referenced this issue Jan 20, 2023
…lation after a failure

Changing compile output folder to a temporary folder for incremental compilation after failure causes some issues with modules (#23067) and with annotation processors (#23066).

But to restore outputs we can just do (not necessarly in the given order):
1. restore stale classes and resources that were deleted
2. remove all newly generated classes and resources
3. restore any overwritten classes (this happens in practice rarely, and only on the incremental compilation, e.g. if we have `B.java` with `class B` and we don't change that file and we create `B1.java` with `class B`).

We already do 1) in the current implementation, for 2) we can use Compiler API data we generate for incremental annotation processors and for "source to class name" mapping.

For step 3) I implemented a [CompilationClassBackupService.java](https://github.com/gradle/gradle/pull/23226/files#diff-21d9cec673f39ed2a74d60de1b8862dd6fa18e6eea1769209e8ec9fb47b53919) that is injected in the Compiler API and backups any `.class` file during compilation.

Backup is done just for "normal compilation". Annotation processors are very limited so this is not needed. They are limited in a way that If two types generate one class/resource, the annotation processor can only be aggregating, for which we always regenerate class/resources and reprocess all types with aggregating annotation.

Since we track types also for Groovy compilation, the same method for restoring outputs works well also with Groovy and  Groovy/Java joint compilation.

---

Todo:
- [x] When #23119 gets merged to master, revert changes done in [GradleStandardJavaFileManager](https://github.com/gradle/gradle/blob/5d8adb4002742924f7327db670eb8f2fffff9414/subprojects/language-java/src/main/java/org/gradle/api/internal/tasks/compile/reflect/GradleStandardJavaFileManager.java#L100-L111).

Co-authored-by: Anže Sodja <asodja@gradle.com>
ijuma pushed a commit to apache/kafka that referenced this issue Feb 24, 2023
Details:
 * gradle upgrade: 7.6 -> 8.0.1
 * spotbugs plugin upgrade: 5.0.9 -> 5.0.13
 * tweaked the mechanics for `-release`/`-source`/`-target` to workaround idiosyncrasies in Gradle 8.0.1 and newer Scala 2.13 versions.
 * streams-scala `test` task no longer triggers the `spotless` task since a newer version is required for Gradle 8 support, but the newer version requires Java 11.
Note: relates to #5479

Gradle upgrade highlights:
* "Scala Incremental Compilation for Multi-Module projects broken in 7.x": gradle/gradle#20101
* "Incremental compilation of java modules is broken with Gradle 7.6": gradle/gradle#23067

Full release notes: https://docs.gradle.org/8.0/release-notes.html

Reviewers: Ismael Juma <ismael@juma.me.uk>
elasticsearchmachine pushed a commit to elastic/elasticsearch that referenced this issue Feb 24, 2023
This updates the gradle wrapper to 7.6.1. This patch release contains a
fix for  incremental compilation of java modules we have raised against
gradle 7.6

see gradle/gradle#23067
breskeby added a commit to elastic/elasticsearch that referenced this issue Feb 25, 2023
This updates the gradle wrapper to 7.6.1. This patch release contains a
fix for  incremental compilation of java modules we have raised against
gradle 7.6

see gradle/gradle#23067
breskeby added a commit to elastic/elasticsearch that referenced this issue Feb 27, 2023
This updates the gradle wrapper to 7.6.1. This patch release contains a
fix for  incremental compilation of java modules we have raised against
gradle 7.6

see gradle/gradle#23067
saarikabhasi pushed a commit to saarikabhasi/elasticsearch that referenced this issue Apr 10, 2023
…ic#94122)

This updates the gradle wrapper to 7.6.1. This patch release contains a
fix for  incremental compilation of java modules we have raised against
gradle 7.6

see gradle/gradle#23067
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:regression This used to work affects-version:7.6 in:java-plugins java-library, java, java-base, java-platform, java-test-fixtures in:modular-java
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants