Kapt incremental processing not working

[theHacker]

Task List

• Steps to reproduce provided
• Stacktrace (if present) provided -> N/A
• Example that reproduces the problem uploaded to Github -> Micronaut Demo is sufficient to reproduce
• Full description of the issue provided (see below)

Steps to Reproduce

1. Download Demo according to https://github.com/micronaut-projects/micronaut-gradle-plugin/blob/master/README.md
2. curl https://launch.micronaut.io/demo.zip?lang=kotlin -o demo.zip && unzip demo.zip -d demo
3. Add or edit a bean in the test file and run ./gradlew test.

Screencast: https://owncloud.senacor.com/index.php/s/cQr6wlFtArBsrGK

Expected Behaviour

The test will run. (Because the test bean is in the ApplicationContext)

Actual Behaviour

The test will not run. (The test bean is missing in the ApplicationContext)

Environment Information

• Operating System: MacOS 10.15.7
• Micronaut Version: 2.2.1
• JDK Version: Kotlin 1.4.21

---

Problem

We have multiple projects running with Micronaut. Right now we are in the process of updating them from Micronaut 1.3.7 to Micronaut 2.x.
We have working solutions with Micronaut 2.2.1 (verified myself) and 2.2.3 (no complains from the team).

However one project has heavy problems after the update to 2.2.1.
When changing test code, the annotation processor completely ignores every other bean, breaking everything.

To get everything working again ./gradlew clean is necessary.

Detailed analysis

• When starting Micronaut, DefaultBeanContext.start() gets call.
• readAllBeanDefinitionClasses() tries to read all bean definitions.
• resolveBeanDefinitionReferences() uses the SoftServiceLoader to load a list of BeanDefinition class names from META-INF/services/io.micronaut.inject.BeanDefinitionReference.

This file gets generated when invoking ./gradlew kaptTestKotlin.
If a bean definition is missing at this point, it does not get loaded!

When changing a file, the annotation processor only writes the new/changed beans into the definition file.

Since this issue very strongly sounds like a problem with incremental annotation processing, I also tried disabling it.
However, for the issue, it does not matter, whether incremental annotation processing is enabled or not.

Impact / additional information

In our project, we have nearly 500 beans (test helper classes, configurations and the tests themselves) in the test scope.

In combination with the test framework (we use Kotest), we get misleading error messages:

• JDBC related exceptions, because the test configuration class does not get executed and the DataSource will point to the wrong database.

• exceptions from Kotest unable to instantiate the test class, because there is no no-arg constructor.
  I debugged into this case also. MicronautKotestExtension.instantiate() gets called, creates the ApplicationContext, but cannot find the test bean and then returns null.
  This will generate a very misleading error message. The cause, that the test bean could not be found, is not printed out.

  Maybe you want to add a fail-fast in https://github.com/micronaut-projects/micronaut-test/blob/v2.2.1/test-kotest/src/main/kotlin/io/micronaut/test/extensions/kotest/MicronautKotestExtension.kt#L96, when createBean is set, but context.bean is null anyway?

In the screencast you see the issue broken down to gradlew commands.
In reality, we use IntelliJ and the Kotest IntelliJ plugin to launch the tests, while working.

• IntelliJ will invoke ./gradlew classes testClasses when doing a build
• Kotest plugin will invoke ./gradlew test --tests "..." when executing particular test methods/classes.

Related issues

• Beans are sometimes not injected in junit tests #4730
  complains about bean injection failures. If the beans are not in the ApplicationContext, injection will fail.

• All bean definitions are recreated during re-compilation #2922
  This looks the issue leading to the problem. I researched the linked Micronaut and Gradle issues.
  For me it verifies the time frame, where the issue was introduced (switching from Micronaut 1.3.7 to 2.x).
  In 1.3.7 https://github.com/micronaut-projects/micronaut-core/blob/2.3.x/inject-java/src/main/java/io/micronaut/annotation/processing/ServiceDescriptionProcessor.java
  did not exist yet.

  I am not so deep into annotation processing and Gradle, to understand the discussion in this ticket about
  aggregating vs isolating processor. The comment

  I found a way to avoid reprocessing the AST by using a hacky solution of setting a static property from the originall processor to another aggregating processor

leads me to think, you have broken something, leading to this bug in certain setups.

---

Right now, I don't know, where the error lies. Is it Micronaut, Gradle and/or specifics to project configuration?
Since I can reproduce with the Micronaut Demo also, I think, a bug report here isn't wrong :-)

Please feel free to ask, if you need additional information.

[graemerocher]

I'm not able to reproduce this following the steps in the video using Micronaut 2.3.1. Note that the comment you quoted

I found a way to avoid reprocessing the AST by using a hacky solution of setting a static property from the originall processor to another aggregating processor

That was an experiment and that change never went into Micronaut, the bugs mostly existed in Gradle and Kotlin.

Make sure when you are upgrading you also upgrade to the latest versions of Gradle and Kotlin both of which had numerous annotation processing related bugs that could be impacting your experience.

[graemerocher]

@theHacker can you try Kotlin 1.4.30 with Gradle 6.8 and feedback if you still see issues?

[theHacker]

Thanks for answering so quickly.
In our project it seems to help to use Kotlin version 1.4.30 :-) (previously we used 1.4.21)

I tried also quickly with your Demo project (this uses Kotlin 1.4.10).
However it does not help to simply change the Kotlin version.

Changing the Kotlin version leads to this error message

> Task :kaptTestKotlin FAILED
error: cannot access AnyBean
  class file for demo.AnyBean not found
FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':kaptTestKotlin'.
> A failure occurred while executing org.jetbrains.kotlin.gradle.internal.KaptExecution
   > java.lang.reflect.InvocationTargetException (no error message)

---

I created a shell script, with which you can easily reproduce the issue:

#!/bin/zsh

# Download
echo "Download Micronaut demo project"
echo "--------------------------------"
echo ""

curl "https://launch.micronaut.io/demo.zip?lang=kotlin" -o demo.zip
unzip demo.zip -d demo
rm demo.zip
cd demo

# Output versions
echo ""
echo ""
echo "Outputting versions of Kotlin and Gradle:"
echo "  EXPECTED:"
echo "    Kotlin 1.4.10 + Gradle 6.8"
echo "  ACTUAL:"
cat gradle.properties | grep kotlin
cat gradle/wrapper/gradle-wrapper.properties | grep Url
echo ""
echo ""

# Build once
echo "Building once"
echo "---------------"
./gradlew kaptTestKotlin
echo ""

echo "After initial build"
echo "--------------------"
echo ""
echo "Test services file META-INF/services/io.micronaut.inject.BeanDefinitionReference:"
echo "  EXPECTED: all beans = 1 bean: \$DemoTestDefinitionClass"
echo "  ACTUAL:"
cat build/tmp/kapt3/classes/test/META-INF/services/io.micronaut.inject.BeanDefinitionReference
echo ""

# Add a bean
echo "We now add a bean and build again"
echo "----------------------------------"

echo "@io.micronaut.context.annotation.Factory class AnyBean" >> src/test/kotlin/demo/DemoTest.kt
./gradlew kaptTestKotlin
echo ""

echo "Now see the bug"
echo "----------------"
echo ""
echo "Test services file META-INF/services/io.micronaut.inject.BeanDefinitionReference:"
echo "  EXPECTED: all beans = 2 beans: \$DemoTestDefinitionClass and new \$AnyBeanDefinitionClass"
echo "  ACTUAL:"
cat build/tmp/kapt3/classes/test/META-INF/services/io.micronaut.inject.BeanDefinitionReference
echo ""

echo ""
echo "The actual output however does not contain \$DemoTestDefinitionClass anymore!!!"
echo ""

[jameskleeh]

I was able to reproduce the issue with Micronaut 2.3.2 and Kotlin 1.4.30

[jameskleeh]

What is interesting is that I can reproduce it with micronaut.processing.incremental set to false

[jameskleeh]

Adding kapt.incremental.apt=false to the gradle.properties fixed the issue. This seems to be a bug in KAPT?

[jameskleeh]

I created this issue for you https://youtrack.jetbrains.com/issue/KT-44989

[jameskleeh]

@theHacker The kotlinVersion needs changed to 1.4.30 and the build plugin versions need changed as well. In that case it works correctly.