JavaCompile tasks may be in up-to-date state when modular dependencies have changed leading to odd runtime errors

[dweiss]

Description

Recent builds failed with:

org.apache.lucene.search.uhighlight.TestLengthGoalBreakIterator > testTargetLen FAILED
    java.lang.NoSuchMethodError: 'org.apache.lucene.util.automaton.Automaton org.apache.lucene.util.automaton.Automata.makeStringUnion(java.util.Collection)'
        at __randomizedtesting.SeedInfo.seed([18019D3C4539B4C7:DBF0A37A952BEBF3]:0)
        at org.apache.lucene.search.uhighlight.CharArrayMatcher.fromTerms(CharArrayMatcher.java:42)
        at org.apache.lucene.search.uhighlight.MemoryIndexOffsetStrategy.buildCombinedAutomaton(MemoryIndexOffsetStrategy.java:71)
        at org.apache.lucene.search.uhighlight.MemoryIndexOffsetStrategy.<init>(MemoryIndexOffsetStrategy.java:53)

As Hoss noticed, there is a repro to get the above error:

It's actually pretty trivial to reproduce this problem locally...

git co 11436a848cbcc8302b31ca01e63e57dd35a93b1e
./gradlew clean classes testClasses
git co 361f0a4524d93fc1a793aa72bfcc91c00fa3599e
./gradlew test --tests TestFlattenGraphFilter

I'll try to figure out why this can happen, it shouldn't.

Version and environment details

No response

[dweiss]

It's not incremental compilation - it's something that is related to task inputs and (likely) modular dependencies. Seems like we're missing something that makes the test compilation task to be marked as up-to-date, even though its dependencies (core) have been recompiled.

I couldn't find the cause yet, but I'll keep digging.

[dweiss]

I think the reason for an up-to-date status is because we handle module path ourselves

        // Add modular dependencies and their transitive dependencies to module path.
        task.options.compilerArgumentProviders.add(modularPaths.compilationArguments)

which means gradle can't inspect the contents of those dependencies (as it would with regular classpath entries).

I have to think how to handle this better. It's already hairy enough.

[dweiss]

I've provided a PR that fixes this. It is a corner case of us providing custom javac parameters (modular classpath). I am surprised this took so long to be discovered and I apologize - it's an oversight - we did have a dependsOn clause but it's different than task inputs (dependsOn is more of a task graph scheduling hint, inputs is a task status hint).

I've checked the PR by running Hoss's repro:

git co 11436a848cbcc8302b31ca01e63e57dd35a93b1e
./gradlew clean classes testClasses
git co [head of PR branch]
./gradlew test --tests TestFlattenGraphFilter

everything works and the code is recompiled as it should now.