Dependency Locking is not resolving all dependencies on testCompileClasspath

[shand-j]

Expected Behavior

./gradlew resolveAndLockAll --write-locks should resolve all dependencies across all configurations with resolution strategy of dependencyLocking { lockAllConfigurations()} and write them to the lockfile.

Current Behavior

Certain transitive dependencies are not being resolved on the testCompileClasspath configuration, as a result the lock file is written and at the time of task testCompileJava the lock state validation is failing as the dependency is resolved but it is not included in the lock file.

Context

Java 17 Spring Project

One of my test dependencies 'au.com.dius.pact.provider:junit5spring:4.2.16', has the following transitives:

'org.jetbrains:annotations:13.0'
'org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.6.21'
'org.jetbrains.kotlin:kotlin-stdlib:1.6.21'
'org.jetbrains.kotlin:kotlin-stdlib-common:1.6.21'
'org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.6.21'

My dependency locking is targeting all configurations, after writing locks the above transitives exist in the gradle.lockfile as follows:

org.jetbrains:annotations:15.0=testRuntimeClasspath
org.jetbrains.kotlin:kotlin-reflect:1.6.21=testRuntimeClasspath
org.jetbrains.kotlin:kotlin-stdlib-common:1.6.21=testRuntimeClasspath
org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.6.21=testRuntimeClasspath
org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.6.21=testRuntimeClasspath

When I run a build with tests I get:

./gradlew clean build
> Task :compileTestJava FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':compileTestJava'.
> Could not resolve all files for configuration ':testCompileClasspath'.
 > Resolved 'org.jetbrains:annotations:13.0' which is not part of the dependency lock state
   > Resolved 'org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.6.21' which is not part of the dependency lock state
   > Resolved 'org.jetbrains.kotlin:kotlin-stdlib:1.6.21' which is not part of the dependency lock state
   > Resolved 'org.jetbrains.kotlin:kotlin-stdlib-common:1.6.21' which is not part of the dependency lock state
   > Resolved 'org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.6.21' which is not part of the dependency lock state

I want/ expected the transitives to have been resolved as part of the resolution process on the testCompileClasspath configuration (others have been from what I can see). I would then have expected the gradle.lockfile to include

org.jetbrains:annotations:15.0=testCompileClasspath,testRuntimeClasspath
org.jetbrains.kotlin:kotlin-reflect:1.6.21=testCompileClasspath,testRuntimeClasspath
org.jetbrains.kotlin:kotlin-stdlib-common:1.6.21=testCompileClasspath,testRuntimeClasspath
org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.6.21=testCompileClasspath,testRuntimeClasspath
org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.6.21=testCompileClasspath,testRuntimeClasspath

and as such my builds should pass.

Steps to Reproduce

run ./gradlew resolveAndLockAll --write-locks

Examine gradle.lockfile and observe:

org.jetbrains:annotations:15.0=testRuntimeClasspath
org.jetbrains.kotlin:kotlin-reflect:1.6.21=testRuntimeClasspath
org.jetbrains.kotlin:kotlin-stdlib-common:1.6.21=testRuntimeClasspath
org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.6.21=testRuntimeClasspath
org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.6.21=testRuntimeClasspath

run ./gradlew clean test and see failure

Example Project: https://github.com/shand-j/issue-21396

Build scan URL: https://gradle.com/s/xot6idbw437fm

[breigo]

Any news or progress on this one?

[oshai]

Hi, we were having a similar issue and since there is a reproduction here looked into it. Didn't find the root cause, but here are some findings:

We were focusing on the missing org.jetbrains:annotations.
When running rALA it yields the following (broken) lock:

org.jetbrains:annotations:15.0=testRuntimeClasspath

When running ./gradlew rALA --update-locks org.jetbrains:annotations dependencies seems ok:

org.jetbrains:annotations:13.0=testCompileClasspath
org.jetbrains:annotations:15.0=testRuntimeClasspath

It seems that it should come transitively from two deps:

• org.jetbrains.kotlin:kotlin-stdlib (v13)
• com.vladsch.flexmark:flexmark-util-ast (v15, test runtime only)

We also found few things on debug:

A repeating message:

[org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.NodeState]
version for org.jetbrains:annotations:13.0(runtime) is not selected. ignoring.

When looking in debugger, component state is Evicted:

Looking at resolveState.nodes.incomingEdges we see the expected two for v15:

[oshai]

Noticed when running update locks the following is logged:

[org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.DependencyGraphBuilder] 
Visiting configuration org.jetbrains:annotations:13.0(compile).

(update locks works as expected). In write locks the dependency only appears as runtime:

[org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.DependencyGraphBuilder] 
Visiting configuration org.jetbrains:annotations:13.0(runtime)

[nivsa]

Not sure if its related to the issue but I've also noticed that when it traverses the testCompileClasspath configuration as part of the resolveAndLockAll --write-locks the queue contains nodes only from apiElements (there isn't a single node which is part of testCompileClasspath)

[oshai]

From the outgoing edges of kotlin-stdlib it's also noticable that this points to annotation v15 (although v13 is requested).

The code at NodeState.visitOutgoingDependencies() for kotlin-stdlib is skipping outgoing edges without adding any outgoing edges.