Detached configurations cannot extend from existing configurations

[melix]

Context

Dependency resolution using attribute matching doesn't work if the configuration being resolved is a detached configuration. I'm actually surprised we went that far without fixing this. Given the following build file:

subprojects {
   apply plugin: 'java-library'
}

project(":app") {
    dependencies {
        implementation project(":lib")
    }

    tasks.create("resolve") {
       def detached = configurations.detachedConfiguration()
       detached.with {
           canBeResolved = true
           canBeConsumed = false
           extendsFrom(configurations.implementation)
           attributes {
               attribute(Attribute.of('org.gradle.usage', Usage), project.objects.named(Usage, 'java-api'))
           }
       }
       println configurations.compileClasspath.files
       println detached.files
    }
}

I would expect the detached.files collection to be the same as compileClasspath.files, given that it asks for the same attributes and both extend implementation. However, in practice, the collection is empty. Replacing configurations.detachedConfiguration() with configuration.create("foo") solves the problem, indicating that the issue is really with detached configurations, not with attribute matching.

[melix]

FWIW, there's a first problem which is that detached configurations cannot inherit from configurations defined in a different container. So, in fact, we get no dependencies because we don't inherit dependencies from the parent configuration here.

[melix]

Yes, I can confirm it works properly if we include the set of configurations from the container the detached configuration was created from in the list.

[melix]

Updated the title to reflect the real problem

[stale]

This issue has been automatically marked as stale because it has not had recent activity. Given the limited bandwidth of the team, it will be automatically closed if no further activity occurs. If you're interested in how we try to keep the backlog in a healthy state, please read our blog post on how we refine our backlog. If you feel this is something you could contribute, please have a look at our Contributor Guide. Thank you for your contribution.

[stale]

This issue has been automatically closed due to inactivity. If you can reproduce this on a recent version of Gradle or if you have a good use case for this feature, please feel free to reopen the issue with steps to reproduce, a quick explanation of your use case or a high-quality pull request.

[vlsi]

counter++

I think I hit this the second or the third time already

[chrisr3]

Interesting. I have tried creating a detached configuration that extends from another detached configuration, both from the same configuration container. The first configuration contains a dependency without a version, and the second contains a relevant platform dependency. My hope was that I could resolve the first configuration and then examine the ResolveConfiguration's only firstLevelModuleDependency, especially since allDependencies does contain what I expect. However, Gradle refuses to resolve this, forcing me to add both dependencies to a single detached configuration instead.

Problem observed when using Gradle 7.1.1.

[kristian]

I stumbled across the same issue and described it on the forum here:

def newConfiguration = project.configurations.detachedConfiguration()
newConfiguration.setCanBeResolved(true)
newConfiguration.extendsFrom(project.configurations.runtimeClasspath)

Resolving results in an empty array:

doFirst {
    println newConfiguration.resolve()
}

So this issue gets a +1 from me. Would love to see this one resolved. Workaround is to create a attached configuration.

[aSemy]

This problem is a little painful when trying to resolve the non-transitive elements of a configuration. The suggested workaround #14880 is to create a non-transitive Configuration that extends the transitive Configuration. It would be nice to create the intransitive Configuration as detached, so it does not get confused with the transitive configuration, generate unnecessary Kotlin DSL accessors, or pollute IDE auto-completion.

Here is an example that reproduces the problem. Copy it into a build.gradle.kts, and run

./gradlew listPluginsIntransitive

It should list the declared dependency, and the transitive dependencies, and then finally the declared dependency. However, the detached and attached intransitive configurations produce different results

Execution failed for task ':listPluginsIntransitive'.
> attached: kotlin-gradle-plugin-1.8.0-gradle76.jar
  detached: 

import org.gradle.api.attributes.Bundling.BUNDLING_ATTRIBUTE
import org.gradle.api.attributes.Bundling.EXTERNAL
import org.gradle.api.attributes.Category.CATEGORY_ATTRIBUTE
import org.gradle.api.attributes.Category.LIBRARY
import org.gradle.api.attributes.LibraryElements.JAR
import org.gradle.api.attributes.LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE
import org.gradle.api.attributes.Usage.JAVA_RUNTIME
import org.gradle.api.attributes.Usage.USAGE_ATTRIBUTE
import org.gradle.api.attributes.java.TargetJvmEnvironment.STANDARD_JVM
import org.gradle.api.attributes.java.TargetJvmEnvironment.TARGET_JVM_ENVIRONMENT_ATTRIBUTE

plugins {
  base
}

val generatorPlugins by configurations.registering {
  isCanBeResolved = true
  isCanBeConsumed = false
  isVisible = false
  attributes {
    attribute(USAGE_ATTRIBUTE, objects.named(JAVA_RUNTIME))
    attribute(CATEGORY_ATTRIBUTE, objects.named(LIBRARY))
    attribute(BUNDLING_ATTRIBUTE, objects.named(EXTERNAL))
    attribute(TARGET_JVM_ENVIRONMENT_ATTRIBUTE, objects.named(STANDARD_JVM))
    attribute(LIBRARY_ELEMENTS_ATTRIBUTE, objects.named(JAR))
  }
}

val generatorPluginsIntransitiveAttachedConfiguration = configurations.create("generatorPluginsIntransitive") {
  isCanBeResolved = true
  isCanBeConsumed = false
  isVisible = false
  isTransitive = false
  extendsFrom(generatorPlugins.get())
  attributes {
    attribute(USAGE_ATTRIBUTE, objects.named(JAVA_RUNTIME))
    attribute(CATEGORY_ATTRIBUTE, objects.named(LIBRARY))
    attribute(BUNDLING_ATTRIBUTE, objects.named(EXTERNAL))
    attribute(TARGET_JVM_ENVIRONMENT_ATTRIBUTE, objects.named(STANDARD_JVM))
    attribute(LIBRARY_ELEMENTS_ATTRIBUTE, objects.named(JAR))
  }
}

val generatorPluginsIntransitiveDetachedConfiguration = configurations.detachedConfiguration().apply {
  isCanBeResolved = true
  isCanBeConsumed = false
  isVisible = false
  isTransitive = false
  extendsFrom(generatorPlugins.get())
  attributes {
    attribute(USAGE_ATTRIBUTE, objects.named(JAVA_RUNTIME))
    attribute(CATEGORY_ATTRIBUTE, objects.named(LIBRARY))
    attribute(BUNDLING_ATTRIBUTE, objects.named(EXTERNAL))
    attribute(TARGET_JVM_ENVIRONMENT_ATTRIBUTE, objects.named(STANDARD_JVM))
    attribute(LIBRARY_ELEMENTS_ATTRIBUTE, objects.named(JAR))
  }
}

dependencies {
  "generatorPlugins"("org.jetbrains.kotlin:kotlin-gradle-plugin:1.8.0")
}

val listPlugins by tasks.registering {
  inputs.files(generatorPlugins)
  doLast {
    generatorPlugins.get().files.forEach { println(it.name) }
  }
}
val listPluginsIntransitive by tasks.registering {
  dependsOn(listPlugins)
  inputs.files(generatorPluginsIntransitiveAttachedConfiguration)
  inputs.files(generatorPluginsIntransitiveDetachedConfiguration)
  doLast {
    val attachedResult = generatorPluginsIntransitiveAttachedConfiguration.files.joinToString { it.name }
    val detachedResult = generatorPluginsIntransitiveDetachedConfiguration.files.joinToString { it.name }

    require(attachedResult == detachedResult) {
      """
        attached: $attachedResult
        detached: $detachedResult
      """.trimIndent()
    }
  }
}

> Task :listPlugins
kotlin-gradle-plugin-1.8.0-gradle76.jar
kotlin-gradle-plugin-model-1.8.0.jar
kotlin-gradle-plugin-api-1.8.0-gradle76.jar
kotlin-gradle-plugin-api-1.8.0.jar
kotlin-gradle-plugin-idea-proto-1.8.0.jar
kotlin-gradle-plugin-idea-1.8.0.jar
kotlin-project-model-1.8.0.jar
kotlin-tooling-core-1.8.0.jar
kotlin-util-klib-1.8.0.jar
kotlin-klib-commonizer-api-1.8.0.jar
kotlin-annotation-processing-gradle-1.8.0.jar
kotlin-android-extensions-1.8.0.jar
kotlin-compiler-runner-1.8.0.jar
kotlin-compiler-embeddable-1.8.0.jar
kotlin-scripting-compiler-embeddable-1.8.0.jar
kotlin-scripting-compiler-impl-embeddable-1.8.0.jar
kotlin-native-utils-1.8.0.jar
kotlin-util-io-1.8.0.jar
kotlin-daemon-embeddable-1.8.0.jar
trove4j-1.0.20200330.jar
jna-5.6.0.jar
kotlin-build-common-1.8.0.jar
kotlin-daemon-client-1.8.0.jar
kotlinx-coroutines-core-jvm-1.5.0.jar
kotlin-scripting-jvm-1.8.0.jar
kotlin-scripting-common-1.8.0.jar

> Task :listPluginsIntransitive FAILED
14 actionable tasks: 2 executed, 12 up-to-date

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':listPluginsIntransitive'.
> attached: kotlin-gradle-plugin-1.8.0-gradle76.jar
  detached: 

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1s
15:34:58: Execution finished 'listPluginsIntransitive'.

[rschmitt]

I just hit this bug yesterday and had to work around it by sort of manually copying the dependencies from the configuration I wanted to extend.

[jvandort]

We should fix this by failing fast instead of failing silently.

[marchermans]

We should fix this by failing fast instead of failing silently.

Why?, The detached configuration is created from a given container, restricting it from extending configuration reachable through that container, seems like an arbitrary blocker.
Is there a functional reason why this should not be possible?

[melix]

I agree that ideally this shouldn't fail but simply make it work. I guess that what Justin is saying is that it is currently broken anyway, so making it fail is a "better" solution in the sense that it would fail eagerly instead of being simply ignored. However it is reasonable to expect that this feature works.