Provided dependency that is also a transitive dependency should be considered provided

[fxbonnet]

• I've read the CONTRIBUTING guidelines

steps

If I have a project project1 that has a dependency (compile) to a library
And a project project2 that has a dependency (compile) to project1 but in project2 the same library is provided

problem

In project2 the library is resolved as a compile dependency. As a result if I generate a fat jar, it is included in the jar while it is not needed.

expectation

As the library is provided in project2 it should be resolved as provided and not included when doing an assembly

notes

Sample project to reproduce the issue:

build.sbt

name := "testProvidedDependencies"
version := "0.1"
scalaVersion in ThisBuild := "2.12.8"

lazy val project1 = project
  .settings(
    libraryDependencies += "org.apache.commons" % "commons-lang3" % "3.5")

lazy val project2 = project
  .dependsOn(project1)
  .settings(
    libraryDependencies += "org.apache.commons" % "commons-lang3" % "3.5" % Provided)

plugins.sbt

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.7")
addSbtPlugin("net.virtual-void" % "sbt-dependency-graph" % "0.9.2")

sbt version: 1.2.8

[eed3si9n]

@fxbonnet Thanks for the report.

This does not match my expectation. Compile configuration extends Runtime configuration. Because project2 depends on project1, you will have "commons-lang3" in the Runtime.

Provided basically means that you have in Compile, but not in Runtime.

If you extend your logic, a direct dependency in Compile configuration will evict dependencies off of Test too?

[fxbonnet]

@eed3si9n thanks for your answer.
I am not familiar with sbt internal logic about configurations and inheritance rules. I am just saying that in that particular case it does not work.

But you are perfectly right, if I have a test dependency in project2 but it is also a compile dependency in project1, the dependency is required at runtime as well as for the tests (compile and runtime) so it should end up as compile.
I am not sure it is possible to define a generic eviction policy that applies to any configuration based only on inheritance between configurations. compile, test and provided are quite different things.

My example works as I expect with maven commons-lang3 ends up as provided and therefore is not included in the fat jar.

If you want to reproduce:

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>test</groupId>
    <artifactId>testProvidedDeps</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>pom</packaging>
    <modules>
        <module>project1</module>
        <module>project2</module>
    </modules>
</project>

project1/pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>testProvidedDeps</artifactId>
        <groupId>test</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>
    <artifactId>project1</artifactId>
    <dependencies>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
            <version>3.9</version>
        </dependency>
    </dependencies>
</project>

project2/pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>testProvidedDeps</artifactId>
        <groupId>test</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>
    <artifactId>project2</artifactId>
    <dependencies>
        <dependency>
            <groupId>${project.groupId}</groupId>
            <artifactId>project1</artifactId>
            <version>${project.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
            <version>3.9</version>
            <scope>provided</scope>
        </dependency>
    </dependencies>
</project>

[eed3si9n]

I think you'd need to mess with sbt-assembly settings to exclude it manually.

[eejbyfeldt]

I agree with @fxbonnet that I don't think the current behavior is the desired one or least some way of achive the other behavior is desired. Am I missing some easy way that currently achieves the same thing? If not I would be willing to help implement such a feature.

My use case is that we currently build a fat jar using sbt-assembly for running against a a provided spark runtime. So it would be desired that the fat jar does not contain spark or any of sparks transitive dependencies (if it does it easy to end up with multiple versions of libraries). From what I can tell sbt-assembly does not provide some easy way to exclude all the transitive dependencies and you end up needing to exclude all (or at least all that you have overlap with) of them manually.

[eed3si9n]

To some extent this is more of a design problem at the layer of dependency management engine. Could either of you open an issue on coursier/coursier? My guess is that any implementation effort would need to happen there.