Duplicated com.google classes

[harti2006]

Issue

The most recent version of this library compiles and includes some extra classes, originally coming from Google proto definitions. Those classes conflict with existing ones of the proto-google-common-protos library.

Affected version

1.4.1

Context

Output of the duplicate-finder maven plugin

[INFO] --- duplicate-finder:2.0.1:check (com.gropyus.tiles_duplicate-finder_0.14.121__default) @ authz-api ---
[INFO] Checking compile classpath
[INFO] Checking runtime classpath
[INFO] Checking test classpath
[WARNING] Found duplicate and different classes in [com.authzed.api:authzed:1.4.1, com.google.api.grpc:proto-google-common-protos:2.61.2]:
[WARNING]   com.google.api.AnnotationsProto
[WARNING]   com.google.api.CustomHttpPattern
[WARNING]   com.google.api.Http
[WARNING]   com.google.api.HttpProto
[WARNING]   com.google.api.HttpRule
[WARNING]   com.google.rpc.Status
[WARNING]   com.google.rpc.StatusProto
[WARNING] Found duplicate classes/resources in compile classpath.
[WARNING] Found duplicate and different classes in [com.authzed.api:authzed:1.4.1, com.google.api.grpc:proto-google-common-protos:2.61.2]:
[WARNING]   com.google.api.AnnotationsProto
[WARNING]   com.google.api.CustomHttpPattern
[WARNING]   com.google.api.Http
[WARNING]   com.google.api.HttpProto
[WARNING]   com.google.api.HttpRule
[WARNING]   com.google.rpc.Status
[WARNING]   com.google.rpc.StatusProto
[WARNING] Found duplicate classes/resources in runtime classpath.
[WARNING] Found duplicate and different classes in [com.authzed.api:authzed:1.4.1, com.google.api.grpc:proto-google-common-protos:2.61.2]:
[WARNING]   com.google.api.AnnotationsProto
[WARNING]   com.google.api.CustomHttpPattern
[WARNING]   com.google.api.Http
[WARNING]   com.google.api.HttpProto
[WARNING]   com.google.api.HttpRule
[WARNING]   com.google.rpc.Status
[WARNING]   com.google.rpc.StatusProto
[WARNING] Found duplicate classes/resources in test classpath.

Output of mvn dependency:tree -Dverbose=true

com.google.api.grpc:proto-google-common-protos is a transitive dependency of grpc-protobuf:

[INFO] --- dependency:3.7.0:tree (default-cli) @ test ---
[INFO] test:test:jar:0.0.1
[INFO] +- com.authzed.api:authzed:jar:1.4.1:compile
[INFO] |  +- com.google.protobuf:protobuf-java:jar:4.31.1:compile
[INFO] |  +- io.grpc:grpc-protobuf:jar:1.73.0:runtime
[INFO] |  |  +- (io.grpc:grpc-api:jar:1.73.0:runtime - omitted for conflict with 1.72.0)
[INFO] |  |  +- (com.google.code.findbugs:jsr305:jar:3.0.2:runtime - omitted for duplicate)
[INFO] |  |  +- com.google.api.grpc:proto-google-common-protos:jar:2.51.0:runtime
[INFO] |  |  +- (com.google.guava:guava:jar:33.3.1-android:runtime - omitted for duplicate)
[INFO] |  |  \- io.grpc:grpc-protobuf-lite:jar:1.73.0:runtime
[INFO] |  |     +- (io.grpc:grpc-api:jar:1.73.0:runtime - omitted for conflict with 1.72.0)
[INFO] |  |     +- (com.google.code.findbugs:jsr305:jar:3.0.2:runtime - omitted for duplicate)
[INFO] |  |     \- (com.google.guava:guava:jar:33.3.1-android:runtime - omitted for duplicate)

Reproduce

Here is a minimal POM file, with the configuration taken from your README + the duplicate-finder-maven-plugin.
Run "mvn clean verify" to see the error.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>test</groupId>
    <artifactId>test</artifactId>
    <version>0.0.1</version>

    <dependencies>
        <dependency>
            <groupId>com.authzed.api</groupId>
            <artifactId>authzed</artifactId>
            <version>1.4.1</version>
        </dependency>
        <dependency>
            <groupId>io.grpc</groupId>
            <artifactId>grpc-api</artifactId>
            <version>1.73.0</version>
        </dependency>
        <dependency>
            <groupId>io.grpc</groupId>
            <artifactId>grpc-stub</artifactId>
            <version>1.73.0</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.basepom.maven</groupId>
                <artifactId>duplicate-finder-maven-plugin</artifactId>
                <version>2.0.1</version>
                <executions>
                    <execution>
                        <phase>validate</phase>
                        <goals>
                            <goal>check</goal>
                        </goals>
                        <configuration>
                            <failBuildInCaseOfDifferentContentConflict>true</failBuildInCaseOfDifferentContentConflict>
                            <failBuildInCaseOfEqualContentConflict>false</failBuildInCaseOfEqualContentConflict>
                            <ignoredClassPatterns>
                                <ignoredClassPattern>.*\.module-info</ignoredClassPattern>
                            </ignoredClassPatterns>
                            <ignoredResourcePatterns>
                                <ignoredResourcePattern>.*</ignoredResourcePattern>
                            </ignoredResourcePatterns>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

Expected behavior

The JAR of com.authzed.api:authzed only includes "com.authzed" classes, as it was back in version 1.3.x.

[tstirrat15]

I'm guessing it was #141 that broke this.

For clarification, is the issue that it conflicts with something that a user would install, or is the issue that multiple copies of the same lib are being included by the build process?

Is there something that I can add to the build process that would automatically deduplicate things by any chance? I'm not well-versed in java build tooling.

[harti2006]

You are probably right, it looks like all the third-party proto definitions are now included into the build process. But I'd assume that those external protos already exist as libraries that can be pulled in from Maven Central.

It seems like "protovalidate" is available here: https://central.sonatype.com/artifact/build.buf/protovalidate.
Is this dependency optional? Then maybe let the clients decide whether to pull it in or not.

[tstirrat15]

Proto is weird... there's an implicit assumption in the protobuf ecosystem that you're collecting all of the protobuf definitions for the services you're talking to and then doing your own generation as a part of your own build process. What we're doing here with generating a library and then publishing it is strictly outside of that expectation.

As for protovalidate, my understanding is that there's two pieces to it: there's the protovalidate proto, which is what this library is including as one of its transitive proto dependencies, and then there's the protovalidate java library, which takes the annotations on the generated classes and uses them to make runtime assertions. I think the artifact you're referencing above is the runtime library, which is not included by this library to my knowledge.

I'll have a look at the repro you've put together and at the build artifacts produced by this library and see if I can make sense of it.

[tstirrat15]

Okay, I have a clearer idea of what it's complaining about. Now I've gotta figure out what the appropriate thing to exclude is...

[tstirrat15]

Does #146 seem like a sane approach to you? I'm a little in the dark on how things ought to work in Java.

[harti2006]

@tstirrat15 thanks for jumping on that issue. The proposed solution might work in a minimal setup (I haven't tried it). In my case I'm building a service that connects different gRPC endpoints. proto-google-common-protos will just be pulled in from any other client library that I'm using :(
I'd still find it "nicer" if a library just declares a dependency on a another library in a specific version, than to take some third-party classes, compile them and ship them in you own artifact.

So my suggestions are:

• exclude google proto definitions from the compile step and simply keep the transitive dependency on proto-google-common-protos (can even make it explicit)
• exclude buf validate proto definitions from the compile step and add an extra dependency on build.buf:protovalidate. I just checked and can confirm that it contains the compiled classes for the protos that your are using in this library

It's basically a revert of #141 + adding an extra dependency.

What do you think?

[tstirrat15]

Reverting #141 is a non-starter because it started failing when we brought in the protovalidate proto definitions in the build step. I was struggling yesterday to exclude the common proto definitions from the compile step, but I'll have another go at that today.