Shadowed JAR is not correctly loaded into `kotlinCompilerPluginClasspath`

[ZacSweers]

Expected and Results

When testing rc2, I found that relocation seems to happen but usages in the jar are not updated to the new locations

Related environment and versions

No response

Reproduction steps

ZacSweers/metro#775

The sample apps consume the compiler plugin there. Can be reproduced with ./gradlew -p samples :integration-tests:compileTestKotlinLinuxX64

Anything else?

No response

[Goooler]

OLD: compiler-0.6.0-SNAPSHOT-all-before.jar
NEW: compiler-0.6.0-SNAPSHOT-all-after.jar

 JAR   │ old     │ new     │ diff 
───────┼─────────┼─────────┼──────
 class │   5 MiB │   5 MiB │  0 B 
 other │ 2.2 KiB │ 2.2 KiB │  0 B 
───────┼─────────┼─────────┼──────
 total │   5 MiB │   5 MiB │  0 B 

 CLASSES │ old  │ new  │ diff      
─────────┼──────┼──────┼───────────
 classes │  623 │  623 │ 0 (+0 -0) 
 methods │ 9153 │ 9153 │ 0 (+0 -0) 
  fields │ 2047 │ 2047 │ 0 (+0 -0) 

[Goooler]

Caused by #1502.

diff --color=auto -r metro-before/compiler/0.6.0-SNAPSHOT/compiler-0.6.0-SNAPSHOT.module metro-after/compiler/0.6.0-SNAPSHOT/compiler-0.6.0-SNAPSHOT.module
93c93
<         "org.gradle.jvm.version": 23,
---
>         "org.gradle.jvm.version": 11,
118,121c118,121
<           "sha512": "5894bb8854f703f843124bedd140988b3312aeefba624d70729d06e0a2c21cb8b62e7001a37e860f81a971f39aa6467dd1b5d91379c0be40ce497930a02ce8e8",
<           "sha256": "d583e2777b164936f21a5ce79d6db01d1754ed03bd7e90e41413d4ad3968e90d",
<           "sha1": "4e4a4aeb349ae94b1ee04a53c0b2f8bfdca2e4d5",
<           "md5": "5c4c778d0a7ae389b978a78cfbf174f1"
---
>           "sha512": "70b4c0bfba095dddf2ada5e2b196d552d70fdfc6bf0ec1b8b168defc4ba3092a515829727d80f2c0ead6671f0dc0c253a1f6bbc1c49f7ee16e8e8479da849483",
>           "sha256": "963885ff7bdb6ade69b77814b070719c037bc7a5e8d65c4ba7b75c41b3cae0c4",
>           "sha1": "98307ae44aa2088e9dfbfcd131c1a5effffd74c2",
>           "md5": "b8051d747c35b4b73a1d6f6e40ad5bb2"

The org.gradle.jvm.version has been changed from 23 to 11, and 11 is the correct value, which reflects https://github.com/ZacSweers/metro/blob/62f45c6f43e11917ee1f30e3f1b0d7fd8708821f/build.gradle.kts#L147

[ZacSweers]

That seems orthogonal to relocation?

[Goooler]

Yep.

[ZacSweers]

Not sure why this was closed? Remains broken in rc3: ZacSweers/metro#864

[Goooler]

I believe this issue is due to your wrong configuration, not related to the relocation stuff.

[ZacSweers]

What is wrong with my configuration? You did send this other PR but it was unusable as it incorrectly attempted to shade all dependencies.

[Goooler]

#1540 (comment)

Your case looks related to #1519.

[Goooler]

e: java.lang.NoClassDefFoundError: com/squareup/wire/Message
  at java.base/java.lang.ClassLoader.defineClass1(Native Method)
  at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
  at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
  at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:524)
  at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:427)
  at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:421)
  at java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
  at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:420)
  at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:592)
  at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
  at dev.zacsweers.metro.compiler.ir.transformers.BindingContainerTransformer.findContainer(BindingContainerTransformer.kt:192)
  at dev.zacsweers.metro.compiler.ir.transformers.BindingContainerTransformer.findContainer$default(BindingContainerTransformer.kt:109)
  at dev.zacsweers.metro.compiler.ir.transformers.DependencyGraphTransformer.visitClass(DependencyGraphTransformer.kt:116)
  at org.jetbrains.kotlin.ir.visitors.IrElementTransformerVoid.visitClass(IrElementTransformerVoid.kt:57)
  at org.jetbrains.kotlin.ir.visitors.IrElementTransformerVoid.visitClass(IrElementTransformerVoid.kt:19)
  at org.jetbrains.kotlin.ir.declarations.IrClass.accept(IrClass.kt:72)
  at org.jetbrains.kotlin.ir.IrElementBase.transform(IrElementBase.kt:33)
  at org.jetbrains.kotlin.ir.util.TransformKt.transformInPlace(transform.kt:35)
  at org.jetbrains.kotlin.ir.declarations.IrClass.transformChildren(IrClass.kt:82)
  at org.jetbrains.kotlin.ir.visitors.IrElementTransformerVoid.visitDeclaration(IrElementTransformerVoid.kt:40)
  at org.jetbrains.kotlin.ir.visitors.IrElementTransformerVoid.visitClass(IrElementTransformerVoid.kt:54)
  at dev.zacsweers.metro.compiler.ir.transformers.DependencyGraphTransformer.visitClass(DependencyGraphTransformer.kt:120)
  at org.jetbrains.kotlin.ir.visitors.IrElementTransformerVoid.visitClass(IrElementTransformerVoid.kt:57)
  at org.jetbrains.kotlin.ir.visitors.IrElementTransformerVoid.visitClass(IrElementTransformerVoid.kt:19)
  at org.jetbrains.kotlin.ir.declarations.IrClass.accept(IrClass.kt:72)
  at org.jetbrains.kotlin.ir.IrElementBase.transform(IrElementBase.kt:33)
  at org.jetbrains.kotlin.ir.util.TransformKt.transformInPlace(transform.kt:35)
  at org.jetbrains.kotlin.ir.declarations.IrClass.transformChildren(IrClass.kt:82)
  at org.jetbrains.kotlin.ir.visitors.IrElementTransformerVoid.visitDeclaration(IrElementTransformerVoid.kt:40)
  at org.jetbrains.kotlin.ir.visitors.IrElementTransformerVoid.visitClass(IrElementTransformerVoid.kt:54)
  at dev.zacsweers.metro.compiler.ir.transformers.DependencyGraphTransformer.visitClass(DependencyGraphTransformer.kt:120)
  at org.jetbrains.kotlin.ir.visitors.IrElementTransformerVoid.visitClass(IrElementTransformerVoid.kt:57)
  at org.jetbrains.kotlin.ir.visitors.IrElementTransformerVoid.visitClass(IrElementTransformerVoid.kt:19)
  at org.jetbrains.kotlin.ir.declarations.IrClass.accept(IrClass.kt:72)
  at org.jetbrains.kotlin.ir.IrElementBase.transform(IrElementBase.kt:33)
  at org.jetbrains.kotlin.ir.util.TransformKt.transformInPlace(transform.kt:35)
  at org.jetbrains.kotlin.ir.declarations.IrPackageFragment.transformChildren(IrPackageFragment.kt:31)
  at org.jetbrains.kotlin.ir.visitors.IrElementTransformerVoid.visitPackageFragment(IrElementTransformerVoid.kt:146)
  at org.jetbrains.kotlin.ir.visitors.IrElementTransformerVoid.visitFile(IrElementTransformerVoid.kt:160)
  at org.jetbrains.kotlin.ir.visitors.IrElementTransformerVoid.visitFile(IrElementTransformerVoid.kt:163)
  at org.jetbrains.kotlin.ir.visitors.IrElementTransformerVoid.visitFile(IrElementTransformerVoid.kt:19)
  at org.jetbrains.kotlin.ir.declarations.IrFile.accept(IrFile.kt:27)
  at org.jetbrains.kotlin.ir.declarations.IrFile.transform(IrFile.kt:30)
  at org.jetbrains.kotlin.ir.declarations.IrFile.transform(IrFile.kt:19)
  at org.jetbrains.kotlin.ir.util.TransformKt.transformInPlace(transform.kt:35)
  at org.jetbrains.kotlin.ir.declarations.IrModuleFragment.transformChildren(IrModuleFragment.kt:46)
  at org.jetbrains.kotlin.ir.visitors.IrElementTransformerVoid.visitModuleFragment(IrElementTransformerVoid.kt:102)
  at org.jetbrains.kotlin.ir.visitors.IrElementTransformerVoid.visitModuleFragment(IrElementTransformerVoid.kt:107)
  at org.jetbrains.kotlin.ir.visitors.IrElementTransformerVoid.visitModuleFragment(IrElementTransformerVoid.kt:19)
  at org.jetbrains.kotlin.ir.declarations.IrModuleFragment.accept(IrModuleFragment.kt:36)
  at org.jetbrains.kotlin.ir.declarations.IrModuleFragment.transform(IrModuleFragment.kt:39)
  at dev.zacsweers.metro.compiler.ir.MetroIrGenerationExtension.generateInner$lambda$4$lambda$3(MetroIrGenerationExtension.kt:69)
  at dev.zacsweers.metro.compiler.tracing.TracerKt.trace(Tracer.kt:86)
  at dev.zacsweers.metro.compiler.tracing.TracerKt.traceNested(Tracer.kt:79)
  at dev.zacsweers.metro.compiler.tracing.TracerKt.traceNested$default(Tracer.kt:73)
  at dev.zacsweers.metro.compiler.ir.MetroIrGenerationExtension.generateInner$lambda$4(MetroIrGenerationExtension.kt:61)
  at dev.zacsweers.metro.compiler.tracing.TracerKt.trace(Tracer.kt:86)
  at dev.zacsweers.metro.compiler.ir.MetroIrGenerationExtension.generateInner(MetroIrGenerationExtension.kt:48)
  at dev.zacsweers.metro.compiler.ir.MetroIrGenerationExtension.generate(MetroIrGenerationExtension.kt:41)
  at org.jetbrains.kotlin.fir.pipeline.ConvertToIrKt.applyIrGenerationExtensions(convertToIr.kt:468)
  at org.jetbrains.kotlin.fir.pipeline.Fir2IrPipeline.runActualizationPipeline(convertToIr.kt:245)
  at org.jetbrains.kotlin.fir.pipeline.Fir2IrPipeline.convertToIrAndActualize(convertToIr.kt:128)
  at org.jetbrains.kotlin.fir.pipeline.ConvertToIrKt.convertToIrAndActualize(convertToIr.kt:97)
  at org.jetbrains.kotlin.fir.pipeline.ConvertToIrKt.convertToIrAndActualize$default(convertToIr.kt:72)
  at org.jetbrains.kotlin.backend.konan.Fir2IrKt.fir2Ir(Fir2Ir.kt:72)
  at org.jetbrains.kotlin.backend.konan.driver.phases.Fir2IrKt.Fir2IrPhase$lambda$1(Fir2Ir.kt:28)
  at org.jetbrains.kotlin.backend.common.phaser.PhaseBuildersKt$createSimpleNamedCompilerPhase$1.phaseBody(PhaseBuilders.kt:58)

com.squareup.wire.Message has already been relocated to dev.zacsweers.metro.compiler.shaded.com.squareup.wire,Message

I guess the non-shadowed JAR was loaded by the Kotlin plugin for some reason, which relects

<         "org.gradle.jvm.version": 23,
---
>         "org.gradle.jvm.version": 11,

When I reverted #1502, published a snapshot version to the maven local, and tried out the local Shadow version for Metro, everything worked well.

[martinbonnin]

Did a little bit more digging. It was only working by chance before because Gradle disambiguates the jvm target to use the highest possible: see JavaEcosystemSupport:120

With the change from #1502, both the runtimeElements and shadowRuntimeElements now have the same jvm target (11) and so the bundling now decides the disambiguation: see JavaEcosystemSupport:370

Because external is favoured, it is taking over the shadowed bundling and therefore the wrong jar is loaded in the classpath.

That at least explains it but TBH I have no idea how to fix it. I would maybe expose an option in shadow to remove the external outgoing variants to make sure they do not get selected by mistake.

[Goooler]

Sounds reasonable!

[Goooler]

I attempted to remove TargetJvmVersion in #1564, but it still didn't work for Metro's case.

[ZacSweers]

Seems like this should be reopened at least? It's surprising that an update like this would break a typical use and doesn't seem to have a workaround