4.4-rc-1 regression: NullPointerException with shadow plugin 1.2.4

[huxi]

A build that was working with Gradle 4.3.x is throwing a NullPointerException with Gradle 4.4-rc-1.

Expected Behavior

No exception is thrown.

Current Behavior

The following exception is thrown:

Caused by: java.lang.NullPointerException
        at org.gradle.api.internal.attributes.DefaultImmutableAttributesFactory.concat(DefaultImmutableAttributesFactory.java:61)
        at org.gradle.api.internal.attributes.DefaultImmutableAttributesFactory.of(DefaultImmutableAttributesFactory.java:56)
        at org.gradle.api.internal.java.JavaLibrary$AbstractUsageContext.<init>(JavaLibrary.java:102)
        at org.gradle.api.internal.java.JavaLibrary$BackwardsCompatibilityUsageContext.<init>(JavaLibrary.java:164)
        at org.gradle.api.internal.java.JavaLibrary$BackwardsCompatibilityUsageContext.<init>(JavaLibrary.java:160)
        at org.gradle.api.internal.java.JavaLibrary.<init>(JavaLibrary.java:78)
        at com.github.jengelman.gradle.plugins.shadow.ShadowJavaPlugin$ShadowJavaLibrary.<init>(ShadowJavaPlugin.groovy:87)
        at com.github.jengelman.gradle.plugins.shadow.ShadowJavaPlugin.configureShadowTask(ShadowJavaPlugin.groovy:62)
        at com.github.jengelman.gradle.plugins.shadow.ShadowJavaPlugin.apply(ShadowJavaPlugin.groovy:35)
        at com.github.jengelman.gradle.plugins.shadow.ShadowJavaPlugin.apply(ShadowJavaPlugin.groovy)
        at org.gradle.api.internal.plugins.ImperativeOnlyPluginTarget.applyImperative(ImperativeOnlyPluginTarget.java:42)
        at org.gradle.api.internal.plugins.RuleBasedPluginTarget.applyImperative(RuleBasedPluginTarget.java:50)
        at org.gradle.api.internal.plugins.DefaultPluginManager.addPlugin(DefaultPluginManager.java:165)
        at org.gradle.api.internal.plugins.DefaultPluginManager.access$200(DefaultPluginManager.java:47)
        at org.gradle.api.internal.plugins.DefaultPluginManager$AddPluginBuildOperation.run(DefaultPluginManager.java:252)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:336)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:328)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:199)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:110)
        at org.gradle.api.internal.plugins.DefaultPluginManager.doApply(DefaultPluginManager.java:144)
        ... 129 more

Context

Example build.gradle file:

buildscript {
    repositories {
    	jcenter()
        mavenCentral()
    }

    dependencies {
        classpath 'com.github.jengelman.gradle.plugins:shadow:1.2.4'
    }
}

wrapper {
    gradleVersion = '4.4-rc-1'
}

apply plugin: 'java'
apply plugin: 'com.github.johnrengelman.shadow'

defaultTasks 'build', 'shadowJar'

Steps to Reproduce (for bugs)

Executing the above file will work with Gradle 4.3 and fail with Gradle 4.4-rc-1.

Your Environment

------------------------------------------------------------
Gradle 4.4-rc-1
------------------------------------------------------------

Build time:   2017-11-21 15:31:05 UTC
Revision:     089fe1e08ca477b49583c6e00abf87f4675d0c06

Groovy:       2.4.12
Ant:          Apache Ant(TM) version 1.9.9 compiled on February 2 2017
JVM:          1.8.0_152 (Oracle Corporation 25.152-b16)
OS:           Mac OS X 10.12.6 x86_64

[jjohannes]

The issue was introduced with these two commits:

• a1095ce
  Introduces DefaultImmutableAttributesFactory.concat() which uses the isolatableFactory.
• 1786557
  Passes null as isolatableFactory.

[jjohannes]

Note this happens inside a code block that was explicitly kept for backwards compatibility with older versions of the shadow plugin.

gradle/subprojects/plugins/src/main/java/org/gradle/api/internal/java/JavaLibrary.java

Lines 69 to 81 in 1786557

	/**
	* This constructor should not be used, and is maintained only for backwards
	* compatibility with the widely used Shadow plugin.
	*/
	@Deprecated
	public JavaLibrary(PublishArtifact jarArtifact, DependencySet runtimeDependencies) {
	this.artifacts.add(jarArtifact);
	this.objectFactory = DEPRECATED_OBJECT_FACTORY;
	this.attributesFactory = new DefaultImmutableAttributesFactory(null);
	this.runtimeUsage = new BackwardsCompatibilityUsageContext(Usage.JAVA_RUNTIME, runtimeDependencies);
	this.compileUsage = new BackwardsCompatibilityUsageContext(Usage.JAVA_API, runtimeDependencies);
	this.configurations = null;
	}

We could consider to remove this code altogether. That would still give an error, but one that would hint at the fact that you need to upgrade the plugin.

Shadow plugin 2.0.0+ seems to work.

[jjohannes]

@huxi we are considering to remove the JavaLibrary construct that causes the issue. It is internal API that is not really supported anymore. However, shadow plugin 1.2.4 would not work anymore afterwards. Is there a specific reason you used 1.2.4 instead of a newer version of the plugin?

[jjohannes]

We actually made the decision to upgrade our plugin integration tests for the shadow plugin: #3080
Thus we should not make explicit efforts to support older versions (if they use internal API as it is the case here).

PR for removing the deprecated internal constructor that can produce the NPE reported in this issue: #3591

[huxi]

I'm still using 1.2.4 of the plugin because 2.0.x fails to work for me and I didn't have the time to isolate the problem for a proper bug report over at the shadow project.

The fact that 2.0.1 is still causing Gradle deprecation warnings didn't exactly increase my motivation to upgrade the plugin, either.

This is the exception thrown with Gradle 4.3.1 and shadow plugin 2.0.1:

Caused by: org.gradle.api.InvalidUserDataException: A POM cannot have multiple artifacts with the same type and classifier. Already have MavenArtifact de.huxhorn.lilith:zip:zip:null, trying to add MavenArtifact de.huxhorn.lilith:zip:zip:null.
        at org.gradle.api.publication.maven.internal.DefaultArtifactPom.addArtifact(DefaultArtifactPom.java:65)
        at org.gradle.api.publication.maven.internal.DefaultArtifactPomContainer.addArtifact(DefaultArtifactPomContainer.java:52)
        at org.gradle.api.publication.maven.internal.deployer.AbstractMavenResolver.collectArtifact(AbstractMavenResolver.java:102)
        at org.gradle.api.publication.maven.internal.deployer.AbstractMavenResolver.publish(AbstractMavenResolver.java:93)
        at org.gradle.api.publication.maven.internal.deployer.BaseMavenDeployer.publish(BaseMavenDeployer.java:34)
        at org.gradle.api.internal.artifacts.ivyservice.publisher.DefaultIvyDependencyPublisher.publish(DefaultIvyDependencyPublisher.java:42)
        at org.gradle.api.internal.artifacts.ivyservice.publisher.IvyBackedArtifactPublisher.publish(IvyBackedArtifactPublisher.java:78)
        at org.gradle.api.internal.artifacts.ivyservice.IvyContextualArtifactPublisher$1.execute(IvyContextualArtifactPublisher.java:43)
        at org.gradle.api.internal.artifacts.ivyservice.IvyContextualArtifactPublisher$1.execute(IvyContextualArtifactPublisher.java:40)
        at org.gradle.internal.Transformers$4.transform(Transformers.java:133)
        at org.gradle.api.internal.artifacts.ivyservice.DefaultIvyContextManager.withIvy(DefaultIvyContextManager.java:61)
        at org.gradle.api.internal.artifacts.ivyservice.DefaultIvyContextManager.withIvy(DefaultIvyContextManager.java:39)
        at org.gradle.api.internal.artifacts.ivyservice.IvyContextualArtifactPublisher.publish(IvyContextualArtifactPublisher.java:40)
        at org.gradle.api.tasks.Upload.upload(Upload.java:65)
        ... 42 more

It's not happening while executing the Shadow plugin itself but instead while executing the uploadPublished task afterwards. I postponed a deep-dive into this issue because I didn't expect breakage before Gradle 5.

[bigdaz]

Thanks for reporting @huxi. We've found a fix and have added back smoke test coverage for Shadow-1.2.4. We'll try our best to keep this working until 5.0.

[bigdaz]

And the fix is in: #3600
@huxi Not sure if you'd like to build from source to confirm it works for you. Otherwise you'll need to wait for RC2

[huxi]

Thanks. I'll give RC2 a shot when it's out.

[huxi]

Everything fine with RC2.