Maven 3.9.10 fails randomly with message: Cannot evaluate expression '${project.artifactMap}'

[mnk]

Affected version

3.9.10

Bug description

After updating to version 3.9.10 we see random errors when testing large multi-module project. We have not seen this error with version 3.9.9.
The command is ./mvnw -T1C surefire:test -DforkCount=1 -B -e --no-snapshot-updates, and the failure is:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.5.2:test (default-cli) on project some-module: Unable to parse configuration of mojo org.apache.maven.plugins:maven-surefire-plugin:3.5.2:test for parameter projectArtifactMap: Cannot evaluate expression '${project.artifactMap}' for configuration entry 'projectArtifactMap': Error evaluating plugin parameter expression: project.artifactMap: InvocationTargetException: Cannot invoke "java.util.Set.add(Object)" because "this.artifacts" is null -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.5.2:test (default-cli) on project some-module: Unable to parse configuration of mojo org.apache.maven.plugins:maven-surefire-plugin:3.5.2:test for parameter projectArtifactMap: Cannot evaluate expression '${project.artifactMap}' for configuration entry 'projectArtifactMap'
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:333)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:193)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:180)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:572)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1144)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:642)
    at java.lang.Thread.run (Thread.java:1583)
Caused by: org.apache.maven.plugin.PluginConfigurationException: Unable to parse configuration of mojo org.apache.maven.plugins:maven-surefire-plugin:3.5.2:test for parameter projectArtifactMap: Cannot evaluate expression '${project.artifactMap}' for configuration entry 'projectArtifactMap'
    at org.apache.maven.plugin.internal.DefaultMavenPluginManager.populateMojoExecutionFields (DefaultMavenPluginManager.java:640)
    at org.apache.maven.plugin.internal.DefaultMavenPluginManager.getConfiguredMojo (DefaultMavenPluginManager.java:573)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:114)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:193)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:180)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:572)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1144)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:642)
    at java.lang.Thread.run (Thread.java:1583)
Caused by: org.codehaus.plexus.component.configurator.ComponentConfigurationException: Cannot evaluate expression '${project.artifactMap}' for configuration entry 'projectArtifactMap'
    at org.codehaus.plexus.component.configurator.converters.AbstractConfigurationConverter.fromExpression (AbstractConfigurationConverter.java:109)
    at org.codehaus.plexus.component.configurator.converters.AbstractConfigurationConverter.fromExpression (AbstractConfigurationConverter.java:60)
    at org.codehaus.plexus.component.configurator.converters.composite.MapConverter.fromConfiguration (MapConverter.java:55)
    at org.eclipse.sisu.plexus.CompositeBeanHelper.convertProperty (CompositeBeanHelper.java:260)
    at org.eclipse.sisu.plexus.CompositeBeanHelper.setProperty (CompositeBeanHelper.java:166)
    at org.codehaus.plexus.component.configurator.converters.composite.ObjectWithFieldsConverter.processConfiguration (ObjectWithFieldsConverter.java:157)
    at org.codehaus.plexus.component.configurator.BasicComponentConfigurator.configureComponent (BasicComponentConfigurator.java:37)
    at org.apache.maven.plugin.internal.DefaultMavenPluginManager.populateMojoExecutionFields (DefaultMavenPluginManager.java:617)
    at org.apache.maven.plugin.internal.DefaultMavenPluginManager.getConfiguredMojo (DefaultMavenPluginManager.java:573)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:114)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:193)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:180)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:572)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1144)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:642)
    at java.lang.Thread.run (Thread.java:1583)
Caused by: org.codehaus.plexus.component.configurator.expression.ExpressionEvaluationException: Error evaluating plugin parameter expression: project.artifactMap
    at org.apache.maven.plugin.PluginParameterExpressionEvaluator.evaluate (PluginParameterExpressionEvaluator.java:222)
    at org.codehaus.plexus.component.configurator.converters.AbstractConfigurationConverter.fromExpression (AbstractConfigurationConverter.java:76)
    at org.codehaus.plexus.component.configurator.converters.AbstractConfigurationConverter.fromExpression (AbstractConfigurationConverter.java:60)
    at org.codehaus.plexus.component.configurator.converters.composite.MapConverter.fromConfiguration (MapConverter.java:55)
    at org.eclipse.sisu.plexus.CompositeBeanHelper.convertProperty (CompositeBeanHelper.java:260)
    at org.eclipse.sisu.plexus.CompositeBeanHelper.setProperty (CompositeBeanHelper.java:166)
    at org.codehaus.plexus.component.configurator.converters.composite.ObjectWithFieldsConverter.processConfiguration (ObjectWithFieldsConverter.java:157)
    at org.codehaus.plexus.component.configurator.BasicComponentConfigurator.configureComponent (BasicComponentConfigurator.java:37)
    at org.apache.maven.plugin.internal.DefaultMavenPluginManager.populateMojoExecutionFields (DefaultMavenPluginManager.java:617)
    at org.apache.maven.plugin.internal.DefaultMavenPluginManager.getConfiguredMojo (DefaultMavenPluginManager.java:573)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:114)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:193)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:180)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:572)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1144)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:642)
    at java.lang.Thread.run (Thread.java:1583)
Caused by: java.lang.reflect.InvocationTargetException
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:115)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.codehaus.plexus.util.introspection.ReflectionValueExtractor.getPropertyValue (ReflectionValueExtractor.java:301)
    at org.codehaus.plexus.util.introspection.ReflectionValueExtractor.evaluate (ReflectionValueExtractor.java:212)
    at org.codehaus.plexus.util.introspection.ReflectionValueExtractor.evaluate (ReflectionValueExtractor.java:145)
    at org.apache.maven.plugin.PluginParameterExpressionEvaluator.evaluate (PluginParameterExpressionEvaluator.java:218)
    at org.codehaus.plexus.component.configurator.converters.AbstractConfigurationConverter.fromExpression (AbstractConfigurationConverter.java:76)
    at org.codehaus.plexus.component.configurator.converters.AbstractConfigurationConverter.fromExpression (AbstractConfigurationConverter.java:60)
    at org.codehaus.plexus.component.configurator.converters.composite.MapConverter.fromConfiguration (MapConverter.java:55)
    at org.eclipse.sisu.plexus.CompositeBeanHelper.convertProperty (CompositeBeanHelper.java:260)
    at org.eclipse.sisu.plexus.CompositeBeanHelper.setProperty (CompositeBeanHelper.java:166)
    at org.codehaus.plexus.component.configurator.converters.composite.ObjectWithFieldsConverter.processConfiguration (ObjectWithFieldsConverter.java:157)
    at org.codehaus.plexus.component.configurator.BasicComponentConfigurator.configureComponent (BasicComponentConfigurator.java:37)
    at org.apache.maven.plugin.internal.DefaultMavenPluginManager.populateMojoExecutionFields (DefaultMavenPluginManager.java:617)
    at org.apache.maven.plugin.internal.DefaultMavenPluginManager.getConfiguredMojo (DefaultMavenPluginManager.java:573)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:114)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:193)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:180)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:572)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1144)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:642)
    at java.lang.Thread.run (Thread.java:1583)
Caused by: java.lang.NullPointerException: Cannot invoke "java.util.Set.add(Object)" because "this.artifacts" is null
    at org.apache.maven.project.MavenProject.getArtifacts (MavenProject.java:683)
    at org.apache.maven.project.MavenProject.getArtifactMap (MavenProject.java:693)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.codehaus.plexus.util.introspection.ReflectionValueExtractor.getPropertyValue (ReflectionValueExtractor.java:301)
    at org.codehaus.plexus.util.introspection.ReflectionValueExtractor.evaluate (ReflectionValueExtractor.java:212)
    at org.codehaus.plexus.util.introspection.ReflectionValueExtractor.evaluate (ReflectionValueExtractor.java:145)
    at org.apache.maven.plugin.PluginParameterExpressionEvaluator.evaluate (PluginParameterExpressionEvaluator.java:218)
    at org.codehaus.plexus.component.configurator.converters.AbstractConfigurationConverter.fromExpression (AbstractConfigurationConverter.java:76)
    at org.codehaus.plexus.component.configurator.converters.AbstractConfigurationConverter.fromExpression (AbstractConfigurationConverter.java:60)
    at org.codehaus.plexus.component.configurator.converters.composite.MapConverter.fromConfiguration (MapConverter.java:55)
    at org.eclipse.sisu.plexus.CompositeBeanHelper.convertProperty (CompositeBeanHelper.java:260)
    at org.eclipse.sisu.plexus.CompositeBeanHelper.setProperty (CompositeBeanHelper.java:166)
    at org.codehaus.plexus.component.configurator.converters.composite.ObjectWithFieldsConverter.processConfiguration (ObjectWithFieldsConverter.java:157)
    at org.codehaus.plexus.component.configurator.BasicComponentConfigurator.configureComponent (BasicComponentConfigurator.java:37)
    at org.apache.maven.plugin.internal.DefaultMavenPluginManager.populateMojoExecutionFields (DefaultMavenPluginManager.java:617)
    at org.apache.maven.plugin.internal.DefaultMavenPluginManager.getConfiguredMojo (DefaultMavenPluginManager.java:573)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:114)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:193)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:180)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:572)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1144)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:642)
    at java.lang.Thread.run (Thread.java:1583)

[Pankraz76]

root cause:

Caused by: java.lang.NullPointerException: Cannot invoke "java.util.Set.add(Object)" because "this.artifacts" is null

[delanym]

Not the same result, but probably related: when I perform a partial build -pl :a, :b I get (consistently with certain projects and not at all with others)

[WARNING] *****************************************************************
[WARNING] * The org.apache.maven.plugins:maven-clean-plugin:3.4.0:clean   *
[WARNING] * mojo is already being executed on the project                 *
[WARNING] * com.package.b. *
[WARNING] * This mojo execution will be blocked until the mojo is done.   *
[WARNING] *****************************************************************
[WARNING] *****************************************************************
[WARNING] * The org.apache.maven.plugins:maven-clean-plugin:3.4.0:clean   *
[WARNING] * mojo is already being executed on the project                 *
[WARNING] * com.package.b. *
[WARNING] * This mojo execution will be blocked until the mojo is done.   *
[WARNING] *****************************************************************

where a warning is printed TWICE for each plugin goal of project b.

Then the build fails BOTH in compilation (missing symbols) and surefire tests (The forked VM terminated without properly saying goodbye).

[slawekjaranowski]

I see a changes which can impact on it:

• [MNG-8256] FilteredProjectDependencyGraph fix for non-transitive case #1724
• [MNG-8711] Fix concurrent cache access #2345
• [MNG-8396] Backport: add cache layer to the filtered dep graph #2393

NPE in code:

    public Set<Artifact> getArtifacts() {
        if (artifacts == null) {
            if (artifactFilter == null || resolvedArtifacts == null) {
                artifacts = new LinkedHashSet<>();
            } else {
                artifacts = new LinkedHashSet<>(resolvedArtifacts.size() * 2);
                for (Artifact artifact : resolvedArtifacts) {
                    if (artifactFilter.include(artifact)) {
                        artifacts.add(artifact); // <!- here artifacts is null
                    }
                }
            }
        }
        return artifacts;
    }

[cstamas]

@mnk given I cannot reproduce this locally (not enough project or just no luck), could you try to build locally and try to reproduce?

This one is "heavy handed" solution, so may be worth to observe perf penalty, if visible/measurable as well:
#2467

[mnk]

@cstamas , I tested with #2467 and it seems to solve the problem. I can reproduce 3 out of 3 times with 3.9.10, and got 0 failures out of 3 runs with this change. I did not notice any performance degradation with this change.

[Pankraz76]

immutability is the key design pattern, maven fails to implement all over.

might be beneficial to only fix the corresponding field, instead of a big bang refactoring, changing everything at once. @elharo had issued to review only one if without tests so this one not be simpler.

[cstamas]

@mnk Thanks for testing. Is the project that produces this failure public? Or could you by any chance create some sort of reproducer for this issue? I tried on several bigger projects, and no luck.

[cstamas]

@delanym please create new issue with full description and possibly reproducer. Also, at least the layout of the project would be needed. Your issue may but may not be related to this one.

[cstamas]

immutability is the key design pattern

@Pankraz76 I agree, but in case of Maven we have a thing called "legacy", and also a huge backward trail of compatibility promise. Just like today Maven 4 is capable to run Maven 3 plugins (plugins compiled against Maven 3 APIs), same stands for Maven 3 and plugins built against Maven 2 (from end of 2000s). Moreover, this class MavenProject is one of the most exposed, most used and most injected class in plugins. Due this, many of the "calls" existing out there for this class are not in Maven codebase at all, but in some plugins codebase out there. Hence, we need to be very cautious to not break anything.

In essence, this is the whole reason for doing Maven 4, as Maven 3 is "fully exposed" with it's own internals to plugins, and it is nearly impossible to fix these issues (or would take insanely much time and effort) without breaking anything out there in the wild that is directly interfacing to any of Maven 3 class.

This venerable class was added in 2004, as part of "initial commit":
https://github.com/apache/maven/blob/601320d28fe961c1d5f01de7af6273488ca0f7a9/maven-core/src/main/java/org/apache/maven/project/MavenProject.java 😄