Deadlock in AbstractRequestCache when resolving parent POMs with empty local repo

[gnodet]

Description

Maven 4 hangs indefinitely during builds when the local repository is empty (e.g. -Dmaven.repo.local=/fresh/path). The build freezes at maven-remote-resources-plugin:process while resolving parent POMs of transitive dependencies.

The same build completes in seconds with a populated local repository.

Steps to Reproduce

git clone --depth 1 https://github.com/apache/httpcomponents-stylecheck.git
cd httpcomponents-stylecheck
# Works fine (cached):
mvn -B clean package -DskipTests
# Hangs forever (empty repo):
mvn -B clean package -DskipTests -Dmaven.repo.local=/tmp/fresh-repo

The build hangs at:

[INFO] --- remote-resources:3.0.0:process (process-resource-bundles) @ hc-stylecheck ---
[INFO] Preparing remote bundle org.apache:apache-jar-resource-bundle:1.4
[INFO] Copying 3 resources from 1 bundle.

Thread Dump Analysis

The main thread is stuck in AbstractRequestCache.lambda$requests$0 (line 95) calling Object.wait() on a HashMap that it already holds the lock on:

"main" in Object.wait()
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait0(Native Method)
    - waiting on <0x000000070dadf9a8> (a java.util.HashMap)
    at org.apache.maven.impl.cache.AbstractRequestCache.lambda$requests$0(AbstractRequestCache.java:95)
    - locked <0x000000070dadf9a8> (a java.util.HashMap)
    at org.apache.maven.impl.cache.CachingSupplier.apply(CachingSupplier.java:50)
    at org.apache.maven.impl.cache.AbstractRequestCache.requests(AbstractRequestCache.java:144)
    at org.apache.maven.impl.AbstractSession.requests(AbstractSession.java:177)
    at org.apache.maven.impl.DefaultArtifactResolver.doResolve(DefaultArtifactResolver.java:106)
    at org.apache.maven.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:65)
    at org.apache.maven.impl.AbstractSession.resolveArtifact(AbstractSession.java:689)
    at org.apache.maven.impl.resolver.DefaultModelResolver.getPath(DefaultModelResolver.java:181)
    at org.apache.maven.impl.resolver.DefaultModelResolver.doResolveModel(DefaultModelResolver.java:155)
    at org.apache.maven.impl.resolver.DefaultModelResolver.resolveModel(DefaultModelResolver.java:116)
    at org.apache.maven.impl.model.DefaultModelBuilder...resolveParent(DefaultModelBuilder.java:1101)
    ...
    at org.apache.maven.plugin.resources.remote.ProcessRemoteResourcesMojo.getProjects(ProcessRemoteResourcesMojo.java:665)

Meanwhile, all 15 worker threads (5× BfDependencyCollector, 5× BasicRepositoryConnector, 5× resolver) are idle in LinkedBlockingQueue.take(), waiting for work that is never submitted.

Root Cause

This appears to be a missed-signal deadlock in AbstractRequestCache / CachingSupplier:

1. The main thread enters CachingSupplier.apply() to resolve an artifact
2. Inside, it calls Object.wait() on a HashMap, expecting another thread to resolve the artifact and call notify()
3. But the resolution work is never dispatched to the worker threads — they sit idle in their thread pool queues
4. The main thread waits forever

When the local repo is populated, the CachingSupplier returns the cached result immediately without entering the wait path. The deadlock only triggers when actual remote resolution is needed.

Impact

This affects any project using maven-remote-resources-plugin (very common — it's bound by default in the Apache parent POM) when built with a fresh local repository. This is the standard CI scenario where each build starts with an empty local repo.

In the maven4-testing suite (testing ~966 Apache projects against Maven 4), this deadlock caused 21 of 57 failures — all were 1-hour timeouts on projects that build fine on Maven 3 in under 6 minutes.

Environment

• Maven: 4.1.0-SNAPSHOT (commit 595458c)
• Java: OpenJDK 25.0.3+9-LTS
• OS: Linux
• Plugin: maven-remote-resources-plugin 3.0.0

Full thread dump attached: all 27 threads, showing the main thread's full stack trace and all idle worker threads.

[gnodet]

Detailed Root-Cause Analysis

Looking at the code and thread dump more carefully, here is the full call chain in the deadlock:

Stack Trace Breakdown

The main thread holds locks on three CachingSupplier instances, nested:

CS_inner locked  <0x000000070dae4860>  ← CachingSupplier for the ResolverRequest (innermost)
CS_outer locked  <0x000000070dadd910>  ← CachingSupplier for ArtifactResolverRequest
CS_model locked  <0x000000070dad7c40>  ← CachingSupplier for model resolution

The call chain is:

1. ProcessRemoteResourcesMojo.getProjects() → Velocity template triggers project building
2. DefaultModelBuilder.resolveParent() → needs external parent POM
3. DefaultModelResolver.resolveModel() → session.request() (singular) → doCache() → CS_model.apply() → runs lambda
4. Lambda → AbstractSession.resolveArtifact() → ArtifactResolver.resolve() → session.request() (singular) → doCache() → CS_outer.apply() → runs doResolve()
5. doResolve() → session.requests() (plural) → creates individualSupplier that waits on nonCachedResults HashMap
6. For each ResolverRequest: doCache(req, individualSupplier) → creates CS_inner(individualSupplier) → added to nonCached
7. Batch execution: batchSupplier.apply(nonCached) → calls resolveArtifacts() → returns results
8. Results put into nonCachedResults, notifyAll() called
9. Result collection: CS_inner.apply(req) → individualSupplier.apply(req) → nonCachedResults.containsKey(req) → false → wait()
10. DEADLOCK: main thread waits forever, all worker threads idle

The Critical Question

At step 9, nonCachedResults.containsKey(req) returns false even though the batch completed at step 7. This means either:

A) The batch supplier returned fewer results than nonCached.size() — in DefaultArtifactResolver.doResolve():

List<ResolverResult> res = new ArrayList<>(resolverResults.size());
for (int i = 0; i < resolverResults.size(); i++) {
    res.add(new ResolverResult(list.get(i), resolverResults.get(i)));
}

This iterates resolverResults.size(), not list.size(). If resolveArtifacts() returns fewer results than requests (e.g., due to deduplication), some requests won't be put into nonCachedResults.

B) doCache() returned a CachingSupplier from a previous requests() invocation (via the session-scoped cache). That older CachingSupplier wraps an individualSupplier from a different call, which waits on a different nonCachedResults HashMap that will never be notified again. This can happen if:

• A previous requests() call created the CachingSupplier
• The CachingSupplier's soft reference was GC'd and recreated incorrectly
• Or computeIfAbsent on the RefConcurrentMap has a race condition with eviction

C) An uncaught exception (not MavenExecutionException) was thrown and handled in a way that makes nonCachedResults empty but still reaches the result collection loop.

Suggested Fix Direction

The requests() method should not use wait()/notifyAll() on a local HashMap shared between the individualSupplier closure and the batch execution. Instead, consider:

1. Direct approach: After batch execution, directly set each CachingSupplier.value from the batch results, bypassing individualSupplier entirely for the result collection phase.

2. Defensive fix: In the result collection loop (line 144), check if the result is already in nonCachedResults before calling cs.apply(). If it is, use the value directly instead of going through the CachingSupplier.

3. Timeout-based safety: Add a timeout to nonCachedResults.wait() with a clear error message, so builds fail fast instead of hanging forever.

Full Thread Dump

The full thread dump is in the issue description. Key observations:

• Only the main thread is active — all 15 worker threads (BfDependencyCollector, BasicRepositoryConnector, resolver) are idle in their thread pool queues
• The main thread has been stuck for 60 seconds (elapsed=60.30s, cpu=3900.83ms)
• No other thread is doing any work related to artifact resolution

[cschanhniem]

Deadlocks in dependency resolution during POM loading are nasty. For build-tool maintainers dealing with complex dependency graphs, AGENTOWNERS addresses a different kind of dependency — governing which agents may propose changes to core build logic and dependency resolution code paths.