Skip to content

Fix dependency selector cache regression in DF collector#1941

Merged
cstamas merged 1 commit into
apache:masterfrom
gnodet:gnodet/fix-selector-cache-regression
Jun 29, 2026
Merged

Fix dependency selector cache regression in DF collector#1941
cstamas merged 1 commit into
apache:masterfrom
gnodet:gnodet/fix-selector-cache-regression

Conversation

@gnodet

@gnodet gnodet commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Summary

ScopeDependencySelector and OptionalDependencySelector (in impl.scope package) always create new instances in deriveChildSelector(), incrementing depth at every level. Since depth is part of equals()/hashCode(), every tree depth produces a unique selector, which makes the DataPool/GraphKey cache in the DF dependency collector miss on every lookup across depths.

This causes exponential node growth in the DF collector for large dependency trees. In a benchmark with a large multi-module project:

Configuration Node count
Resolver 1.9.27 (Maven 3.9.16), DF collector 1,054,704
Resolver 2.0.x (Maven 3.10.x), BF collector 1,915,842
Resolver 2.0.x (Maven 3.10.x), DF collector 9,893,981

The DF collector's 9.4x node increase is caused by cache misses cascading exponentially — each miss triggers full recursion, discovering more nodes that also miss the cache.

Root cause

The old selectors (resolver 1.x, in util.graph.selector package) returned this from deriveChildSelector() once their behavior stabilized:

  • ScopeDependencySelector: returned this once transitive=true (after depth 1)
  • OptionalDependencySelector: returned this once depth >= 2

The AndDependencySelector already optimizes for this pattern (line 119): when all child selectors return this (reference equality), the AndDependencySelector also returns this. This meant the entire composite selector was the same instance at all depths 2+, enabling cache hits in the GraphKey.

The new impl.scope selectors never return this — they always create new instances with depth + 1, breaking this optimization chain.

Fix

Both selectors now return this from deriveChildSelector() once depth >= applyFrom (the point after which their selectDependency() behavior no longer changes with depth). For ScopeDependencySelector, the only exception is depth == applyTo where behavior transitions from "filter by scope" to "accept all".

For the default Maven 3.10.x configuration (ScopeDependencySelector.legacy(null, ["test", "provided"])), this means the selector stabilizes at depth 2, matching the old resolver 1.x behavior.

Claude Code on behalf of Guillaume Nodet

ScopeDependencySelector and OptionalDependencySelector always create
new instances in deriveChildSelector(), even after their behavior has
stabilized. Since depth is part of equals/hashCode, every tree depth
gets a unique selector, which makes the GraphKey cache in the DF
collector miss on every lookup across depths. This causes exponential
node growth in large dependency trees.

The old selectors (resolver 1.x) returned 'this' once their state
stabilized (ScopeDependencySelector after depth 1, OptionalDependency
Selector after depth 2). The AndDependencySelector already optimizes
for this: when all child selectors return 'this', it also returns
'this' (reference equality check at line 119).

This fix restores that pattern: both selectors now return 'this' once
depth >= applyFrom (the point after which behavior no longer changes
with depth). For ScopeDependencySelector, the only exception is when
depth == applyTo (transition from filtering to accept-all).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@cstamas cstamas merged commit 46acf12 into apache:master Jun 29, 2026
14 checks passed
@github-actions github-actions Bot added this to the 2.0.20 milestone Jun 29, 2026
@github-actions

Copy link
Copy Markdown

@cstamas Please assign appropriate label to PR according to the type of change.

@cstamas cstamas added the enhancement New feature or request label Jun 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants