Store DependencyInsight's dependency graph as a DAG to prevent OOM on ubiquitous transitive deps

[jkschneider]

What's the problem?

DependencyInsight (Gradle/Maven) can OOM on a single repository when the target is a ubiquitous, deeply-transitive dependency such as com.fasterxml.jackson*. DependencyTreeWalker.walkRecursive enumerates every root→match path — it prunes cycles within the current path but has no DAG-level dedup — so a diamond-heavy graph expands combinatorially, and the per-match callback appends each path into a non-deduplicated DependencyGraph. One repo materialized ~37M DependencyGraph$DependencyNode and blew a ~16 GB heap.

• Full analysis in Dependency Insight OOMs on ubiquitous transitive deps (jackson): DependencyTreeWalker enumerates all root→match paths with no DAG dedup #8181.

The fix

Give the walk a single visited-set spanning the whole traversal (not just the current path). It both prevents cycles and dedups subtrees reachable through multiple paths, so a match reachable via k diamonds fires the callback once with a representative path instead of k times. Traversal goes from O(paths) (exponential) to O(nodes) (linear).

This also makes good on the walker's existing Javadoc — "Uses caching to avoid re-traversing duplicate dependencies" — which the implementation never actually honored.

The visited-set is per direct-dependency root (each walk(root, …) gets its own), so attribution is preserved: the SearchResult markers on the POM/build.gradle are unchanged (DFS reachability is identical). Only the DependenciesInUse / ExplainDependenciesInUse data tables change — count becomes the number of direct dependencies a match is pulled in through, and the dependency tree shows one representative path per root instead of the combinatorial expansion.

Benchmark

Reproducing the exact OOM structure — one matched dependency reachable from the root via 2^depth diamond paths — and running the pre-fix algorithm against the post-fix one on the identical graph:

Total traversal work — O(2ᵈ) → O(d):

depth	graph nodes	OLD node-visits	NEW node-visits
10	31	4,093	31
18	55	1,048,573	55
22	67	16,777,213	67
24	73	67,108,861	73

DependencyGraph built the way DependencyInsight does (depth 18 = 262,144 paths to one match):

OLD: size=262,144 appends, time=676 ms, retained ≈ 224 MB
NEW: size=1 append,        time=0 ms,   retained ≈ 0 MB

Extrapolated to the issue's ~depth-25 scale (~33M paths): OLD ≈ 28 GB, NEW stays flat — matching the reported 37M nodes / 16 GB heap in order of magnitude. For non-pathological graphs the walk visits each node once, so there's no slowdown.

Tests

• New DependencyTreeWalkerTest (network-free): a 20-deep diamond chain (2²⁰ ≈ 1M paths, 61 nodes) asserts the callback fires 61 times, with a fail-fast cap so a regression trips instead of OOMing; plus cycle-termination and path-order coverage.

• Gradle DependencyInsightTest expectations updated to the deduped values (spring-boot 10→3, jackson-core 11→2, one representative-path tree). Markers unchanged.

• Fixes Dependency Insight OOMs on ubiquitous transitive deps (jackson): DependencyTreeWalker enumerates all root→match paths with no DAG dedup #8181