v2.5.0 "The Last Excuse": parallel indexing + 23x faster PageRank - #158
Merged
Conversation
🔴 Arbor PR Walk
Changed Files
🎯 Production Entry Points ReachedThis change propagates to these entry points (HTTP handlers, jobs, CLI commands):
✅ Before You Merge
🔍 Sensitive Path Check — REVIEW REQUIRED
📊 Analysis confidence: High · 1572 nodes · 6251ms
Arbor · View full report → · 6251ms · 1572 nodes · Deterministic PR blast-radius · Was this useful? 👍 👎 |
🌳 Arbor Impact ReportRisk Level: 🔴 Critical | Blast Radius: 64 nodes | Changed Symbols: 300 Changed Files
📊 Visual Impact Graphgraph TD
classDef changed fill:#ef4444,stroke:#333,stroke-width:2px,color:#fff;
classDef caller fill:#f59e0b,stroke:#333,stroke-width:1px,color:#fff;
class Why changed;
class Index changed;
class What changed;
class Install changed;
class Quickstart changed;
Impact Summary
Powered by Arbor v2.5.0 — graph-native code intelligence |
…nsport, new CLI commands)
Walk once to collect candidates, fan the cache-check/parse phase across all cores, and assemble results in walk order so graph construction stays deterministic. parse_file creates its own tree-sitter parser per call and sled handles concurrent access, so the hot path needs no locks. Measured (median of 3, warm cache, RAYON_NUM_THREADS=1 as baseline): Arbor 253ms -> 95ms (2.7x), tokio 2.7s -> 1.6s (1.7x).
Replace per-iteration get_callers() traversal and string-ID hash lookups with a one-pass adjacency build and dense Vec iteration. Add compute_centrality_warm (seeds from previous scores with an analytic rescale of the max-normalized stored values) and a 1e-9 convergence early-exit. Semantics preserved: Calls edges only, 10% test-caller weight, [0,1] normalization - covered by new unit tests. Measured: 149.8ms -> 6.6ms on a 10k-node graph (23x), old implementation run side-by-side as reference in the same Criterion session.
After a one-file patch the previous scores are already near the fixed point, so the recompute converges in a couple of rounds instead of the full 20-iteration budget.
Fan-in graph shaped like real code (hubs with callers, 25% test files) instead of a chain, so the numbers reflect production topology.
BENCHMARKS.md now leads with the measured real-repo table and publishes the VS Code-scale known limit (405s cold, 12.4k files / 366k nodes, assembly-dominated) instead of the old extrapolated projections.
Lead with measured numbers; collapse the v2.4.0 feature table into a details block. Ignore local release scratch docs.
Anandb71
force-pushed
the
feat/v2.5.0-perf
branch
from
July 15, 2026 07:51
d5a481f to
fb65d09
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR prepares Arbor v2.5.0 (“The Last Excuse”) by delivering major indexing/ranking performance improvements (parallel indexing + faster centrality/PageRank), wiring warm-start recomputation into the sync server, and updating release packaging/docs/versioning across the workspace.
Changes:
- Parallelize the watcher’s cache-check/parse phase with Rayon while keeping deterministic graph assembly order.
- Rewrite centrality computation to precompute call adjacency and iterate over dense vectors, adding warm-start + early convergence exit and new tests/benches.
- Bump versions to 2.5.0 and publish updated release notes/benchmarks/README and related docs.
Reviewed changes
Copilot reviewed 20 out of 22 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| README.md | Updates top-level release messaging and “What’s new” section for v2.5.0. |
| packaging/npm/package.json | Bumps npm wrapper package version to 2.5.0. |
| extensions/arbor-vscode/package.json | Bumps VS Code extension version to 2.5.0. |
| docs/RELEASE_NOTES_v2.5.0.md | Adds v2.5.0 release notes with benchmark tables and upgrade notes. |
| docs/BENCHMARKS.md | Refreshes benchmark documentation and adds measured v2.5.0 results. |
| crates/arbor-watcher/src/indexer.rs | Implements parallel parse/cache-check pipeline (Rayon) with ordered result collection. |
| crates/arbor-watcher/Cargo.toml | Bumps internal crate deps to 2.5.0 and adds Rayon dependency. |
| crates/arbor-server/src/sync_server.rs | Switches centrality recomputation to warm-started variant for patch/delete flows. |
| crates/arbor-server/Cargo.toml | Bumps internal crate deps to 2.5.0. |
| crates/arbor-mcp/Cargo.toml | Bumps internal crate deps to 2.5.0. |
| crates/arbor-gui/Cargo.toml | Bumps internal crate deps to 2.5.0. |
| crates/arbor-graph/src/ranking.rs | Centrality/PageRank rewrite + warm-start + early exit + new unit tests. |
| crates/arbor-graph/src/lib.rs | Re-exports compute_centrality_warm. |
| crates/arbor-graph/src/graph.rs | Adds centrality_map() accessor to support warm-starting. |
| crates/arbor-graph/Cargo.toml | Bumps arbor-core dependency to 2.5.0. |
| crates/arbor-graph/benches/graph_bench.rs | Adds new 10k-node centrality benches (cold + warm). |
| crates/arbor-cli/Cargo.toml | Bumps internal crate deps to 2.5.0. |
| CLAUDE.md | Updates local dev commands/docs (adds bench command; expands module notes). |
| CHANGELOG.md | Adds unreleased v2.5.0 entry describing performance improvements and benchmarks. |
| Cargo.toml | Bumps workspace version to 2.5.0 and adds Rayon to workspace deps. |
| Cargo.lock | Updates lockfile for 2.5.0 crate versions and Rayon addition. |
| .gitignore | Ignores local scratch docs pattern docs/_*.md. |
| "displayName": "Arbor — Graph Intelligence for Code", | ||
| "description": "Impact analysis, refactor prediction, and AI-aware code navigation powered by semantic dependency graphs", | ||
| "version": "2.4.0", | ||
| "version": "2.5.0", |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
v2.5.0 "The Last Excuse" — parallel indexing + 23x PageRank
First two items from the v2.5.0 roadmap (deferred from v2.4.0), plus full release packaging.
Performance (all measured, all reproducible)
Methodology: sequential baseline is the identical binary pinned to
RAYON_NUM_THREADS=1, median of 3, warm FS cache. The PageRank comparison ran the v2.4.0 implementation side-by-side as a reference function in the same Criterion run.Changes
arbor-watcher): the cache-check/parse phase fans out via rayon (parse_fileis stateless per call; sled handles concurrent access). Results collect in walk order, so graph construction stays deterministic.arbor-graph/ranking.rs): one-pass flat call-adjacency build + dense Vec iteration replaces per-iterationget_callers()and string-ID lookups. Semantics preserved (Calls edges only, 10% test-caller weight, [0,1] normalization) — covered by new unit tests.compute_centrality_warmseeds from previous scores with an analytic rescale (stored scores are max-normalized; naive warm-start converges no faster — the rescale recovers the fixed-point scale in one edge pass). Sync server re-index/delete paths now warm-start.compute_centrality_10k,compute_centrality_10k_warm.Verification
cargo test --workspace: 243 tests passing (including 3 new ranking tests: test-caller de-weighting, warm/cold fixed-point equivalence, Calls-edge-only contribution)cargo clippy --workspace --all-targets --all-features: cleancargo fmt --all: appliedKnown limit (published deliberately)
At VS Code scale (366k nodes) single-threaded graph assembly dominates end-to-end index time — parallel parsing can't save it. Documented in BENCHMARKS.md rather than hidden; first target for v2.6.0.