Add incremental invalidation/resolution

[st0012]

Summary

Replace the full re-resolution strategy (clear_declarations + resolve everything from scratch) with incremental invalidation. Graph mutations (update/delete_document) now compute the minimal set of definitions, references, and ancestor chains that need re-resolution, and the resolver processes only that subset.

How it works

Graph mutations follow a three-step pipeline:

1. invalidate — Detaches old definitions from declarations. Identifies affected declarations (definition removed, new definition/reference touching existing names). Feeds them into invalidate_graph, a unified worklist that processes declarations and names.
2. remove_document_data — Removes old refs/defs/names/strings from maps. Cleans up empty declarations.
3. extend — Merges new LocalGraph data and queues new definitions/references for resolution.

Work items are accumulated as Unit (definitions, references, ancestors) and drained by the resolver. For the initial full index, this contains everything. For incremental updates, only the invalidated subset.

Reverse index: name_dependents

A single reverse index name_dependents: NameId → Vec<NameDependent> maps each name to the definitions and references that depend on it. Each dependent is stored under its own name_id and under the nesting/parent_scope of that name, so invalidation can trace from any scope level directly to affected refs/defs without an intermediate name-to-name layer.

Invalidation worklist

invalidate_graph processes two kinds of items:

• Declaration — Two modes: removal (no definitions left or orphaned owner) removes the declaration tree and unresolves all resolved names; ancestor-stale (still has definitions) clears ancestors/descendants and cascades to dependent names.
• Name — Two modes: structural cascade (nesting or parent_scope dependency broken) unresolves the name and detaches its refs/defs from the old declaration; ancestor-only (dependencies intact but ancestor context changed) unresolves references for re-evaluation.

Other changes

• without_resolution option — Skip accumulating pending work for tools that only need definitions.
• name_dependents populated during indexing — LocalGraph builds name_dependents entries in add_definition and add_constant_reference, then merged into the global graph during extend.
• Resolution refactored — resolve_all renamed to resolve, drains pending_work via Unit enum instead of scanning all names.

TODO: smarter ancestor invalidation

Currently, any definition change on a declaration enters ancestor-stale mode, which conservatively unresolves all nested references. A future optimization: compare old vs new definitions for ancestor-affecting fields (mixins, parent_class). If they haven't changed, skip ancestor invalidation entirely — only re-resolve the definition itself.

Compared to main

Correctness: identical — all declaration counts, definition counts, orphan rates, and linked/orphan breakdowns match exactly between main and the branch.

Performance (initial full index on 94,036 files):

Stage	main	branch	Delta
Listing	0.645s	0.701s	+0.056s (+8.7%)
Indexing	10.309s	10.261s	-0.048s (-0.5%)
Resolution	41.570s	25.275s	-16.295s (-39.2%)
Querying	0.647s	0.696s	+0.049s (+7.6%)
Total	53.171s	36.933s	-16.238s (-30.5%)

Memory:

	main	branch	Delta
Max RSS	3,909 MB	4,437 MB	+528 MB (+13.5%)

Resolution is 39% faster at the cost of 13.5% more memory from the reverse index and pending work accumulation. (Note: benchmarks predate some of the recent changes — numbers may have shifted slightly.)

[vinistock]

I'm still trying to reason about the algorithm, so not done reviewing yet.

[vinistock]

This PR is highly complex. I think the mechanism works, but it's hard to ship so many changes in one go. Here's what I think we should do to minimize the changes and ship with confidence:

• Move the rename of resolve_all to resolve to a separate PR
• Create a PR implementing unresolve_name and unresolve_reference that can be shipped separate from the algorithm
• Add the name_dependents hashmap of IdentityHashMap<NameId, IdentityHashSet<NameDependent>> using a NameDependent enum. Populate this map during indexing when creating names, so that the global graph only has to merge the work at the end

With this foundation, it will be significantly easier for reviewers to focus on the algorithm. What do you think?

[vinistock]

This is one of the reasons why I'm not a fan of the multiple hashmaps. We need to ensure that the data on the auxiliary maps is also consistent and the benefit is just avoiding tracing the graph from declaration -> definitions -> names. I'm not convinced tbh.

Also, to make sure we're making progress, I think you can probably ship a separate PR with just the unresolve_name and unresolve_reference methods separately.

[st0012]

I'll address all the feedback first and then see how we can split this PR.

[st0012]

First one: #627

[vinistock]

Why do we need these two maps? The idea of having an enum for name dependents is so that you can go NameId -> ReferenceId | DefinitionId -> NameId.

[st0012]

I merged the maps but I think the value should also include NameId. So it'd be NameId -> ReferenceId | DefinitionId | NameId.

I prototyped using reference/definition to look up name but it doesn't work well with parent_scope/nesting cases. For example:

class Baz
  include Foo
  CONST
end

The reference CONST creates a name with nesting=baz_name, but it doesn't create a member declaration under Baz. So Baz.members() is empty — there's no path from baz_name to the reference's name through the declaration/definitions. When Baz's ancestors change (e.g., a new prepend Bar is added), we need to re-evaluate CONST, but without the explicit Name(NameId) entry under baz_name, we can't discover it.

[vinistock]

there's no path from baz_name to the reference's name through the declaration/definitions

This is the only reason why we need name_dependents. In your example, this is what I would expect the hashmap to look like:

# name_dependents

{
  NameId(Baz) => Set[ReferenceId(Foo), ReferenceId(CONST)]
}

This allows the graph to remember which references and definitions will be potentially impacted by a name change. We then trace name_dependents and the rest of the graph to unresolve names.

In this case, if we had to unresolve all names due to a change to Baz, I would expect the algorithm to do something like this:

1. Baz changed. Loop through name dependents
2. Regardless whether the dependent is a reference or definition, get its name_id, pull the name from the graph and unresolve it
3. Now, unresolving the definition and reference may invalidate other things. Go back to 1. and invalidate the name_id for the reference/definition

This also involves invalidating ancestors, but you get the idea.

[vinistock]

We still need an answer for the ever growing memory if users don't call resolve.