RFC: a resolved-reference export for Python

[lolpack]

Feature request

Who is asking

@mollydoo from this issue: #4683 . They maintain a tool that builds a queryable map of a codebase — what calls what, which model attribute a screen reads, which guard sits on which endpoint — so that AI coding agents can navigate a project instead of grepping it.

Two properties of that tool shape this request:

• The map is rebuilt on every run, not stored. So the cost of a scan is the thing that constrains the tool. Anything that makes a full pass over the tree happen twice is felt directly.
• The map has to be right at the level of individual declarations, not names. An agent that is told "this screen reads apply" has learned nothing if eleven classes declare apply.

The user story

As a tool author consuming Python type information, I want a single export, produced by one pass over a source tree, that tells me for every name occurrence which declaration it resolves to and what type the surrounding expression has, so that I can attribute an attribute access or a call to the exact class or function that declares it rather than to a name shared by many declarations.

Why a name-keyed index is not enough

Everything below is something a text or name-keyed scan gets wrong, and the reason the request is for resolved data rather than more data.

Question the map must answer	What a name-keyed scan produces	What resolution is needed for
Which class declares the apply that this call reaches?	All classes declaring apply, merged into one node	Attributing the call to one declaration
Which attribute does this screen actually read?	Every declaration of that attribute name, everywhere	Distinguishing User.name from Product.name
What contains this reference?	A file and a line	Building a call graph — an edge needs both endpoints to be definitions
Is this self.handler the base class's or the override's?	One fused node	Following an override chain

What the export should contain

Three kinds of fact, over the same tree, from the same run. Each is stated as a question the output has to answer, not as a record layout.

1. Declarations, each with a stable identity

For every module, class, function, method, parameter, and module- or class-level variable in the tree: where it is declared, and an identity that distinguishes it from every other declaration in the tree — including same-named declarations in sibling classes and shadowed names in nested scopes.

The identity has to be reconstructible by a consumer that is looking at a reference elsewhere in the tree, so that a reference and its declaration can be joined without a second resolution pass.

2. References, each carrying its enclosing definition

For every occurrence of a name that resolves to one of those declarations: where the occurrence is, which declaration it resolves to, whether it reads or writes, and — this is the part that is usually missing — which declaration encloses the occurrence.

The enclosing definition is what turns a reference list into a graph. Without it I can see that Widget.apply is referenced 40 times; with it I can see that it is referenced by checkout.submit and admin.retry, which is the edge an agent needs.

3. Expression-level types

For every expression, the type Pyrefly inferred for it, in a form where a type that names a class can be joined back to that class's declaration in part 1.

This is the layer that lets a receiver be attributed. Given w.apply(...), knowing that w has type widgets.Widget is what makes the access resolvable to Widget.apply rather than to the set of all apply declarations.

---

Worked example

Input

Two files, in a directory checked as one tree. The example is built to contain exactly the two cases a name-keyed scan cannot handle: a method name declared by more than one class, and an attribute name declared by more than one class.

widgets.py

class Widget:
    label: str

    def apply(self, n: int) -> int:
        return n + 1


class Gadget:
    label: str

    def apply(self, n: int) -> int:
        return n * 2

app.py

from widgets import Gadget
from widgets import Widget


def go(w: Widget, g: Gadget) -> int:
    return w.apply(1) + g.apply(2)


def describe(w: Widget) -> str:
    return w.label

Output the tool needs

Shown below as an illustrative rendering — the shape is not the request. What matters is that every claim in the "Answers" column is derivable from the export without a second analysis pass and without any name-based guessing by the consumer.

Declarations

Identity	Kind	Location
widgets.Widget	class	widgets.py:1
widgets.Widget.label	attribute	widgets.py:2
widgets.Widget.apply	method	widgets.py:4
widgets.Widget.apply.n	parameter	widgets.py:4
widgets.Gadget	class	widgets.py:8
widgets.Gadget.label	attribute	widgets.py:9
widgets.Gadget.apply	method	widgets.py:11
app.go	function	app.py:5
app.go.w	parameter	app.py:5
app.describe	function	app.py:9

References

Location	Resolves to	Enclosed by	Kind
app.py:1 Gadget	widgets.Gadget	app (module)	import
app.py:5 Widget	widgets.Widget	app.go	read
app.py:6 w	app.go.w	app.go	read
app.py:6 apply (first)	widgets.Widget.apply	app.go	read/call
app.py:6 apply (second)	widgets.Gadget.apply	app.go	read/call
app.py:10 label	widgets.Widget.label	app.describe	read

Expression types

Location	Expression	Type
app.py:6	w	widgets.Widget
app.py:6	w.apply	(n: int) -> int bound to widgets.Widget
app.py:6	w.apply(1)	int
app.py:6	g.apply(2)	int
app.py:10	w.label	str

The three claims that justify the request

Everything else in the example is table stakes. These are the claims that only a resolved export supports:

1. The two apply occurrences on line 6 of app.py are different edges. One reaches widgets.Widget.apply, the other widgets.Gadget.apply. A name-keyed index reports two references to "apply" and the map fuses two unrelated methods into one node.
2. app.py:10's label is Widget.label, not Gadget.label. This follows from the expression type of the receiver. Attribute names are the most-shared identifiers in a typical codebase, and this is where a name-keyed map is least reliable.
3. Both apply calls are enclosed by app.go. That is what produces the call-graph edges app.go → widgets.Widget.apply and app.go → widgets.Gadget.apply. Without the enclosing definition, the references are attributable to a file but not to a caller.

---

Acceptance criteria

Stated as behaviour of the export, not of any particular option:

1. One pass, one export. Producing declarations, references, and expression types over a tree does not require running the checker more than once over that tree. The second pass is currently the single most expensive step in my scan.
2. Declaration identities are joinable. A reference and an expression type both name declarations in a form that joins to the declaration records with no name-based heuristics on my side.
3. Same-named declarations stay distinct. Two classes declaring the same method or attribute name produce two identities, and every reference picks exactly one of them.
4. Every reference carries its enclosing definition. Including references at module level, inside comprehensions, inside nested functions, and in decorator position — the enclosing definition is whatever definition lexically contains the occurrence, or the module.
5. Expression types name classes joinably. When an inferred type is or contains a class defined in the tree, that class is identified the same way it is in the declaration records.
6. Unresolvable is explicit, not absent. A reference the checker could not resolve, or an expression whose type is unknown, is stated as such rather than silently omitted. A consumer must be able to tell "no edge here" from "the analysis did not cover this".
7. Partial output is distinguishable from complete output. If a run fails part way, what it leaves behind must not read as a successful export over a smaller codebase. This is the failure mode that actually bit us; see below.
8. Third-party and stdlib references are attributed, not dropped. A call into a dependency should resolve to an identity for that dependency's declaration even when the dependency's source is not part of the checked tree, so the map can show the boundary rather than ending at it.

Non-goals

• Not asking for a new query interface, server, or daemon. A file that a consumer reads after a run is enough.
• Not asking for incremental or watch-mode output.
• Not asking Pyrefly to model dynamic behaviour it does not already infer. Where the checker does not know, criterion 6 covers it.
• Not asking for any particular serialisation, and not asking to adopt a specific existing format.

Prior art

TypeScript consumers have SCIP for exactly this shape of data: qualified declarations, references that carry their enclosing definition, and enough type information to attribute a receiver. As far as I could find, Python has no published equivalent.

Pyrefly already computes all three layers internally. The gap is not analysis, it is that there is no single consumable export of it, so external tooling either reconstructs it from partial outputs or does without.

Motivating incident

Today the three layers reach me through two separate reporting paths, and asking for both in one run crashes: the run aborts with a panic, and the output directory of the report that had already started is left holding a subdirectory of per-file data with none of the index files that belong beside it. That directory is indistinguishable from a complete export unless a consumer happens to check for the sidecar files, which is worse than the crash itself.

Filed separately as a bug, and noted here only as motivation — the workaround is to run the checker twice over the same tree, which is what criterion 1 is asking to remove. A fix to the crash alone would make the two paths coexist; a single resolved export would make the question of how they coexist moot, and is worth considerably more to me.

Open questions for the discussion

• Position convention. Line/column, or byte offsets? Consumers that re-read source files want offsets; consumers that display want lines. Either is workable, but it should be one of them, stated.
• Identity for declarations that have no source. Builtins, C extensions, and stubs need identities under criterion 8, but they have no declaration site in the tree. Synthesised identity, or the stub's location?
• Expression coverage. Every expression, or a defined subset? Every subexpression is a large volume of data for a big tree. A stated subset that includes at minimum every attribute-access receiver and every call target would satisfy this use case.
• Volume. For a large tree the expression-type layer dominates the output size. Is there appetite for the export to be streamable, so a consumer can process it without holding it all in memory?