Python: share the ty-types type registry across a project parse#7932
Merged
Conversation
ty's `--serve` session deduplicates type descriptors: each type is emitted in full only once, in the first `getTypes` response that references it, and later responses reference it by id only. `handle_parse_project` parses every file in a project through a single shared session, but each file built its own per-file type registry from only that file's response. A type first seen in an earlier file (e.g. `pydantic.BaseModel`) was therefore absent from later files' registries, so first-party classes in every file but the first lost their supertypes and `self` stopped resolving as a subclass. Accumulate every descriptor returned across a session into a cumulative `TyTypesClient.session_types` table (keyed by ty's session-stable ids), and have `PythonTypeMapping` back-fill any descriptor missing from a file's own response from that table. File-local descriptors take precedence; the table is reset when a new session starts so no state leaks across parses. This keeps ty's dedup performance benefit while restoring resolution in every file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
rewrite-python's whole-project parse path (handle_parse_project, used by the CLI'smod buildviarpc.parseProject) parses every.pyfile in a project through a single shared ty-types--servesession. ty's session deduplicates type descriptors: each type is emitted in full exactly once — in the firstgetTypesresponse that references it — and later responses reference it by id only.But the parser builds a fresh per-file type registry, populated solely from that file's
getTypesresponse. So any type first seen in an earlier file (e.g.pydantic.BaseModel) is absent from later files' registries, and supertypes pointing into it can't resolve. The net effect on a real multi-file repo: first-party classes in every file but the first lose their supertypes,selfno longer resolves as aBaseModelsubclass, and a type-aware recipe likeReplaceModelFieldsInstanceAccesssilently does nothing across most of the project.tytype attribution #7930 (the dependency-path work that made third-party supertypes resolvable in the first place). Single-filehandle_parsedoes not hit it, because it creates a fresh ty session per parse, so every file is "first".ty type ids are stable within a session (e.g. id 1 ==
BaseModelacross all files), which is what makes a session-scoped registry sound.Summary
TyTypesClientnow accumulates every descriptor returned across allgetTypescalls into a cumulative, session-scopedsession_typestable, keyed by ty's session-stable type ids. The table is reset when a new session starts (a fresh client, orinitializewith a different project root) so no state leaks across unrelated parses.PythonTypeMappingreferences that table and, in_build_index, back-fills any descriptor missing from the current file's own response (and seeds_class_literal_indexfrom the cumulative class literals). File-local descriptors take precedence, preserving the existing FQN-based dedup ofClassobjects.Alternatives considered: (a) a fresh ty session per file in
handle_parse_project— correct but re-initializes ty per file and loses the shared-session speedup; (b) a ty--serve"no-dedup" mode — least control and would regress payload size. The session-scoped registry retains the performance benefit and is fully under our control.Test plan
TestProjectParseSupertypeAcrossFilestotests/python/test_type_attribution.py, exercisinghandle_parse_projectexactly asmod builddoes. It uses two peer model files so the failure is order-independent (whichever file ty parses second loses its base), asserting thatselfresolves as apydantic.main.BaseModelsubclass in both files. Fails before this change (the second file drops the supertype), passes after.TestExternalSupertypeResolutionInParsePathpositive/negative tests still pass.tests/python/test_type_attribution.pygreen (130 passed).tests/python+tests/recipessuites green (1565 passed, 6 skipped).(The new test relies on
ty-typesanduvbeing available andpydanticnot being importable in the test interpreter; it skips otherwise by design.)