Fix: Make DataType.type return self#7708
Merged
Merged
Conversation
VaggelisD
force-pushed
the
fix/share-type-in-deepcopy
branch
from
6985873 to
c60f6ab
Compare
Contributor
SQLGlot Integration Test ResultsComparing:
Overallmain: 113231 total, 112182 passed (pass rate: 99.1%) sqlglot:fix/share-type-in-deepcopy: 113231 total, 112182 passed (pass rate: 99.1%) Transitions: ✅ 70 test(s) passed |
VaggelisD
force-pushed
the
fix/share-type-in-deepcopy
branch
from
c60f6ab to
d48ccbf
Compare
VaggelisD
changed the title
_type by reference in Expr.__deepcopy__DataType.type return self
Collaborator
|
@VaggelisD there's a reported |
georgesittas
reviewed
Jun 4, 2026
annotate_types stored a deep clone of each DataType node in the node's own _type slot via `self._set_type(e, e.copy())`. The .copy() existed to avoid a self-cycle that would break recursive consumers of _type (serde.dump, etc). But Expr.__deepcopy__ also recursively walked _type, so every later e.copy() of an ancestor dragged the shadow type along. The work compounded across nested STRUCT levels and turned annotate into a >100s operation on wide, deeply nested STRUCT casts. Conceptually, asking "what type is this DataType?" should answer "itself". Special-case DataType in Expression.type the same way Cast already is, via an is_data_type ClassVar: - Expression.type returns self when self.is_data_type is True. - DataType (and all its subclasses) set is_data_type = True. - The DataType annotator becomes a no-op (no copy needed; no _type to set). - serde.dump skips the TYPE field when node.type is node, since storing a self-reference is meaningless and was the original reason the annotator needed the .copy(). - _to_s uses node.is_data_type so DataType subclasses (IntervalSpan, PseudoType, ObjectIdentifier) are skipped uniformly during repr. Behavior change to call out: on a DataType `dt`, `dt.type is dt` is now true (previously it returned a clone, equal but not identical). Equality semantics are unchanged.
VaggelisD
force-pushed
the
fix/share-type-in-deepcopy
branch
from
d48ccbf to
bf139a9
Compare
georgesittas
approved these changes
Jun 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Copying expressions after
annotate_typeshas run can be extremely slow e.g a deeply nested 150-fieldSTRUCTcan take upwarads of 100 seconds to copy, long enough to look like a hang. Most expressions are fine; the cost only shows up once the annotated type is large and nested.Two innocuous pieces compound:
The
DataTypeannotator stores a clone of each node in its own_type. As far as I can tell,.copy()is in place so that walkinge._typedoes not run into infinite recursion (e.g. inserde.dump).Expr.__deepcopy__recursively deep-clones_typeAfter (1), every
DataTypecarries a near-clone of itself in_type. After (2), copying a node walks both the node and that clone.The clone has the same nested
DataTypechildren, each with their own_typeclone, so the work doubles at every nesting level. Width W, depth D -> onecopy()walks ~2^D · W nodes, and the annotator makes ~W * D of them.PS: The memo additions on
commentsand_metaare a separate correctness issue. Those lines already calleddeepcopy(...)without forwardingmemowhich silently breaks cycle detection and shared-reference preservation in the stdlibdeepcopyprotocol.