Static input enforcement on `TagList[TagifiedNode]` via `self`-typed overloads

[schloerke]

Context

Following #105/#106 (PR that made Tag/TagList generic in TagNodeT) and #112 / commit a9ec156 (which added a runtime TypeError at the TagList.tagify() boundary when a child's .tagify() returned un-tagified content), there is a remaining static-typing gap:

t: TagList[TagifiedNode] = some_taglist.tagify()
t.append(div("hi"))            # silently allowed by pyright today
t.get_html_string()            # works (div is already tagified — happens to be fine)
t.append(SomeTagifiable())     # also silently allowed at .append() time
t.get_html_string()            # raises RuntimeError at render

The append of an un-tagified Tagifiable to a TagList[TagifiedNode] is statically silent. It only surfaces at render time, or — if .tagify() is re-run — at the boundary check from #112.

Proposal

Keep TagNodeT = TypeVar(\"TagNodeT\", bound=TagNode, default=TagNode) as-is (do not narrow to a constrained TypeVar(..., TagNode, TagifiedNode)), and add Self-typed @overload signatures to TagList's input methods:

TagifiedChild = Union[
    TagifiedNode,
    \"TagList[TagifiedNode]\",
    float,
    None,
    Sequence[\"TagifiedChild\"],
]  # non-generic alias parallel to TagChild — avoids the recursive-generic
   # pyright leak that motivated keeping TagChild non-generic in #105.

class TagList(UserList[TagNodeT]):
    @overload
    def append(self: \"TagList[TagifiedNode]\", item: TagifiedChild, *args: TagifiedChild) -> None: ...
    @overload
    def append(self: \"TagList[TagNode]\", item: TagChild, *args: TagChild) -> None: ...
    def append(self, item, *args): ...

Apply to: __init__, append, extend, insert, __add__, __radd__ (and consider __setitem__ for completeness).

Decisions reached in design discussion

• Keep bound=TagNode (not constrained). Survey of 9 Posit Python repos (py-shiny, py-shinywidgets, py-shinyswatch, shinychat, chatlas, querychat, py-shinylive, py-shiny-templates, py-shiny-output-binding-example) found 0 in-the-wild parameterizations of TagList[T] or Tag[T] — so a constrained TypeVar would have no practical cost today, but keeping bound preserves theoretical room for TagList[MySpecificTag] at zero extra design cost.
• TagList only — not Tag. Tag(Generic[TagNodeT]) has children: TagList[TagNodeT], so overloads on TagList cover tag.children.append(...) transitively. Direct construction of Tag[TagifiedNode](...) only happens internally via casts, never via user code.
• Static-only enforcement. No new runtime TypeError on .append(). The existing render-time RuntimeError and the feat: TagList.tagify() raises TypeError on un-tagified content #112 tagify-boundary TypeError remain the runtime safety net. Users who # type: ignore the static error are knowingly opting out.

Why this wasn't done in #105

The note at _core.py:187-199 explains: the previous attempt made TagChild itself a generic TypeAliasType with a recursive Sequence[\"TagChild[TagNodeT]\"] arm, which leaked Sequence[Unknown] into every Tag function signature in downstream strict-mode pyright (2500+ reports in Shiny CI). A non-generic parallel alias TagifiedChild sidesteps that — it's a separate union, not a parameterization of TagChild. Worth verifying against a fresh Shiny strict-mode CI run before merging.

Open question — subclass vs. alias and the variance trap

TagifiedTagList is currently a type alias:

TagifiedTagList = TypeAliasType(\"TagifiedTagList\", \"TagList[TagifiedNode]\")

We discussed replacing it with a real subclass: class TagifiedTagList(TagList[TagifiedNode]): .... Possible, but TagList is invariant (mutable container), so TagList[TagifiedNode] is not assignable to TagList[TagNode]. With the subclass approach, every function annotated def f(t: TagList) (= TagList[TagNode]) would reject the output of .tagify().

The alias should have the same variance issue in principle — but in practice it may not surface because most consumers use bare TagList, which pyright resolves via the default. Needs spot-check of real consumers of .tagify()'s return before deciding whether subclass would be a net improvement, or whether the alias is correct as-is.

Subclassing the union types themselves (TagifiedNode, Tagified) is not possible — you can't subclass a Union.

Scope checklist

• Define non-generic TagifiedChild alias next to TagChild.
• Add Self-typed @overloads to TagList.{__init__, append, extend, insert, __add__, __radd__}. Evaluate __setitem__.
• Update tests/test_types.py::test_TagifiedTagList_append_accepts_Tagifiable — that test's premise (referenced in _core.py:198) inverts under this proposal: appending a Tagifiable to a TagList[TagifiedNode] should now be a static error. Decide whether to flip the test to use assert_type / reveal_type patterns or a pyright-on-fixture test.
• Verify no Sequence[Unknown] regression in Shiny strict-mode CI from the new TagifiedChild alias.
• Investigate the variance question above and decide subclass vs. alias for TagifiedTagList (potentially out-of-scope as a follow-up).

Related

• Make Tag/TagList generic in their child type so .tagify() can statically guarantee a fully-tagified tree #105 — original design issue (generic Tag/TagList)
• feat: tagified-tag type system (#105) #106 — original PR
• feat: TagList.tagify() raises TypeError on un-tagified content #112 / a9ec156 — runtime TypeError at tagify boundary
• Update remaining Posit-owned repos for htmltools 0.7.0 Tagified contract #114 — downstream Tagified contract updates

[schloerke]

Findings from prototyping the Self-overload proposal

I prototyped the proposal in this issue and discovered the central premise — Self-typed @overload signatures to dispatch on receiver parameterization — does not actually reject Tagifiable arguments. Documenting here so the next attempt doesn't re-tread the same ground.

Reproducer: the Self-overload pattern as proposed

Adding the proposed TagifiedChild alias plus the proposed Self-typed append overload to TagList:

TagifiedChild = Union[
    TagifiedNode,
    "TagList[TagifiedNode]",
    float,
    None,
    Sequence["TagifiedChild"],
]

class TagList(UserList[TagNodeT]):
    @overload
    def append(
        self: "TagList[TagifiedNode]",
        item: TagifiedChild,
        *args: TagifiedChild,
    ) -> None: ...
    @overload
    def append(
        self: "TagList[TagNode]",
        item: TagChild,
        *args: TagChild,
    ) -> None: ...
    def append(self, item: TagChild, *args: TagChild) -> None:
        self.extend([item, *args])

Then call:

class _SomeTagifiable:
    def tagify(self) -> Tagified:
        return "x"

tl: TagifiedTagList = TagList("hi").tagify()
tl.append(_SomeTagifiable())  # expected: static error.  actual: clean.

Pyright reports 0 errors. reveal_type(tl.append) shows pyright recognizes both overload arms:

Type of "tl.append" is "Overload[
  (item: Tag[...] | ... | Sequence[TagifiedChild] | None, ...) -> None,
  (item: Tagifiable | ... | Sequence[TagChild] | None, ...) -> None
]"

Pyright tries the narrow arm first (rejects _SomeTagifiable because it's not in TagifiedChild), then the wide arm (accepts _SomeTagifiable because Tagifiable is in TagChild). One matching arm => no error.

Why it doesn't reject

The self: TagList[TagNode] second arm is supposed to be the "default" path — selected when the receiver is bare TagList. But pyright treats TagList[TagifiedNode] as also compatible with self: TagList[TagNode]. This is despite UserList[T] being nominally invariant.

The same loose-invariance behavior shows up elsewhere — e.g., def f(t: TagList): pass; f(some_taglist.tagify()) is accepted today without error. So TagList[TagifiedNode] is being treated as a subtype of TagList[TagNode] (TagList with the default), which makes both overload arms applicable to a TagifiedTagList receiver.

Subclass approach: works statically, and the variance trap doesn't surface

Replacing the alias with a real subclass that overrides append with a single narrowed signature:

class TagifiedTagList(TagList[TagifiedNode]):
    def append(self, item: TagifiedChild, *args: TagifiedChild) -> None:  # type: ignore[override]
        super().append(item, *args)

ttl: TagifiedTagList = TagifiedTagList("hi")
ttl.append(_SomeTagifiable())

Pyright reports reportArgumentType cleanly — the override fully replaces the inherited overloads, so there's no wide fallback arm to slip through.

Bonus finding: the variance trap warned about in the original "Open question" section does not surface in practice — def takes_taglist(t: TagList): pass; takes_taglist(ttl) is accepted by pyright (same loose-invariance behavior as above). So a subclass TagifiedTagList doesn't break callers that take bare TagList.

__init__ overload is also unsalvageable

For completeness: I also tried adding a Self-typed overload pair to TagList.__init__. Pyright's Self-on-__init__ does not narrow at construction sites, AND adding the overload cascades errors into _core.py itself (because __add__/__radd__/Tag.__init__ construct TagList(...) internally and pyright then can't pick the right arm). So __init__ is not a path either.

Implications

• The Self-overload approach is a dead end for static enforcement. It produces overloads pyright recognizes but cannot use to reject Tagifiable.
• The subclass approach is viable and doesn't hit the variance trap, but it's a structural design change (.tagify() would need to instantiate a different class than TagList), and is the original "Open question" of this issue.
• Runtime safety remains via the feat: TagList.tagify() raises TypeError on un-tagified content #112 tagify-boundary TypeError and the render-time RuntimeError in get_html_string — those are unchanged.

Closing this issue as won't fix without breaking change. If we want static enforcement later, the path is the subclass redesign and that should be its own scoped issue with proper migration consideration for .tagify()'s return type.