Support for __builtins__.pyi?

[hoodmane]

Feature

In #12860 @erictraut said:

We decided to support this in pyright through the use of a specially-named builtins.pyi stub file. If a stub with this name is found within the root directory of a project, pyright treats any symbols declared within that stub as though they are appended to the bottom of builtins.pyi.

and @AlexWaygood closed the issue saying:

if anybody wants to request a mypy equivalent of the more limited feature Eric's introduced over at pyright, they can open another issue to do so.

I would like to request a mypy version of this feature, so that custom Python distributions that provide extra builtins can ship type definitions that are understood by mypy. In specific, I am interested in this because PyScript is discussing adding a builtin called display which is like a web aware print and they want to make it a builtin.
pyscript/pyscript#1687

It's worth pointing out that typescript also has a version of this feature with declare global. Though the JavaScript ecosystem also has a lot of tools that pollute the "builtins"-level scope so they have a much stronger need for this sort of thing.

[ferdnyc]

@hoodmane

Suggestion: Modify the title to,

Support for __builtins__.pyi?

...as that's the actual name of the file that pyright supports, and if mypy ever does come to support this feature, it makes sense to use the same filename rather than fracturing things.

[ferdnyc]

That being said,

I would like to request a mypy version of this feature, so that custom Python distributions that provide extra builtins can ship type definitions that are understood by mypy.

...AIUI not even pyright's current feature would support that use case. They look for a __builtins__.pyi placed locally to where pyright is being run: either in the project root directory or the configured extra typing stubs path. But a Python runtime would have no way of providing a __builtins__.pyi "centrally" so that it's picked up by all projects / all pyright invocations.

To achieve what you're looking for, IIUC both pyright and mypy would require feature requests for new (in mypy) or expanded (in pyright's case) __builtins__.pyi functionality, if it's going to be usable by runtimes as well as by endusers.

[JelleZijlstra]

I could see a more general feature that would cover this use case: a way to provide the type checker with an "overlay" over an existing stub that adds additional features. This could be useful not only for your use case, but also e.g. for patching over an incorrect stub provided by a library. Some of the pytype folks have been talking about such a feature, but not sure they'll have time to push it forward.

[ikonst]

Also useful for monkey-patching.

[ferdnyc]

@ikonst

It is, but... that's also been one of the arguments against supporting it. If you read @erictraut's #12860 (comment) in the issue that spawned this one... well, I'll quote it here (up to the point where @hoodmane's quote starts above):

Dynamically modifying the namespace of other modules goes against the whole notion of type checking and type safety. This is especially true for builtins. If you care about code maintainability, robustness, and module composability, it is something I would highly discourage.

That said, there are legitimate uses for declaring new (non-standard) symbols in the builtins namespace. In particular, there are alternate Python distros for specialized use cases and environments that include additional builtins symbols.

[ferdnyc]

(Such a legit usage is exactly the inspiration for @hoodmane's request. Which, come to think of it, makes it a bit weird if pyright really doesn't support a centralized __builtins__.pyi.)