Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PEP 604 Union (int | str) doesn't have __parameters__ #88656

Closed
Fidget-Spinner opened this issue Jun 22, 2021 · 19 comments
Closed

PEP 604 Union (int | str) doesn't have __parameters__ #88656

Fidget-Spinner opened this issue Jun 22, 2021 · 19 comments
Labels
3.10 3.11 type-feature

Comments

@Fidget-Spinner
Copy link
Member

@Fidget-Spinner Fidget-Spinner commented Jun 22, 2021

BPO 44490
Nosy @gvanrossum, @ambv, @serhiy-storchaka, @ilevkivskyi, @JelleZijlstra, @miss-islington, @brandtbucher, @uriyyo, @Fidget-Spinner, @jdevries3133, @ROpdebee
PRs
  • #26980
  • #27048
  • #27203
  • #27207
  • #27215
  • #27220
  • #27222
  • #27368
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-07-17.03:45:14.170>
    created_at = <Date 2021-06-22.13:29:59.812>
    labels = ['type-feature', '3.10', '3.11']
    title = "PEP 604 Union (int | str) doesn't have __parameters__"
    updated_at = <Date 2021-07-26.19:32:08.916>
    user = 'https://github.com/Fidget-Spinner'

    bugs.python.org fields:

    activity = <Date 2021-07-26.19:32:08.916>
    actor = 'lukasz.langa'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-07-17.03:45:14.170>
    closer = 'gvanrossum'
    components = []
    creation = <Date 2021-06-22.13:29:59.812>
    creator = 'kj'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 44490
    keywords = ['patch']
    message_count = 19.0
    messages = ['396329', '396346', '396351', '396709', '396710', '396755', '396756', '396758', '396759', '396760', '396767', '396770', '396895', '397051', '397686', '397728', '397801', '398241', '398251']
    nosy_count = 11.0
    nosy_names = ['gvanrossum', 'lukasz.langa', 'serhiy.storchaka', 'levkivskyi', 'JelleZijlstra', 'miss-islington', 'brandtbucher', 'uriyyo', 'kj', 'jack__d', 'ROpdebee']
    pr_nums = ['26980', '27048', '27203', '27207', '27215', '27220', '27222', '27368']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue44490'
    versions = ['Python 3.10', 'Python 3.11']

    @Fidget-Spinner
    Copy link
    Member Author

    @Fidget-Spinner Fidget-Spinner commented Jun 22, 2021

    Recently I noticed that the new PEP-604 Union type doesn't collect type variables:

    from typing import TypeVar
    T = TypeVar('T')

    (int | list[T]).__parameters__

    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    AttributeError: 'types.Union' object has no attribute '__parameters__'

    Whereas the typing.Union version has __parameters__. Is this behavior intentional?

    The downside to this is that things like this don't work:

    alias: TypeAlias = int | list[T]
    alias[str] # Error!

    @JelleZijlstra
    Copy link
    Member

    @JelleZijlstra JelleZijlstra commented Jun 22, 2021

    I agree that this is a bug. types.Union is also missing a getitem implementation.

    And typing.Union supports pickling while types.Union doesn't:

    >>> pickle.loads(pickle.dumps(int | str))
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: cannot pickle 'types.Union' object
    >>> pickle.loads(pickle.dumps(Union[int, str]))
    typing.Union[int, str]

    I don't have a use case for pickling types but someone might.

    @gvanrossum
    Copy link
    Member

    @gvanrossum gvanrossum commented Jun 22, 2021

    Let's first see whether the type (int | list[T]) is accepted by static checkers.

    IMO introspecting unions should be done by looking at __args__, nothing more.

    @uriyyo
    Copy link
    Member

    @uriyyo uriyyo commented Jun 29, 2021

    Should __getitem__ be implemented for types.Union?
    I can implement it if no one is working on it.

    P.S. mypy currently does not support it:

    Value of type "types.Union" is not indexable
    

    @Fidget-Spinner
    Copy link
    Member Author

    @Fidget-Spinner Fidget-Spinner commented Jun 29, 2021

    Yurii, thanks for the offer.

    We only need to implement __getitem__ if union supports TypeVars. Which
    means __parameters__ need to be implemented too (or at least a private
    internal implementation of it).

    I interpreted Guido's message above as to wait and see if static type
    checkers even accept things like int | list[T]. Maybe he meant that if that
    syntax isnt valid then theres no point for us to implement it? PEP-604
    leaves out how it deals with TypeVars, so adding this behavior may require
    us to update the PEP first (which will require approval from others).

    Anyways, there's no rush. We probably can't backport this so we have until
    the next minor release of Python (3.11 or later) to decide.

    @gvanrossum
    Copy link
    Member

    @gvanrossum gvanrossum commented Jun 29, 2021

    I intended for someone to write some test programs and report back here what mypy actually supports and what it doesn't.

    @jdevries3133
    Copy link
    Mannequin

    @jdevries3133 jdevries3133 mannequin commented Jun 29, 2021

    mypy does not support __parameters__:

    (venv) ➜  cpython git:(main) cat repro.py 
    from typing import TypeVar
    T = TypeVar('T')
    
    (int | list[T]).__parameters__
    (venv) ➜  cpython git:(main) mypy --version
    mypy 0.920+dev.cae5d3c8b5f14d0796914aa6a113473ca3ffc38e
    (venv) ➜  cpython git:(main) python --version
    Python 3.11.0a0
    (venv) ➜  cpython git:(main) mypy repro.py 
    repro.py:4: error: "types.Union" has no attribute "__parameters__"
    Found 1 error in 1 file (checked 1 source file)
    (venv) ➜  cpython git:(main)
    

    mypy also does not support __getitem__

    (venv) ➜  cpython git:(main) cat repro.py 
    from typing import TypeVar
    T = TypeVar('T')
    
    (int | list[T]).__getitem__
    (venv) ➜  cpython git:(main) mypy --version
    ./mypy 0.920+dev.cae5d3c8b5f14d0796914aa6a113473ca3ffc38e
    (venv) ➜  cpython git:(main) python --version
    Python 3.11.0a0
    (venv) ➜  cpython git:(main) mypy repro.py 
    repro.py:4: error: Value of type "types.Union" is not indexable
    Found 1 error in 1 file (checked 1 source file)
    (venv) ➜  cpython git:(main)
    

    @JelleZijlstra
    Copy link
    Member

    @JelleZijlstra JelleZijlstra commented Jun 29, 2021

    Mypy is definitely not going to support direct access to __parameters__; what Guido is referring to is whether usage of types.Union that would require __parameters__ at runtime is accepted by mypy.

    For example, this:

    from typing import TypeVar
    
    T = TypeVar("T")
    
    Alias = int | list[T]
    
    def f(x: Alias[str]) -> None:
        pass

    But this produces main.py:7: error: Variable "main.Alias" is not valid as a type: mypy doesn't even recognize | as a type yet.

    I'd rather not focus too much though on what mypy does; it is one of many type checkers by now and does not tend to be the quickest in adding support for new features.

    Pyright does handle the file above as I'd expect, for what it's worth.

    @gvanrossum
    Copy link
    Member

    @gvanrossum gvanrossum commented Jun 29, 2021

    So I guess to expand on Jelle's example, Alias[str] should return (int |
    list[str]), right? That makes sense since that's how Union works.

    @JelleZijlstra
    Copy link
    Member

    @JelleZijlstra JelleZijlstra commented Jun 29, 2021

    I'd also be OK with returning a types.GenericAlias(int | list[T], str), which might be simpler. It doesn't matter for static type checkers, and runtime type checkers can extract what they need anyway.

    @Fidget-Spinner
    Copy link
    Member Author

    @Fidget-Spinner Fidget-Spinner commented Jun 29, 2021

    I don't think we need the types.GenericAlias(int | list[T], str)
    workaround. GenericAlias already has code for extracting __parameters__ and
    implementing __getitem__. Someone would need to refactor and reuse them for
    Union too. That should be enough to trick GenericAlias to treat Union as
    another GenericAlias(I don't remember if it checks for __origin__ as well)
    and cover other edge cases as well.

    @yurii do you still plan to take this? If not, I'll start working on
    something later today.

    @uriyyo
    Copy link
    Member

    @uriyyo uriyyo commented Jun 30, 2021

    @ken I will pick up this issue, thanks for asking.

    @ROpdebee
    Copy link
    Mannequin

    @ROpdebee ROpdebee mannequin commented Jul 3, 2021

    It also lacks the __module__ attribute, causing it to be unusable in PEP-593 typing.Annotated types:

    from typing import Annotated
    x: Annotated[int | str, 'test']
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/Users/ruben/.pyenv/versions/3.10.0b3/lib/python3.10/typing.py", line 298, in inner
        return cached(*args, **kwds)
      File "/Users/ruben/.pyenv/versions/3.10.0b3/lib/python3.10/typing.py", line 1594, in __class_getitem__
        return _AnnotatedAlias(origin, metadata)
      File "/Users/ruben/.pyenv/versions/3.10.0b3/lib/python3.10/typing.py", line 1520, in __init__
        super().__init__(origin, origin)
      File "/Users/ruben/.pyenv/versions/3.10.0b3/lib/python3.10/typing.py", line 976, in __init__
        self.__module__ = origin.__module__
    AttributeError: 'types.Union' object has no attribute '__module__'. Did you mean: '__reduce__'?

    @gvanrossum
    Copy link
    Member

    @gvanrossum gvanrossum commented Jul 6, 2021

    New changeset c45fa1a by Yurii Karabas in branch 'main':
    bpo-44490: Add __parameters__ and __getitem__ to types.Union (GH-26980)
    c45fa1a

    @gvanrossum
    Copy link
    Member

    @gvanrossum gvanrossum commented Jul 17, 2021

    New changeset bf89ff9 by Yurii Karabas in branch 'main':
    bpo-44490: Improve typing module compatibility with types.Union (GH-27048)
    bf89ff9

    @gvanrossum gvanrossum added the type-feature label Jul 17, 2021
    @gvanrossum gvanrossum added the type-feature label Jul 17, 2021
    @serhiy-storchaka
    Copy link
    Member

    @serhiy-storchaka serhiy-storchaka commented Jul 17, 2021

    New changeset 2d055ce by Serhiy Storchaka in branch '3.10':
    [3.10] bpo-44490: Add __parameters__ and __getitem__ to types.Union (GH-26980) (GH-27207)
    2d055ce

    @gvanrossum
    Copy link
    Member

    @gvanrossum gvanrossum commented Jul 19, 2021

    New changeset a272164 by Ken Jin in branch '3.10':
    bpo-44490: Improve typing module compatibility with types.Union (GH-27048) (bpo-27222)
    a272164

    @ambv
    Copy link
    Contributor

    @ambv ambv commented Jul 26, 2021

    New changeset 6c1b57d by Yurii Karabas in branch 'main':
    bpo-44490: Add 'Whats New' docs regarding types.Union changes (GH-27215)
    6c1b57d

    @ambv
    Copy link
    Contributor

    @ambv ambv commented Jul 26, 2021

    New changeset 4a5457d by Miss Islington (bot) in branch '3.10':
    bpo-44490: Add 'Whats New' docs regarding types.Union changes (GH-27215) (GH-27368)
    4a5457d

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.10 3.11 type-feature
    Projects
    None yet
    Development

    No branches or pull requests

    6 participants