Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate the typing.io and typing.re pseudo-modules #82472

Closed
dgelessus mannequin opened this issue Sep 27, 2019 · 28 comments
Closed

Deprecate the typing.io and typing.re pseudo-modules #82472

dgelessus mannequin opened this issue Sep 27, 2019 · 28 comments
Labels
3.11 docs stdlib type-bug

Comments

@dgelessus
Copy link
Mannequin

@dgelessus dgelessus mannequin commented Sep 27, 2019

BPO 38291
Nosy @gvanrossum, @srittau, @ambv, @serhiy-storchaka, @ilevkivskyi, @JelleZijlstra, @miss-islington, @tirkarthi, @dgelessus, @Fidget-Spinner
PRs
  • #26113
  • #26719
  • #26720
  • #26721
  • #26811
  • #27039
  • #27312
  • #27872
  • #27907
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-07-24.04:12:34.707>
    created_at = <Date 2019-09-27.11:48:37.318>
    labels = ['3.11', 'type-bug', 'library', 'docs']
    title = 'Deprecate the typing.io and typing.re pseudo-modules'
    updated_at = <Date 2021-08-23.17:32:39.463>
    user = 'https://github.com/dgelessus'

    bugs.python.org fields:

    activity = <Date 2021-08-23.17:32:39.463>
    actor = 'miss-islington'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-07-24.04:12:34.707>
    closer = 'kj'
    components = ['Documentation', 'Library (Lib)']
    creation = <Date 2019-09-27.11:48:37.318>
    creator = 'dgelessus'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 38291
    keywords = ['patch']
    message_count = 28.0
    messages = ['353356', '353400', '392730', '393611', '393613', '395794', '395800', '395803', '395805', '396147', '396183', '396217', '396219', '396317', '396322', '396324', '396968', '396975', '397001', '397012', '397013', '397016', '397018', '397020', '397042', '398082', '400151', '400152']
    nosy_count = 11.0
    nosy_names = ['gvanrossum', 'srittau', 'docs@python', 'lukasz.langa', 'serhiy.storchaka', 'levkivskyi', 'JelleZijlstra', 'miss-islington', 'xtreak', 'dgelessus', 'kj']
    pr_nums = ['26113', '26719', '26720', '26721', '26811', '27039', '27312', '27872', '27907']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue38291'
    versions = ['Python 3.11']

    @dgelessus
    Copy link
    Mannequin Author

    @dgelessus dgelessus mannequin commented Sep 27, 2019

    According to bpo-35089 (and the GitHub issues linked there), the typing.io and typing.re modules should no longer be used. Starting with Python 3.6, the typing documentation no longer mentions the typing.io and typing.re modules, and instead documents their contents as part of the main typing module.

    However, the typing module at runtime still supports typing.io and typing.re as before. Since these modules are not meant to be used anymore and are no longer documented, I would expect at least a DeprecationWarning when using them at runtime.

    The documentation on this could also be a bit clearer. As long as the modules are still supported at runtime, I would expect the documentation to mention that they still exist, but shouldn't be used anymore. In its current state, the documentation is confusing when coming from Python 3.5 (where typing.io and typing.re were the only documented way to access IO, Pattern, etc., but accessing typing.IO and typing.Pattern directly also works at runtime).

    @dgelessus dgelessus mannequin assigned docspython Sep 27, 2019
    @dgelessus dgelessus mannequin added docs stdlib type-bug labels Sep 27, 2019
    @ilevkivskyi
    Copy link
    Contributor

    @ilevkivskyi ilevkivskyi commented Sep 27, 2019

    Guido, what is your final opinion on this?

    @JelleZijlstra
    Copy link
    Member

    @JelleZijlstra JelleZijlstra commented May 2, 2021

    The documentation (https://docs.python.org/3.10/library/typing.html#typing.BinaryIO) now mentions the typing.io and typing.re namespaces.

    I would prefer that they be deprecated and eventually completely removed. They don't work with most static type checkers. https://bugs.python.org/issue42001 makes the same points.

    @JelleZijlstra JelleZijlstra changed the title Unclear status of the typing.io and typing.re pseudo-modules in docs and runtime Deprecate the typing.io and typing.re pseudo-modules May 2, 2021
    @gvanrossum
    Copy link
    Member

    @gvanrossum gvanrossum commented May 13, 2021

    I agree, these namespaces were a mistake. Let's start deprecating them (and remove their mention from the docs again).

    @gvanrossum gvanrossum removed the 3.10 label May 13, 2021
    @srittau
    Copy link
    Mannequin

    @srittau srittau mannequin commented May 13, 2021

    I opened a PR to remove their mention from the docs for now. I can look into how to add a deprecation warning to a module if no one else beats me to it.

    @srittau
    Copy link
    Mannequin

    @srittau srittau mannequin commented Jun 14, 2021

    And I opened a second PR (for Python 3.11 only) to issue a deprecation warning when typing.io or typing.re gets imported.

    @gvanrossum
    Copy link
    Member

    @gvanrossum gvanrossum commented Jun 14, 2021

    New changeset 8a76683 by Sebastian Rittau in branch 'main':
    bpo-38291: Remove mention of typing.io and typing.re again (GH-26113)
    8a76683

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Jun 14, 2021

    New changeset fc310cb by Miss Islington (bot) in branch '3.10':
    bpo-38291: Remove mention of typing.io and typing.re again (GH-26113)
    fc310cb

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Jun 14, 2021

    New changeset 7f02195 by Miss Islington (bot) in branch '3.9':
    bpo-38291: Remove mention of typing.io and typing.re again (GH-26113)
    7f02195

    @gvanrossum
    Copy link
    Member

    @gvanrossum gvanrossum commented Jun 19, 2021

    New changeset 09eb817 by Sebastian Rittau in branch 'main':
    bpo-38291: DeprecationWarning when importing typing.{io,re} (bpo-26719)
    09eb817

    @Fidget-Spinner
    Copy link
    Member

    @Fidget-Spinner Fidget-Spinner commented Jun 20, 2021

    Hello Sebastian, the tests are failing on the Azure pipelines buildbot running appx tests:

    https://dev.azure.com/Python/cpython/_build/results?buildId=82827&view=logs&j=0fcf9c9b-89fc-526f-8708-363e467e119e&t=fa5ef4ee-3911-591e-4444-19482ab189b7

    I suspect it's the same problem as this issue:
    https://bugs.python.org/issue44451

    Sending a PR shortly to test if that fixes it.

    @srittau
    Copy link
    Mannequin

    @srittau srittau mannequin commented Jun 21, 2021

    Thank you for fixing this, Ken, the PR looks good to me.

    Overall it looks as if the the Azure pipeline should be fixed, though. It's not ideal that the tests pass while running the PR, but not in other situations. Also, the warnings suppression is not really obvious.

    @Fidget-Spinner
    Copy link
    Member

    @Fidget-Spinner Fidget-Spinner commented Jun 21, 2021

    Overall it looks as if the the Azure pipeline should be fixed, though. It's not ideal that the tests pass while running the PR, but not in other situations.

    I agree. This specific test only seems to run after a commit is made. I don't see a way to run this test on a PR, even the test-with-buildbots test doesn't seem to trigger it.

    Also, the warnings suppression is not really obvious.

    Yup, it's hidden away in the docs here, so I was rather lost too: https://docs.python.org/3/library/warnings.html#testing-warnings

    Anyways, I'm not actually sure that the PR will fix it. AFAICS, the only way to test it is to commit it and let the buildbot run ;). I can't reproduce it locally.

    @Fidget-Spinner
    Copy link
    Member

    @Fidget-Spinner Fidget-Spinner commented Jun 22, 2021

    That specific buildbot is now green. Thanks everyone :)!
    https://dev.azure.com/Python/cpython/_build/results?buildId=82931&view=results

    I'm closing the issue now as it seems there isn't anything left to do.

    @srittau
    Copy link
    Mannequin

    @srittau srittau mannequin commented Jun 22, 2021

    Ken: We still need to remove these modules in Python 3.12+. Should we open a separate issue, reopen this one, or just handle it after the Python 3.11 branch has been created?

    @Fidget-Spinner
    Copy link
    Member

    @Fidget-Spinner Fidget-Spinner commented Jun 22, 2021

    @sebastian,

    Woops, thanks for the reminder. I think both options are fine. But I'm leaning towards creating a new issue. The title of the current issue is "Deprecate the typing.io and typing.re pseudo-modules" not "Remove the typing.io and typing.re pseudo-modules" after all ;-).

    Jokes aside, the actual removal of a large chunk of code is usually quite contentious. And usually a lot of discussion happens when the time comes to remove it. So IMO having a separate issue for that when the time comes is better.

    @tirkarthi
    Copy link
    Member

    @tirkarthi tirkarthi commented Jul 5, 2021

    This change seems to emit deprecation warnings in test_pydoc.

    PYTHONWARNINGS=always ./python.exe -Wall -m test test_pydoc
    0:00:00 load avg: 2.57 Run tests sequentially
    0:00:00 load avg: 2.57 [1/1] test_pydoc
    /Users/kasingar/stuff/python/cpython/Lib/pydoc.py:1344: DeprecationWarning: typing.io is deprecated, import directly from typing instead. typing.io will be removed in Python 3.12.
      if not cls.__name__.startswith("_") and cls.__module__ == "builtins"),
    /Users/kasingar/stuff/python/cpython/Lib/pydoc.py:1344: DeprecationWarning: typing.re is deprecated, import directly from typing instead. typing.re will be removed in Python 3.12.
      if not cls.__name__.startswith("_") and cls.__module__ == "builtins"),
    test_pydoc passed in 31.6 sec

    == Tests result: SUCCESS ==

    1 test OK.

    Total duration: 31.7 sec
    Tests result: SUCCESS

    @Fidget-Spinner
    Copy link
    Member

    @Fidget-Spinner Fidget-Spinner commented Jul 5, 2021

    This change seems to emit deprecation warnings in test_pydoc.

    test_typing emits them too as it tests importing things from __all__.

    @gvanrossum
    Copy link
    Member

    @gvanrossum gvanrossum commented Jul 5, 2021

    Can that be fixed by changing test_pydoc?

    @srittau
    Copy link
    Mannequin

    @srittau srittau mannequin commented Jul 5, 2021

    We could easily do what Ken did in PR-26811, and add "warnings.filterwarnings("default", category=DeprecationWarning)" to the test, but I would like to understand what is going on first. It seems the warnings are generated by test_builtin_with_more_than_four_children, which doesn't operate on typing directly. Here is a repro, without tests:

    srittau@blitz:~/Projekte/cpython (typing-io *)$ ./python -Wall
    Python 3.11.0a0 (heads/main:17f94e2888, Jul  5 2021, 19:19:04) [GCC 8.3.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import pydoc
    >>> import typing
    >>> doc = pydoc.TextDoc()
    >>> x = doc.docclass(object)
    /home/srittau/Projekte/cpython/Lib/pydoc.py:1344: DeprecationWarning: typing.io is deprecated, import directly from typing instead. typing.io will be removed in Python 3.12.
      if not cls.__name__.startswith("_") and cls.__module__ == "builtins"),
    /home/srittau/Projekte/cpython/Lib/pydoc.py:1344: DeprecationWarning: typing.re is deprecated, import directly from typing instead. typing.re will be removed in Python 3.12.
      if not cls.__name__.startswith("_") and cls.__module__ == "builtins"),
    >>> 

    Same warning when calling "help(object)". This is not great. I assume that it is somehow related to pydoc.docclass() iterating over the the items of typing, but only if typing was imported. Please not that "io" and "re" are not part of typing.__all__.

    I will investigate more.

    @srittau
    Copy link
    Mannequin

    @srittau srittau mannequin commented Jul 5, 2021

    pr-27039 suppresses warnings when accessing typing.{re,io}.__module__, which should fix this warning.

    @gvanrossum
    Copy link
    Member

    @gvanrossum gvanrossum commented Jul 5, 2021

    Why not just change "typing.io" to "typing"?

    @srittau
    Copy link
    Mannequin

    @srittau srittau mannequin commented Jul 5, 2021

    typing.io is not referenced in either the tests or the implementation for pydoc. What happens is basically: typing.io is a class, so its derived from object. When formatting the docstring for object, the formatter queries the __module__ attribute of all sub-classes of object, including typing.io, so that it can include the sub-classes in its output:

    --------------

    Help on class object in module builtins:

    class object
    | The base class of the class hierarchy.
    |
    | When called, it accepts no arguments and returns a new featureless
    | instance that has no instance attributes and cannot be given any.
    |
    | Built-in subclasses:
    | anext_awaitable
    | async_generator
    | async_generator_asend
    | async_generator_athrow
    | ... and 88 other subclasses
    |
    [...]

    --------------

    Before the PR this failed, as accessing any typing.io.__module__ printed the warning.

    @gvanrossum
    Copy link
    Member

    @gvanrossum gvanrossum commented Jul 5, 2021

    Thanks for the explanation. Let's reopen the issue for now.

    @gvanrossum gvanrossum reopened this Jul 5, 2021
    @gvanrossum
    Copy link
    Member

    @gvanrossum gvanrossum commented Jul 6, 2021

    New changeset 8b849ea by Sebastian Rittau in branch 'main':
    bpo-38291: Fix a spurious warning when using help(object) (bpo-27039)
    8b849ea

    @serhiy-storchaka
    Copy link
    Member

    @serhiy-storchaka serhiy-storchaka commented Jul 23, 2021

    Currently deprecation warnings are emitted in test_typing:

    $ ./python -m test test_typing
    0:00:00 load avg: 3.38 Run tests sequentially
    0:00:00 load avg: 3.38 [1/1] test_typing
    /home/serhiy/py/cpython/Lib/test/test_typing.py:4657: DeprecationWarning: typing.io is deprecated, import directly from typing instead. typing.io will be removed in Python 3.12.
      k not in typing.io.__all__ and
    /home/serhiy/py/cpython/Lib/test/test_typing.py:4658: DeprecationWarning: typing.re is deprecated, import directly from typing instead. typing.re will be removed in Python 3.12.
      k not in typing.re.__all__ and

    == Tests result: SUCCESS ==

    1 test OK.

    Total duration: 564 ms
    Tests result: SUCCESS

    @ambv
    Copy link
    Contributor

    @ambv ambv commented Aug 23, 2021

    New changeset 1a995b0 by Ken Jin in branch 'main':
    bpo-38291: Update 3.10 WhatsNew with typing.{io|re} DeprecationWarning (GH-27872)
    1a995b0

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Aug 23, 2021

    New changeset 27f2f62 by Miss Islington (bot) in branch '3.10':
    bpo-38291: Update 3.10 WhatsNew with typing.{io|re} DeprecationWarning (GH-27872)
    27f2f62

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.11 docs stdlib type-bug
    Projects
    None yet
    Development

    No branches or pull requests

    8 participants