Deprecate the typing.io and typing.re pseudo-modules

[dgelessus]

BPO	38291
Nosy	@gvanrossum, @srittau, @ambv, @serhiy-storchaka, @ilevkivskyi, @JelleZijlstra, @miss-islington, @tirkarthi, @dgelessus, @Fidget-Spinner
PRs	bpo-38291: Remove mention of typing.io and typing.re again #26113 bpo-38291: DeprecationWarning when importing typing.{io,re} #26719 [3.10] bpo-38291: Remove mention of typing.io and typing.re again (GH-26113) #26720 [3.9] bpo-38291: Remove mention of typing.io and typing.re again (GH-26113) #26721 bpo-38291: Reset DeprecationWarning filters in test_typing io and re #26811 bpo-38291: Fix a spurious warning when using help(object) #27039 bpo-38291: Fix a deprecation warning in typing tests #27312 bpo-38291: Update 3.10 WhatsNew with typing.{io|re} DeprecationWarning #27872 [3.10] bpo-38291: Update 3.10 WhatsNew with typing.{io|re} DeprecationWarning (GH-27872) #27907

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-07-24.04:12:34.707>
created_at = <Date 2019-09-27.11:48:37.318>
labels = ['3.11', 'type-bug', 'library', 'docs']
title = 'Deprecate the typing.io and typing.re pseudo-modules'
updated_at = <Date 2021-08-23.17:32:39.463>
user = 'https://github.com/dgelessus'

bugs.python.org fields:

activity = <Date 2021-08-23.17:32:39.463>
actor = 'miss-islington'
assignee = 'none'
closed = True
closed_date = <Date 2021-07-24.04:12:34.707>
closer = 'kj'
components = ['Documentation', 'Library (Lib)']
creation = <Date 2019-09-27.11:48:37.318>
creator = 'dgelessus'
dependencies = []
files = []
hgrepos = []
issue_num = 38291
keywords = ['patch']
message_count = 28.0
messages = ['353356', '353400', '392730', '393611', '393613', '395794', '395800', '395803', '395805', '396147', '396183', '396217', '396219', '396317', '396322', '396324', '396968', '396975', '397001', '397012', '397013', '397016', '397018', '397020', '397042', '398082', '400151', '400152']
nosy_count = 11.0
nosy_names = ['gvanrossum', 'srittau', 'docs@python', 'lukasz.langa', 'serhiy.storchaka', 'levkivskyi', 'JelleZijlstra', 'miss-islington', 'xtreak', 'dgelessus', 'kj']
pr_nums = ['26113', '26719', '26720', '26721', '26811', '27039', '27312', '27872', '27907']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue38291'
versions = ['Python 3.11']

[dgelessus]

According to bpo-35089 (and the GitHub issues linked there), the typing.io and typing.re modules should no longer be used. Starting with Python 3.6, the typing documentation no longer mentions the typing.io and typing.re modules, and instead documents their contents as part of the main typing module.

However, the typing module at runtime still supports typing.io and typing.re as before. Since these modules are not meant to be used anymore and are no longer documented, I would expect at least a DeprecationWarning when using them at runtime.

The documentation on this could also be a bit clearer. As long as the modules are still supported at runtime, I would expect the documentation to mention that they still exist, but shouldn't be used anymore. In its current state, the documentation is confusing when coming from Python 3.5 (where typing.io and typing.re were the only documented way to access IO, Pattern, etc., but accessing typing.IO and typing.Pattern directly also works at runtime).

[ilevkivskyi]

Guido, what is your final opinion on this?

[JelleZijlstra]

The documentation (https://docs.python.org/3.10/library/typing.html#typing.BinaryIO) now mentions the typing.io and typing.re namespaces.

I would prefer that they be deprecated and eventually completely removed. They don't work with most static type checkers. https://bugs.python.org/issue42001 makes the same points.

[gvanrossum]

I agree, these namespaces were a mistake. Let's start deprecating them (and remove their mention from the docs again).

[srittau]

I opened a PR to remove their mention from the docs for now. I can look into how to add a deprecation warning to a module if no one else beats me to it.

[srittau]

And I opened a second PR (for Python 3.11 only) to issue a deprecation warning when typing.io or typing.re gets imported.

[gvanrossum]

New changeset 8a76683 by Sebastian Rittau in branch 'main':
bpo-38291: Remove mention of typing.io and typing.re again (GH-26113)
8a76683

[miss-islington]

New changeset fc310cb by Miss Islington (bot) in branch '3.10':
bpo-38291: Remove mention of typing.io and typing.re again (GH-26113)
fc310cb

[miss-islington]

New changeset 7f02195 by Miss Islington (bot) in branch '3.9':
bpo-38291: Remove mention of typing.io and typing.re again (GH-26113)
7f02195

[gvanrossum]

New changeset 09eb817 by Sebastian Rittau in branch 'main':
bpo-38291: DeprecationWarning when importing typing.{io,re} (bpo-26719)
09eb817

[Fidget-Spinner]

Hello Sebastian, the tests are failing on the Azure pipelines buildbot running appx tests:

https://dev.azure.com/Python/cpython/_build/results?buildId=82827&view=logs&j=0fcf9c9b-89fc-526f-8708-363e467e119e&t=fa5ef4ee-3911-591e-4444-19482ab189b7

I suspect it's the same problem as this issue:
https://bugs.python.org/issue44451

Sending a PR shortly to test if that fixes it.

[srittau]

Thank you for fixing this, Ken, the PR looks good to me.

Overall it looks as if the the Azure pipeline should be fixed, though. It's not ideal that the tests pass while running the PR, but not in other situations. Also, the warnings suppression is not really obvious.

[Fidget-Spinner]

Overall it looks as if the the Azure pipeline should be fixed, though. It's not ideal that the tests pass while running the PR, but not in other situations.

I agree. This specific test only seems to run after a commit is made. I don't see a way to run this test on a PR, even the test-with-buildbots test doesn't seem to trigger it.

Also, the warnings suppression is not really obvious.

Yup, it's hidden away in the docs here, so I was rather lost too: https://docs.python.org/3/library/warnings.html#testing-warnings

Anyways, I'm not actually sure that the PR will fix it. AFAICS, the only way to test it is to commit it and let the buildbot run ;). I can't reproduce it locally.

[Fidget-Spinner]

That specific buildbot is now green. Thanks everyone :)!
https://dev.azure.com/Python/cpython/_build/results?buildId=82931&view=results

I'm closing the issue now as it seems there isn't anything left to do.

[srittau]

Ken: We still need to remove these modules in Python 3.12+. Should we open a separate issue, reopen this one, or just handle it after the Python 3.11 branch has been created?

[Fidget-Spinner]

@sebastian,

Woops, thanks for the reminder. I think both options are fine. But I'm leaning towards creating a new issue. The title of the current issue is "Deprecate the typing.io and typing.re pseudo-modules" not "Remove the typing.io and typing.re pseudo-modules" after all ;-).

Jokes aside, the actual removal of a large chunk of code is usually quite contentious. And usually a lot of discussion happens when the time comes to remove it. So IMO having a separate issue for that when the time comes is better.

[tirkarthi]

This change seems to emit deprecation warnings in test_pydoc.

PYTHONWARNINGS=always ./python.exe -Wall -m test test_pydoc
0:00:00 load avg: 2.57 Run tests sequentially
0:00:00 load avg: 2.57 [1/1] test_pydoc
/Users/kasingar/stuff/python/cpython/Lib/pydoc.py:1344: DeprecationWarning: typing.io is deprecated, import directly from typing instead. typing.io will be removed in Python 3.12.
  if not cls.__name__.startswith("_") and cls.__module__ == "builtins"),
/Users/kasingar/stuff/python/cpython/Lib/pydoc.py:1344: DeprecationWarning: typing.re is deprecated, import directly from typing instead. typing.re will be removed in Python 3.12.
  if not cls.__name__.startswith("_") and cls.__module__ == "builtins"),
test_pydoc passed in 31.6 sec

== Tests result: SUCCESS ==

1 test OK.

Total duration: 31.7 sec
Tests result: SUCCESS

[Fidget-Spinner]

This change seems to emit deprecation warnings in test_pydoc.

test_typing emits them too as it tests importing things from __all__.

[gvanrossum]

Can that be fixed by changing test_pydoc?

[srittau]

We could easily do what Ken did in PR-26811, and add "warnings.filterwarnings("default", category=DeprecationWarning)" to the test, but I would like to understand what is going on first. It seems the warnings are generated by test_builtin_with_more_than_four_children, which doesn't operate on typing directly. Here is a repro, without tests:

srittau@blitz:~/Projekte/cpython (typing-io *)$ ./python -Wall
Python 3.11.0a0 (heads/main:17f94e2888, Jul  5 2021, 19:19:04) [GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pydoc
>>> import typing
>>> doc = pydoc.TextDoc()
>>> x = doc.docclass(object)
/home/srittau/Projekte/cpython/Lib/pydoc.py:1344: DeprecationWarning: typing.io is deprecated, import directly from typing instead. typing.io will be removed in Python 3.12.
  if not cls.__name__.startswith("_") and cls.__module__ == "builtins"),
/home/srittau/Projekte/cpython/Lib/pydoc.py:1344: DeprecationWarning: typing.re is deprecated, import directly from typing instead. typing.re will be removed in Python 3.12.
  if not cls.__name__.startswith("_") and cls.__module__ == "builtins"),
>>> 

Same warning when calling "help(object)". This is not great. I assume that it is somehow related to pydoc.docclass() iterating over the the items of typing, but only if typing was imported. Please not that "io" and "re" are not part of typing.__all__.

I will investigate more.

[srittau]

pr-27039 suppresses warnings when accessing typing.{re,io}.__module__, which should fix this warning.

[gvanrossum]

Why not just change "typing.io" to "typing"?

[srittau]

typing.io is not referenced in either the tests or the implementation for pydoc. What happens is basically: typing.io is a class, so its derived from object. When formatting the docstring for object, the formatter queries the __module__ attribute of all sub-classes of object, including typing.io, so that it can include the sub-classes in its output:

--------------

Help on class object in module builtins:

class object
| The base class of the class hierarchy.
|
| When called, it accepts no arguments and returns a new featureless
| instance that has no instance attributes and cannot be given any.
|
| Built-in subclasses:
| anext_awaitable
| async_generator
| async_generator_asend
| async_generator_athrow
| ... and 88 other subclasses
|
[...]

--------------

Before the PR this failed, as accessing any typing.io.__module__ printed the warning.

[gvanrossum]

Thanks for the explanation. Let's reopen the issue for now.

[gvanrossum]

New changeset 8b849ea by Sebastian Rittau in branch 'main':
bpo-38291: Fix a spurious warning when using help(object) (bpo-27039)
8b849ea

[serhiy-storchaka]

Currently deprecation warnings are emitted in test_typing:

$ ./python -m test test_typing
0:00:00 load avg: 3.38 Run tests sequentially
0:00:00 load avg: 3.38 [1/1] test_typing
/home/serhiy/py/cpython/Lib/test/test_typing.py:4657: DeprecationWarning: typing.io is deprecated, import directly from typing instead. typing.io will be removed in Python 3.12.
  k not in typing.io.__all__ and
/home/serhiy/py/cpython/Lib/test/test_typing.py:4658: DeprecationWarning: typing.re is deprecated, import directly from typing instead. typing.re will be removed in Python 3.12.
  k not in typing.re.__all__ and

== Tests result: SUCCESS ==

1 test OK.

Total duration: 564 ms
Tests result: SUCCESS

[ambv]

New changeset 1a995b0 by Ken Jin in branch 'main':
bpo-38291: Update 3.10 WhatsNew with typing.{io|re} DeprecationWarning (GH-27872)
1a995b0

[miss-islington]

New changeset 27f2f62 by Miss Islington (bot) in branch '3.10':
bpo-38291: Update 3.10 WhatsNew with typing.{io|re} DeprecationWarning (GH-27872)
27f2f62