collections: remove deprecated aliases to ABC classes #81505

vstinner · 2019-06-17T20:38:32Z

BPO	37324
Nosy	@vstinner, @serhiy-storchaka, @yan12125, @hugovk, @tirkarthi
PRs	bpo-37324: Remove ABC aliases from collections #14171 bpo-37324: Remove ABC aliases from collections #23754
Superseder	bpo-25988: collections.abc.Indexable

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-10-07.13:20:44.080>
created_at = <Date 2019-06-17.20:38:32.149>
labels = ['library', '3.9']
title = 'collections: remove deprecated aliases to ABC classes'
updated_at = <Date 2021-01-12.23:16:45.583>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2021-01-12.23:16:45.583>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2019-10-07.13:20:44.080>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2019-06-17.20:38:32.149>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 37324
keywords = ['patch']
message_count = 8.0
messages = ['345905', '345906', '345908', '345923', '348011', '348424', '354092', '384981']
nosy_count = 5.0
nosy_names = ['vstinner', 'serhiy.storchaka', 'yan12125', 'hugovk', 'xtreak']
pr_nums = ['14171', '23754']
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '25988'
type = None
url = 'https://bugs.python.org/issue37324'
versions = ['Python 3.9']

vstinner · 2019-06-17T20:38:32Z

Extract of collections documentation:
https://docs.python.org/dev/library/collections.html

"Deprecated since version 3.3, will be removed in version 3.9: Moved Collections Abstract Base Classes to the collections.abc module. For backwards compatibility, they continue to be visible in this module through Python 3.8."

Attached PR removes these aliases.

vstinner · 2019-06-17T20:47:07Z

pip is broken by this change, causing test_venv to fail: html5lib still uses collections to get ABC.

FAIL: test_with_pip (test.test_venv.EnsurePipTest)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/vstinner/prog/python/master/Lib/test/test_venv.py", line 407, in do_test_with_pip
    self.run_with_capture(venv.create, self.env_dir,
  File "/home/vstinner/prog/python/master/Lib/test/test_venv.py", line 69, in run_with_capture
    func(*args, **kwargs)
subprocess.CalledProcessError: Command '['/tmp/tmpkxga42lx/bin/python', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/vstinner/prog/python/master/Lib/test/test_venv.py", line 466, in test_with_pip
    self.do_test_with_pip(False)
  File "/home/vstinner/prog/python/master/Lib/test/test_venv.py", line 415, in do_test_with_pip
    self.fail(msg.format(exc, details))
AssertionError: Command '['/tmp/tmpkxga42lx/bin/python', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.

**Subprocess Output**
/tmp/tmpwhk_ss_e/pip-19.0.3-py2.py3-none-any.whl/pip/_vendor/pyparsing.py:3068: SyntaxWarning: invalid escape sequence \w
/tmp/tmpwhk_ss_e/pip-19.0.3-py2.py3-none-any.whl/pip/_vendor/pyparsing.py:3068: SyntaxWarning: invalid escape sequence \w
Traceback (most recent call last):
  File "/home/vstinner/prog/python/master/Lib/runpy.py", line 192, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/vstinner/prog/python/master/Lib/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/vstinner/prog/python/master/Lib/ensurepip/__main__.py", line 5, in <module>
    sys.exit(ensurepip._main())
  File "/home/vstinner/prog/python/master/Lib/ensurepip/__init__.py", line 198, in _main
    return _bootstrap(
  File "/home/vstinner/prog/python/master/Lib/ensurepip/__init__.py", line 117, in _bootstrap
    return _run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
  File "/home/vstinner/prog/python/master/Lib/ensurepip/__init__.py", line 27, in _run_pip
    import pip._internal
  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmpwhk_ss_e/pip-19.0.3-py2.py3-none-any.whl/pip/_internal/__init__.py", line 40, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmpwhk_ss_e/pip-19.0.3-py2.py3-none-any.whl/pip/_internal/cli/autocompletion.py", line 8, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmpwhk_ss_e/pip-19.0.3-py2.py3-none-any.whl/pip/_internal/cli/main_parser.py", line 12, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmpwhk_ss_e/pip-19.0.3-py2.py3-none-any.whl/pip/_internal/commands/__init__.py", line 6, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmpwhk_ss_e/pip-19.0.3-py2.py3-none-any.whl/pip/_internal/commands/completion.py", line 6, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmpwhk_ss_e/pip-19.0.3-py2.py3-none-any.whl/pip/_internal/cli/base_command.py", line 25, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmpwhk_ss_e/pip-19.0.3-py2.py3-none-any.whl/pip/_internal/index.py", line 14, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmpwhk_ss_e/pip-19.0.3-py2.py3-none-any.whl/pip/_vendor/html5lib/__init__.py", line 25, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmpwhk_ss_e/pip-19.0.3-py2.py3-none-any.whl/pip/_vendor/html5lib/html5parser.py", line 8, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmpwhk_ss_e/pip-19.0.3-py2.py3-none-any.whl/pip/_vendor/html5lib/_tokenizer.py", line 16, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmpwhk_ss_e/pip-19.0.3-py2.py3-none-any.whl/pip/_vendor/html5lib/_trie/__init__.py", line 3, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmpwhk_ss_e/pip-19.0.3-py2.py3-none-any.whl/pip/_vendor/html5lib/_trie/py.py", line 6, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmpwhk_ss_e/pip-19.0.3-py2.py3-none-any.whl/pip/_vendor/html5lib/_trie/_base.py", line 3, in <module>
ImportError: cannot import name 'Mapping' from 'collections' (/home/vstinner/prog/python/master/Lib/collections/__init__.py)

vstinner · 2019-06-17T20:54:21Z

html5lib has already been fixed:
https://github.com/vstinner/html5lib-python/commit/4f9235752cea29c5a31721440578b430823a1e69

But latest html5lib release was version 1.0.1 at... Dec 7, 2017 :-(

Paul Ganssle requested a new html5lib release... to fix the collections warning :-)
html5lib/html5lib-python#419

vstinner · 2019-06-17T21:58:35Z

Oh. I wasn't aware of the previous attempt: #10596 there is a long discussion there. The blocker issue was also html5lib.

vstinner · 2019-07-16T10:09:28Z

Right now (pip 19.1.1?), "./python -m venv env" still fails because of html5lib/_trie/py.py. There is still no activitiy on the request to get a new release:
html5lib/html5lib-python#419

yan12125 · 2019-07-25T02:39:51Z

In CPython commit 5380def, bundled pip is updated to 19.2.1, and since pip 19.2 an html5lib patch is backported [1] to "prefer importing from collections.abc instead of collections" [2]. I believe #10596 can be revisited now.

[1] pypa/pip@ef7ca14
[2] pypa/pip@3d6bb3a

tirkarthi · 2019-10-07T13:12:36Z

I think this can be closed since https://bugs.python.org/issue25988 has the PR merged.

vstinner · 2021-01-12T23:16:46Z

New changeset c47c78b by Hugo van Kemenade in branch 'master':
bpo-37324: Remove ABC aliases from collections (GH-23754)
c47c78b

vivainio · 2022-11-10T15:03:18Z

This broke setuptools 38.5.2

hugovk · 2022-11-10T15:09:59Z

setuptools 38.5.2 was released in 2018.

Python 3.10 was released on 2021-10-04, when setuptools was at 58.2.0, and is now at 65.5.1.

I recommend upgrading.

vivainio · 2022-11-10T15:27:37Z

@hugovk I was pinned to old version because newer setuptools created metadata that was incompatible with old version of Artifactory

vstinner · 2022-11-10T23:34:14Z

Well, if you start to pin versions, you should also pin the Python version: use Python 3.9 or older, which still provide deprecated aliases to collections ABC.

vivainio · 2022-11-11T07:55:48Z

@vstinner that's what we are doing, but we are now evaluating updating to python 3.10 and this (soft) blocker came up

vivainio · 2022-11-11T14:53:38Z

For posteriority, I packaged up the compatibility shims to a python package in pypi: https://github.com/vivainio/fix-py-collections

vstinner · 2022-11-12T17:42:08Z

For posteriority, I packaged up the compatibility shims to a python package in pypi: https://github.com/vivainio/fix-py-collections

I proposed https://peps.python.org/pep-0606/ but it got rejected.

vivainio · 2022-11-14T11:20:17Z

I proposed https://peps.python.org/pep-0606/ but it got rejected.

If there are a lot of breaking changes like this, maybe they could be addressed with a single "compat" package like six. I don't mind if it's a third party package, really.

Edit: I changed my mind. Third party package works very badly with setuptools, as you would need it available before running setup.py. This should come with Python

- PythonCharmers/python-future@02fdb32 - python/cpython#81505

Imports like Sequence, MutableSequence, etc. are deprecated since python3.3 and are removed from 3.8. This commit fixes these import problems for users with 3.8+ ref: python/cpython#81505

vstinner added 3.9 only security fixes stdlib Python modules in the Lib dir labels Jun 17, 2019

serhiy-storchaka closed this as completed Oct 7, 2019

ezio-melotti transferred this issue from another repository Apr 10, 2022

tik-stbuehler mentioned this issue May 11, 2022

breaks with python3.10 due to deprecated (and now removed) use of collections.Callable knipknap/exscript#219

Closed

sethaxen mentioned this issue Jun 2, 2022

ImportError: cannot import name 'Iterator' from 'collections' keiserlab/e3fp#62

Closed

wolfgang42 mentioned this issue Jun 4, 2022

Not compatible with Python 3.10 (cannot import name 'Mapping' from 'collections') syrusakbary/pyjade#293

Open

jeffbr13 mentioned this issue Sep 29, 2022

Python 3.10 AttributeError: module 'collections' has no attribute 'Iterable' chobeat/hypothesis-csv#3

Closed

dalito mentioned this issue Nov 9, 2022

Schema-automator is not yet 3.10 compatible. linkml/schema-automator#105

Closed

civanescu mentioned this issue Nov 14, 2022

"azureml.core" incompatible with Python 3.10.5 MicrosoftDocs/azure-docs#101397

Closed

kciy mentioned this issue Jan 13, 2023

Update SPAdes 3.12.0 bioconda/bioconda-recipes#38819

Merged

delta1513 mentioned this issue Feb 17, 2023

Python 3.10 compatibility - Use of deprecated alias to Collections.abc.Sequence cjdrake/pyeda#171

Closed

James-E-A mentioned this issue May 4, 2023

import utils problem Wiston999/python-rubik#14

Open

James-E-A added a commit to James-E-A/python-rubik that referenced this issue May 5, 2023

Python 3.9+ compatibility

e0764d5

- PythonCharmers/python-future@02fdb32 - python/cpython#81505

James-E-A mentioned this issue May 5, 2023

Fix ImportError on Python 3.9+ Wiston999/python-rubik#16

Open

rseragon mentioned this issue Jul 27, 2023

Fix: deprecated collections imports mcs07/ChemDataExtractor#41

Closed

phantom-duck mentioned this issue Sep 29, 2023

Building docs with python >= 3.10 and sphinx < 2 fails Trusted-AI/AIF360#496

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

collections: remove deprecated aliases to ABC classes #81505

collections: remove deprecated aliases to ABC classes #81505

vstinner commented Jun 17, 2019

vstinner commented Jun 17, 2019

vstinner commented Jun 17, 2019

vstinner commented Jun 17, 2019

vstinner commented Jun 17, 2019

vstinner commented Jul 16, 2019

yan12125 mannequin commented Jul 25, 2019

tirkarthi commented Oct 7, 2019

vstinner commented Jan 12, 2021

vivainio commented Nov 10, 2022

hugovk commented Nov 10, 2022

vivainio commented Nov 10, 2022

vstinner commented Nov 10, 2022

vivainio commented Nov 11, 2022

vivainio commented Nov 11, 2022

vstinner commented Nov 12, 2022

vivainio commented Nov 14, 2022 •

edited

Loading

collections: remove deprecated aliases to ABC classes #81505

collections: remove deprecated aliases to ABC classes #81505

Comments

vstinner commented Jun 17, 2019

vstinner commented Jun 17, 2019

vstinner commented Jun 17, 2019

vstinner commented Jun 17, 2019

vstinner commented Jun 17, 2019

vstinner commented Jul 16, 2019

yan12125 mannequin commented Jul 25, 2019

tirkarthi commented Oct 7, 2019

vstinner commented Jan 12, 2021

vivainio commented Nov 10, 2022

hugovk commented Nov 10, 2022

vivainio commented Nov 10, 2022

vstinner commented Nov 10, 2022

vivainio commented Nov 11, 2022

vivainio commented Nov 11, 2022

vstinner commented Nov 12, 2022

vivainio commented Nov 14, 2022 • edited Loading

vivainio commented Nov 14, 2022 •

edited

Loading