Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"ImportError: cannot import name 'Mapping' from 'collections'" with Python 3.10 #5478

Closed
simonkrenger opened this issue Dec 24, 2021 · 14 comments

Comments

@simonkrenger
Copy link

Environment information (required)

Diagnostics output
--- check: autoidentify
INFO: diagnose_tensorboard.py version e43767ef2b648d0d5d57c00f38ccbd38390e38da

--- check: general
INFO: sys.version_info: sys.version_info(major=3, minor=10, micro=1, releaselevel='final', serial=0)
INFO: os.name: posix
INFO: os.uname(): posix.uname_result(sysname='Linux', nodename='fedora-ml.example.com', release='5.15.10-200.fc35.x86_64', version='#1 SMP Fri Dec 17 14:46:39 UTC 2021', machine='x86_64')
INFO: sys.getwindowsversion(): N/A

--- check: package_management
INFO: has conda-meta: False
INFO: $VIRTUAL_ENV: None

--- check: installed_packages
INFO: installed: tensorboard==2.7.0
INFO: installed: tensorflow==2.8.0rc0
INFO: installed: tf-estimator-nightly==2.8.0.dev2021122109
INFO: installed: tensorboard-data-server==0.6.1

--- check: tensorboard_python_version
INFO: tensorboard.version.VERSION: '2.7.0'

--- check: tensorflow_python_version
2021-12-24 22:11:13.556272: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2021-12-24 22:11:13.556660: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
INFO: tensorflow.__version__: '2.8.0-rc0'
INFO: tensorflow.__git_version__: 'v1.12.1-69099-g0d00ab2fd15'

--- check: tensorboard_data_server_version
INFO: data server binary: '/home/simon/.local/lib/python3.10/site-packages/tensorboard_data_server/bin/server'
INFO: data server binary version: b'rustboard 0.6.1'

--- check: tensorboard_binary_path
INFO: which tensorboard: b'/home/simon/.local/bin/tensorboard\n'

--- check: addrinfos
socket.has_ipv6 = True
socket.AF_UNSPEC = <AddressFamily.AF_UNSPEC: 0>
socket.SOCK_STREAM = <SocketKind.SOCK_STREAM: 1>
socket.AI_ADDRCONFIG = <AddressInfo.AI_ADDRCONFIG: 32>
socket.AI_PASSIVE = <AddressInfo.AI_PASSIVE: 1>
Loopback flags: <AddressInfo.AI_ADDRCONFIG: 32>
Loopback infos: [(<AddressFamily.AF_INET6: 10>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('::1', 0, 0, 0)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 0))]
Wildcard flags: <AddressInfo.AI_PASSIVE: 1>
Wildcard infos: [(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('0.0.0.0', 0)), (<AddressFamily.AF_INET6: 10>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('::', 0, 0, 0))]

--- check: readable_fqdn
INFO: socket.getfqdn(): 'fedora-ml.example.com'

--- check: stat_tensorboardinfo
INFO: directory: /tmp/.tensorboard-info
INFO: os.stat(...): os.stat_result(st_mode=16895, st_ino=2071, st_dev=38, st_nlink=2, st_uid=1000, st_gid=1000, st_size=60, st_atime=1640379357, st_mtime=1640379357, st_ctime=1640379357)
INFO: mode: 0o40777

--- check: source_trees_without_genfiles
INFO: tensorboard_roots (1): ['/home/simon/.local/lib/python3.10/site-packages']; bad_roots (0): []

--- check: full_pip_freeze
INFO: pip freeze --all:
absl-py==1.0.0
argcomplete==1.12.3
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
astunparse==1.6.3
attrs==21.2.0
backcall==0.2.0
Beaker==1.10.0
beautifulsoup4==4.9.3
bleach==4.1.0
blivet==3.4.2
blivet-gui==2.3.0
Brlapi==0.8.2
cachetools==4.2.4
cffi==1.14.6
chardet==4.0.0
charset-normalizer==2.0.4
chrome-gnome-shell==0.0.0
click==8.0.1
cryptography==3.4.7
cupshelpers==1.0
cycler==0.11.0
dasbus==1.6
dbus-python==1.2.18
debugpy==1.5.1
decorator==5.1.0
defusedxml==0.7.1
distro==1.6.0
entrypoints==0.3
fedora-third-party==0.8
flatbuffers==2.0
fonttools==4.28.5
fros==1.1
gast==0.5.3
google-auth==2.3.3
google-auth-oauthlib==0.4.6
google-pasta==0.2.0
gpg==1.15.1
grpcio==1.43.0
h5py==3.6.0
humanize==0.5.1
idna==3.2
ipykernel==6.6.0
ipython==7.30.1
ipython-genutils==0.2.0
ipywidgets==7.6.5
jedi==0.18.1
Jinja2==3.0.3
jsonschema==4.3.2
jupyter==1.0.0
jupyter-client==7.1.0
jupyter-console==6.4.0
jupyter-core==4.9.1
jupyterlab-pygments==0.1.2
jupyterlab-widgets==1.0.2
keras==2.8.0rc0
Keras-Preprocessing==1.1.2
kiwisolver==1.3.2
langtable==0.0.56
libclang==12.0.0
libcomps==0.1.18
lxml==4.6.3
Mako==1.1.4
Markdown==3.3.6
MarkupSafe==2.0.0
matplotlib==3.5.1
matplotlib-inline==0.1.3
mistune==0.8.4
nbclient==0.5.9
nbconvert==6.3.0
nbformat==5.1.3
nest-asyncio==1.5.4
nftables==0.1
notebook==6.4.6
numpy==1.21.5
oauthlib==3.1.1
olefile==0.46
opencv-python==4.5.4.60
opt-einsum==3.3.0
packaging==21.0
pandocfilters==1.5.0
parso==0.8.3
Paste==3.5.0
pexpect==4.8.0
pickleshare==0.7.5
pid==2.2.3
Pillow==8.3.2
pip==21.2.3
ply==3.11
productmd==1.33
prometheus-client==0.12.0
prompt-toolkit==3.0.24
protobuf==3.19.1
ptyprocess==0.6.0
pwquality==1.4.4
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycairo==1.20.1
pycparser==2.20
pycrypto==2.6.1
pycups==2.0.1
pycurl==7.44.1
pyenchant==3.2.2
Pygments==2.10.0
PyGObject==3.42.0
pykickstart==3.34
pyOpenSSL==21.0.0
pyparsing==2.4.7
pyparted==3.11.7
pyrsistent==0.18.0
PySocks==1.7.1
python-augeas==1.1.0
python-dateutil==2.8.1
python-meh==0.50
pyudev==0.22.0
pyxdg==0.27
pyzmq==22.3.0
qtconsole==5.2.2
QtPy==2.0.0
regex==2021.10.23
requests==2.26.0
requests-file==1.5.1
requests-ftp==0.3.1
requests-oauthlib==1.3.0
rpm==4.17.0
rsa==4.8
selinux==3.3
Send2Trash==1.8.0
sepolicy==3.3
setools==4.4.0
setuptools==57.4.0
simpleaudio==1.0.4
simpleline==1.8.2
six==1.16.0
sos==4.2
soupsieve==2.3.1
systemd-python==234
Tempita==0.5.2
tensorboard==2.7.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.0
tensorflow==2.8.0rc0
tensorflow-io-gcs-filesystem==0.23.1
termcolor==1.1.0
terminado==0.12.1
testpath==0.5.0
tf-estimator-nightly==2.8.0.dev2021122109
tornado==6.1
tqdm==4.62.3
traitlets==5.1.1
typing_extensions==4.0.1
urllib3==1.26.6
wcwidth==0.2.5
webencodings==0.5.1
Werkzeug==2.0.2
wheel==0.37.1
widgetsnbextension==3.5.2
wrapt==1.13.3

Issue description

When running TensorBoard with Python 3.10.1, it immediately fails with the following ImportError:

$ tensorboard --logdir=logs/
Traceback (most recent call last):
  File "/home/simon/.local/bin/tensorboard", line 5, in <module>
    from tensorboard.main import run_main
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/main.py", line 27, in <module>
    from tensorboard import default
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/default.py", line 33, in <module>
    from tensorboard.plugins.audio import audio_plugin
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/plugins/audio/audio_plugin.py", line 23, in <module>
    from tensorboard import plugin_util
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/plugin_util.py", line 20, in <module>
    from tensorboard._vendor.bleach.sanitizer import Cleaner
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/_vendor/bleach/__init__.py", line 5, in <module>
    from tensorboard._vendor.bleach.linkifier import (
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/_vendor/bleach/linkifier.py", line 4, in <module>
    from tensorboard._vendor import html5lib
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/_vendor/html5lib/__init__.py", line 25, in <module>
    from .html5parser import HTMLParser, parse, parseFragment
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/_vendor/html5lib/html5parser.py", line 8, in <module>
    from . import _tokenizer
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/_vendor/html5lib/_tokenizer.py", line 16, in <module>
    from ._trie import Trie
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/_vendor/html5lib/_trie/__init__.py", line 3, in <module>
    from .py import Trie as PyTrie
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/_vendor/html5lib/_trie/py.py", line 6, in <module>
    from ._base import Trie as ABCTrie
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/_vendor/html5lib/_trie/_base.py", line 3, in <module>
    from collections import Mapping
ImportError: cannot import name 'Mapping' from 'collections' (/usr/lib64/python3.10/collections/__init__.py)

Root cause seems to be that the html5lib version used by TensorBoard does not yet include html5lib/html5lib-python#403. For a proper fix, html5lib should be updated to 1.1, which includes the fix.

Workaround: Manually changing from collections import [..] to from collections.abc import [..] in ~/.local/lib/python3.10/site-packages/tensorboard/_vendor/html5lib/_trie/_base.py resolves the issue.

@pindinagesh pindinagesh self-assigned this Dec 26, 2021
@pindinagesh
Copy link

@simonkrenger

Have you tried from collections import Mapping to from collections.abc import Mapping ? and moreover please have a look at similar issue which is described here #69381312. Thanks

@simonkrenger
Copy link
Author

simonkrenger commented Dec 26, 2021

@pindinagesh Yes, as written above I can confirm that using

from collections.abc import Mapping

works around the issue (which is also basically what was introduced in html5lib/html5lib-python#403). Thanks.

@bileschi
Copy link
Collaborator

Thanks for the report @simonkrenger , it looks like we will need to make a PR similar to #2549 to upgrade html5lib

@bileschi
Copy link
Collaborator

bileschi commented Dec 29, 2021

Hrm - looks like we can't upgrade to html5lib 1.1 right away - 1.1 is unreleased.

https://github.com/html5lib/html5lib-python/blob/1.1/CHANGES.rst

@nfelt
Copy link
Collaborator

nfelt commented Jan 7, 2022

I dunno why the release notes say UNRELEASED but it was definitely released - there's a 1.1 tag on GitHub and it's also on PyPI (https://pypi.org/project/html5lib/1.1/).

That said, since we vendor html5lib anyway rather than depending on it, we don't need it to be any particular release - we could use an arbitrary GitHub commit. We just need to update the dependencies here: https://cs.opensource.google/tensorflow/tensorboard/+/master:third_party/python.bzl;l=47-56;drc=c4ad956e4225b2b85330002745cff09f9e06b2f6

We'll probably also need to update third_party/html5lib.BUILD to match the expected set of files.

@nfelt
Copy link
Collaborator

nfelt commented Jan 7, 2022

@bmd3k This should be a pretty straightforward fix - let's try to get this into the coming release as well?

@bmd3k bmd3k assigned bmd3k and unassigned pindinagesh Jan 7, 2022
@bmd3k
Copy link
Contributor

bmd3k commented Jan 7, 2022

Additional note: On master branch it does not have the UNRELEASED text. It just seems like the 1.1 branch does not document its own release :).

bmd3k added a commit that referenced this issue Jan 11, 2022
As noted by #5478 , tensorboard fails when run in a python 3.10 environment. The reporter fortunately discovered that the problems would be fixed with an upgrade of our html5lib dependency.

I tested by building a python3.10 virtualenv on my machine (this required building python 3.10 from source as we don't have it natively).

I also ran the tensorboard with some logdirs and did some basic sanity. I specifically checked that markdown rendering seemed to be working for a couple text plugin logs that I have.
bmd3k added a commit to bmd3k/tensorboard that referenced this issue Jan 19, 2022
As noted by tensorflow#5478 , tensorboard fails when run in a python 3.10 environment. The reporter fortunately discovered that the problems would be fixed with an upgrade of our html5lib dependency.

I tested by building a python3.10 virtualenv on my machine (this required building python 3.10 from source as we don't have it natively).

I also ran the tensorboard with some logdirs and did some basic sanity. I specifically checked that markdown rendering seemed to be working for a couple text plugin logs that I have.
bmd3k added a commit that referenced this issue Jan 20, 2022
As noted by #5478 , tensorboard fails when run in a python 3.10 environment. The reporter fortunately discovered that the problems would be fixed with an upgrade of our html5lib dependency.

I tested by building a python3.10 virtualenv on my machine (this required building python 3.10 from source as we don't have it natively).

I also ran the tensorboard with some logdirs and did some basic sanity. I specifically checked that markdown rendering seemed to be working for a couple text plugin logs that I have.
simeoncarstens added a commit to simeoncarstens/nixpkgs that referenced this issue Jan 26, 2022
tensorboard is currently not compatible with Python 3.10, but will be
in the next release (tensorflow/tensorboard#5478).
@athiruma
Copy link

athiruma commented Apr 6, 2022

I was running elasticsearch and elasticsearch-dsl in python3.10. It raises error collections.abs.

  File "/usr/local/lib/python3.10/site-packages/elasticsearch/compat.py", line 36, in <module>
    from collections.abs import Mapping
ModuleNotFoundError: No module named 'collections.abs'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/cloud_governance/main.py", line 11, in <module>
    from cloud_governance.main.es_uploader import ESUploader
  File "/usr/local/lib/python3.10/site-packages/cloud_governance/main/es_uploader.py", line 2, in <module>
    from cloud_governance.common.elasticsearch.elasticsearch_operations import ElasticSearchOperations
  File "/usr/local/lib/python3.10/site-packages/cloud_governance/common/elasticsearch/elasticsearch_operations.py", line 11, in <module>
    from elasticsearch_dsl import Search
  File "/usr/local/lib/python3.10/site-packages/elasticsearch_dsl/__init__.py", line 18, in <module>
    from . import connections
  File "/usr/local/lib/python3.10/site-packages/elasticsearch_dsl/connections.py", line 18, in <module>
    from elasticsearch import Elasticsearch
  File "/usr/local/lib/python3.10/site-packages/elasticsearch/__init__.py", line 36, in <module>
    from .client import Elasticsearch
  File "/usr/local/lib/python3.10/site-packages/elasticsearch/client/__init__.py", line 22, in <module>
    from ..transport import Transport, TransportError
  File "/usr/local/lib/python3.10/site-packages/elasticsearch/transport.py", line 21, in <module>
    from .connection import Urllib3HttpConnection
  File "/usr/local/lib/python3.10/site-packages/elasticsearch/connection/__init__.py", line 19, in <module>
    from .http_requests import RequestsHttpConnection
  File "/usr/local/lib/python3.10/site-packages/elasticsearch/connection/http_requests.py", line 35, in <module>
    from ..compat import urlencode, string_types
  File "/usr/local/lib/python3.10/site-packages/elasticsearch/compat.py", line 38, in <module>
    from collections import Mapping
ImportError: cannot import name 'Mapping' from 'collections' (/usr/local/lib/python3.10/collections/__init__.py)

@HormatHamidov
Copy link

HormatHamidov commented May 2, 2022

Friends, the problem is very easy to solve. Delete and reinstall the django you downloaded for your project:
-> pip uninstall django
-> pip install django
-> py manage.py runserver
Done :D

@Rishabh-Commits
Copy link

@pindinagesh Yes, as written above I can confirm that using

from collections.abc import Mapping

works around the issue (which is also basically what was introduced in html5lib/html5lib-python#403). Thanks.

@pindinagesh Yes, as written above I can confirm that using

from collections.abc import Mapping

works around the issue (which is also basically what was introduced in html5lib/html5lib-python#403). Thanks.

where to use this?
'''from collections.abc import Mapping'''

@Rishabh-Commits
Copy link

@pindinagesh Yes, as written above I can confirm that using

@pindinagesh Yes, as written above I can confirm that using

from collections.abc import Mapping

works around the issue (which is also basically what was introduced in html5lib/html5lib-python#403). Thanks.

@simonkrenger can you tell me where to us this

@simonkrenger
Copy link
Author

simonkrenger commented Nov 21, 2022

@Rishabh-Commits

works around the issue (which is also basically what was introduced in html5lib/html5lib-python#403). Thanks.

@simonkrenger can you tell me where to us this

See the original description above:

Workaround: Manually changing from collections import [..] to from collections.abc import [..] in ~/.local/lib/python3.10/site-packages/tensorboard/_vendor/html5lib/_trie/_base.py resolves the issue.

@Sh0gunBR
Copy link

i,ve facing this issue too, but i,m used the module requests, and my code show me this error, someone help plz
cannot import name 'Mapping' from 'collections

@nfelt
Copy link
Collaborator

nfelt commented Jan 13, 2023

This issue should be resolved for recent versions of TensorBoard. If you are not using TensorBoard, please do not comment on this issue, we cannot help with resolving this error for you.

yatbear pushed a commit to yatbear/tensorboard that referenced this issue Mar 27, 2023
As noted by tensorflow#5478 , tensorboard fails when run in a python 3.10 environment. The reporter fortunately discovered that the problems would be fixed with an upgrade of our html5lib dependency.

I tested by building a python3.10 virtualenv on my machine (this required building python 3.10 from source as we don't have it natively).

I also ran the tensorboard with some logdirs and did some basic sanity. I specifically checked that markdown rendering seemed to be working for a couple text plugin logs that I have.
dna2github pushed a commit to dna2fork/tensorboard that referenced this issue May 1, 2023
As noted by tensorflow#5478 , tensorboard fails when run in a python 3.10 environment. The reporter fortunately discovered that the problems would be fixed with an upgrade of our html5lib dependency.

I tested by building a python3.10 virtualenv on my machine (this required building python 3.10 from source as we don't have it natively).

I also ran the tensorboard with some logdirs and did some basic sanity. I specifically checked that markdown rendering seemed to be working for a couple text plugin logs that I have.
yqshao pushed a commit to yqshao-archive/PiNN_Lab that referenced this issue May 1, 2023
yqshao pushed a commit to yqshao-archive/PiNN_Lab that referenced this issue May 1, 2023
yqshao pushed a commit to yqshao-archive/PiNN_Lab that referenced this issue May 1, 2023
yqshao pushed a commit to yqshao-archive/PiNN_Lab that referenced this issue May 1, 2023
yqshao pushed a commit to yqshao-archive/PiNN_Lab that referenced this issue May 1, 2023
yqshao pushed a commit to yqshao-archive/PiNN_Lab that referenced this issue May 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

9 participants