New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Some imports from standard lib collections are breaking on python 3.10 #160
Comments
It looks that this pull request (#159) covers this issue. |
Hi @apalala I made some import tests using docker. Here are the results: Tests with scrapinghubTest 1 - Python 3.9, scrapinghub installed from pipdocker run -ti python:3.9-slim bash -c "pip install scrapinghub; python -c 'import scrapinghub'" Result: No error Test 2 - Python 3.10, scrapinghub installed from pipdocker run -ti python:3.10-slim bash -c "pip install scrapinghub; python -c 'import scrapinghub'" Result: Error described in this issue Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/local/lib/python3.10/site-packages/scrapinghub/__init__.py", line 13, in <module>
from .hubstorage import HubstorageClient
File "/usr/local/lib/python3.10/site-packages/scrapinghub/hubstorage/__init__.py", line 6, in <module>
from .client import HubstorageClient
File "/usr/local/lib/python3.10/site-packages/scrapinghub/hubstorage/client.py", line 8, in <module>
from .project import Project
File "/usr/local/lib/python3.10/site-packages/scrapinghub/hubstorage/project.py", line 3, in <module>
from .job import Job
File "/usr/local/lib/python3.10/site-packages/scrapinghub/hubstorage/job.py", line 2, in <module>
from .resourcetype import (ItemsResourceType, DownloadableResource,
File "/usr/local/lib/python3.10/site-packages/scrapinghub/hubstorage/resourcetype.py", line 5, in <module>
from collections import MutableMapping
ImportError: cannot import name 'MutableMapping' from 'collections' (/usr/local/lib/python3.10/collections/__init__.py) Test 3 - Python 3.9, scrapinghub installed from github (branch update_py310_minus_py27)docker run -ti python:3.9-slim bash -c "apt update && apt install git -y; pip install git+https://github.com/scrapinghub/python-scrapinghub.git@update_py310_minus_py27#egg=scrapinghub; python -c 'import scrapinghub'" Result: No error Test 4 - Python 3.10, scrapinghub installed from github (branch update_py310_minus_py27)docker run -ti python:3.10-slim bash -c "apt update && apt install git -y; pip install git+https://github.com/scrapinghub/python-scrapinghub.git@update_py310_minus_py27#egg=scrapinghub; python -c 'import scrapinghub'" Result: No error 🎉 🎉 🎉 Tests with shubTest 1 - Python 3.9, shub installed from pipdocker run -ti python:3.9-slim bash -c "pip install shub; python -c 'import shub'" Result: No error Test 2 - Python 3.10, shub installed from pipdocker run -ti python:3.10-slim bash -c "pip install shub; python -c 'import shub'" Result: No error Test 3 - Python 3.9, shub installed from github (branch update_dependencies)docker run -ti python:3.9-slim bash -c "apt update && apt install git -y; pip install git+https://github.com/scrapinghub/shub.git@update_dependencies#egg=shub; python -c 'import shub'" Result: No error Test 4 - Python 3.10, shub installed from github (branch update_dependencies)docker run -ti python:3.10-slim bash -c "apt update && apt install git -y; pip install git+https://github.com/scrapinghub/shub.git@update_dependencies#egg=shub; python -c 'import shub'" Result: No error |
Thanks for those tests, @lbmendes. They are like the ones the libraries are running using Python 3.10 is not the only reason for the upgrade. These libraries were pinning other libraries to very old versions, and that has been fixed too. |
Fixed by #166 and released in 2.4.0. |
Hi everyone,
Based on an issue from another repo (okfn-brasil/querido-diario#502), I noticed that scrapinghub is using some imports from standard lib collections that are deprecated and not working on Python 3.10.
In Python 3.8 I have these results on ipython console:
According to this, it is necessary to change the imports of Iterable, Iterator and MutableMapping to get these items from "collections.abc" instead of just "collections"
Here are the list of imports that I found:
The text was updated successfully, but these errors were encountered: