SSLv3_METHOD where did you come from???????????????

[mvecchione145]

WHAT THE F

does not exist in pyopenssl

no help anywhere

[NguyenDa18]

I am getting the same issue too with Scrapy being used by GitHub Actions. SSL.SSLv3_METHOD exists in OpenSSL version 21.0.0 but not in latest 22.1.0 version which just came out today 😭
https://pypi.org/project/pyOpenSSL/#history

[NguyenDa18]

It is being addressed but not merged yet #5634

[elena]

Work around:

scrapy/core/downloader/tls.py#23

METHOD_SSLv3 = "SSLv3"
METHOD_TLS = "TLS"
METHOD_TLSv10 = "TLSv1.0"
METHOD_TLSv11 = "TLSv1.1"
METHOD_TLSv12 = "TLSv1.2"


openssl_methods = {
    METHOD_TLS: SSL.SSLv23_METHOD,  # protocol negotiation (recommended)
    # METHOD_SSLv3: SSL.SSLv3_METHOD,  # SSL 3 (NOT recommended).   <------------ COMMENT OUT THIS LINE
    METHOD_TLSv10: SSL.TLSv1_METHOD,  # TLS 1.0 only
    METHOD_TLSv11: getattr(SSL, "TLSv1_1_METHOD", 5),  # TLS 1.1 only
    METHOD_TLSv12: getattr(SSL, "TLSv1_2_METHOD", 6),  # TLS 1.2 only
}

Obviously not a production-ready solution, but will get your through the day.

[elena]

Further this was introduced by release here: https://github.com/pyca/pyopenssl/tags -- don't go blaming scrapy the wonderful folk here have actually picked this up and run with it immediately.

Thank you @NguyenDa18 , @gliptak , @Gallaecio and anyone else involved for being so proactive -- we are incredibly grateful!

Alternative work around is to revert to the previous release (the one that is >9h old at time of writing):

pyopenssl=22.0.0

[ilias-ant]

@Gallaecio are you also thinking of backporting the fix for v2.5.x as well (e.g. as v2.5.2)?

If not, it would be nice if you could include a temporary suggested workaround in the release notes for users that don't have the time/capacity to upgrade to v2.6.x yet.

[Gallaecio]

Is it under discussion, but I personally think backporting to 2.6.x is enough. That is where the most obvious issue happens, because pip install Scrapy breaks.

When installing older Scrapy versions, you should be aware that installing newer versions of dependencies, released after the release of the Scrapy version that you are installing, can break Scrapy. If you installed Scrapy 2.5 back when it was the latest release, you should not be affected by this issue as long as you are freezing dependencies, which is a good practice for non-packages. If we had to update all Scrapy versions every time we found out a new version of some package broke them, it would be a maintenance nightmare for us.

So, I am OK with bakporting this to 2.6 because 2.7 may not release for a while, and pip install Scrapy should work. But if 2.7 were to be release soon, I would not even bother backporting to 2.6.

[ilias-ant]

@Gallaecio makes sense! thank you for your prompt reply.

We only got a broken v2.5 build out of a custom Scrapy docker image we have (because it directly installs Scrapy). But ok, i guess i can just explicitly restrict pyopenssl until I find the time to update the image to Scrapy==2.6.x

[trinity652]

Hi, I resolved this issue by uninstalling pyOpenSSL==22.1.0 and installing pyOpenSSL==22.0.0, because a fresh install of Scrapy on my virtual environment broke it.

pip install Scrapy
pip uninstall pyOpenSSL==22.1.0
pip uninstall pyOpenSSL==22.0.0

Run Scrapy shell to check if it works

Current requirements.txt for my Scrapy install.

attrs==22.1.0
Automat==20.2.0
certifi==2022.9.24
cffi==1.15.1
charset-normalizer==2.1.1
constantly==15.1.0
cryptography==38.0.1
cssselect==1.1.0
filelock==3.8.0
hyperlink==21.0.0
idna==3.4
incremental==21.3.0
itemadapter==0.7.0
itemloaders==1.0.6
jmespath==1.0.1
lxml==4.9.1
parsel==1.6.0
Protego==0.2.1
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.21
PyDispatcher==2.0.6
pyOpenSSL==22.0.0
queuelib==1.6.2
requests==2.28.1
requests-file==1.5.1
Scrapy==2.6.2
service-identity==21.1.0
six==1.16.0
tldextract==3.3.1
Twisted==22.8.0
typing_extensions==4.3.0
urllib3==1.26.12
w3lib==2.0.1
zope.interface==5.4.0

[Yang66267]

pyOpenSSL==22.0.0
完美解决 上边的问题

[Gallaecio]

I am reopening this until we release 2.6.3, hopefully that will mean fewer duplicate reports.

[Gallaecio]

2.6.3 is out 🎉

[Aananda-giri]

I got the same error while deploying the code to scrapy hub.

Turned out Scrapy_hub was using an older version of scrapy i.e. version 2.0 and this forum helped me set the newer version.

For me, This worked:

• add this code at the end of the file: scrapinghub.yml

stacks:
  default: scrapy:2.11-20240514