Skip to content

Commit

Permalink
Use newer ssl context proto version method.
Browse files Browse the repository at this point in the history
  • Loading branch information
@VeNoMouS
VeNoMouS committed Apr 25, 2023
1 parent 9855d8e commit cbb3c0e
Showing 1 changed file with 3 additions and 4 deletions.
7 changes: 3 additions & 4 deletions cloudscraper/__init__.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@

# ------------------------------------------------------------------------------- #

__version__ = '1.2.70'
__version__ = '1.2.71'

# ------------------------------------------------------------------------------- #

Expand Down Expand Up @@ -83,9 +83,8 @@ def __init__(self, *args, **kwargs):
self.ssl_context.set_ciphers(self.cipherSuite)
self.ssl_context.set_ecdh_curve(self.ecdhCurve)

self.ssl_context.options |= (ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3 | ssl.OP_NO_TLSv1_1)
if sys.version_info[:2] < (3, 10): # Remove deprecation warning in >= py3.10
self.ssl_context.options |= ssl.OP_NO_TLSv1
self.ssl_context.minimum_version = ssl.TLSVersion.TLSv1_2
self.ssl_context.maximum_version = ssl.TLSVersion.TLSv1_3

super(CipherSuiteAdapter, self).__init__(**kwargs)

Expand Down

4 comments on commit cbb3c0e

@2dareis2do
Copy link

@2dareis2do 2dareis2do commented on cbb3c0e Sep 27, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK so it looks like this explains why I am receiving the following errors on some sites:

e.g.

File "/usr/local/lib/python3.9/site-packages/cloudscraper/__init__.py", line 259, in request self.perform_request(method, url, *args, **kwargs) File "/usr/local/lib/python3.9/site-packages/cloudscraper/__init__.py", line 192, in perform_request return super(CloudScraper, self).request(method, url, *args, **kwargs) File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 517, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='www.kentonline.co.uk', port=443): Max retries exceeded with url: /medway/sport/dynamos-fight-back-against-solent-but-lose-out-to-the-reigni-312973/ (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1123)'))) ". 0

Is there a recommended way to handle this sort of error?

I am using like so:

newurl = accept_cookies_and_redirect_and_get_url(url)

browser={
    'browser': 'chrome',
    'platform': 'android',
    'desktop': False
}

scraper = cloudscraper.create_scraper(browser)  # returns a CloudScraper instance

sys.stdout = open(os.devnull, "w") #To prevent a function from printing in the batch console in Python

# url = functionName = sys.argv[1]

scraped = scraper.get(newurl).text

@2dareis2do
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually even with 1.2.68 I am getting some errors:

e.g.

File "/lib/python3.11/site-packages/cloudscraper/__init__.py", line 190, in perform_request
    return super(CloudScraper, self).request(method, url, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lib/python3.11/site-packages/requests/adapters.py", line 698, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='www.kentonline.co.uk', port=443): Max retries exceeded with url: /medway/sport/dynamos-fight-back-against-solent-but-lose-out-to-the-reigni-312973/ (Caused by SSLError(SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:992)')))

@2dareis2do
Copy link

@2dareis2do 2dareis2do commented on cbb3c0e Sep 27, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and similar when using dev version .71. So probably not related to this change

e.g.

File "//lib/python3.11/site-packages/requests/adapters.py", line 698, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='www.kentonline.co.uk', port=443): Max retries exceeded with url: /medway/sport/dynamos-fight-back-against-solent-but-lose-out-to-the-reigni-312973/ (Caused by SSLError(SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:992)')))

@2dareis2do
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually I was able to solve work around be changing the cipher

e.g.

browser={
'browser': 'chrome',
'platform': 'android',
'desktop': False
}

scraper = cloudscraper.create_scraper(browser, ecdhCurve='secp384r1')

https://github.com/VeNoMouS/cloudscraper?tab=readme-ov-file#example-14

thanks

Please sign in to comment.