Is urlwatch using OpenSSL somehow

[onrea]

If it's using OpenSSL, I need to force it using older version of OpenSSL on Windows10! My service is using an old crypto that is implemented in openssl with version<v1.1.0.

[thp]

It's using whatever SSL implementation is used with Python and Request. Have you tried the ssl_no_verify option?

[cfbao]

On Windows, OpenSSL is bundled with your Python installation.
The official installer of Python 3.7 bundles OpenSSL 1.1.0.
Installers of Python 3.4 - 3.6 bundles OpenSSL 1.0.2.

[onrea]

Installers of Python 3.4 - 3.6 bundles OpenSSL 1.0.2

Right, thanks. Is OpenSSL bundled into requests too?

If I try to connect to my site with the latest version of OpenSSL, I get a fatal error related to certificate as a deprecated encryption algorithm is used client-side for my site, while the legacy version of OpenSSL, v1.0.2, has still support for that encryption method and can connect to my site. But the OpenSSL1.0.2 module, bundled in python3.4-3.6 for windows, returns same error that I see with the latest OpenSSL. I was wondering if an OpenSSL package is bundled within requests. Do you have any information about this?

[cfbao]

I don't think requests bundles OpenSSL.

Are you sure the cipher you use is in all versions of OpenSSL 1.0.2? There are lettered sub-versions under 1.0.2. Some old ciphers are deprecated or removed in later sub-versions.
https://www.openssl.org/news/openssl-1.0.2-notes.html
https://www.openssl.org/news/changelog.html

You can check the OpenSSL version used by your Python installation by

import ssl
print(ssl.OPENSSL_VERSION)

[onrea]

Python34: OpenSSL 1.0.2d Jul 2015
Python36: OpenSSL 1.0.2j Sep 2016

I don't have problem with open ssl v1.0.2q itself (installed directly on Windows10) as it can connect to the site. See more details at the below link:

> openssl s_client -connect <IP>:<Port>
output: https://dpaste.de/3ZDp/raw

But when I run urlwatch (with all these version of Python: 3.7, 3.6 and 3.4), I get this error:

> python Scripts\urlwatch

============================
01. ERROR: SITE
============================

----------------------------
ERROR: SITE (https://<LINK>)
----------------------------
HTTPSConnectionPool(host='<DOMAIN>', port=<Port>): Max retries exceeded with url: <PART_of_DOMAIN> (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:600)'),))

---

--
urlwatch 2.16, Copyright 2008-2019 Thomas Perl
Website: https://thp.io/2008/urlwatch/
watched 1 URLs in 1 seconds

How do you use OpenSSL in urlwatch? May you give me the exact command to test it directly via OpenSSL on my machin?

[cfbao]

urlwatch doesn't use OpenSSL directly. Everything is handled in requests.

If the site is using an old insecure cipher, it's possible that requests itself has removed support for it (see this comment), even when the underlying OpenSSL still supports it.

You can try

import requests
r = requests.get('<your_url>')

and see if it raises errors.

You can also try the ssl_no_verify flag (if the cipher is insecure anyway):

kind: url
url: <your_url>
ssl_no_verify: true

[onrea]

You're right. I also get the error, given by urlwatch, exactly with requests: https://dpaste.de/Uohd/raw

I also installed old version of requests (v2.8, 2015 | v2.2, 2014), still getting that error.

[onrea]

Does ssl_no_verify: true equal to verify=False in:
r = requests.get('<your_url>, verify=False')

[cfbao]

Yes, setting ssl_no_verify = true is the same as setting verify=False in requests.
Come to think of it, there'll probably still be error if requests can't even complete the SSL handshake (presumably due to incompatible cipher suite). This option only disables certificate checking.

This blog post probably has your solution. But urlwatch doesn't support it yet.

[cfbao]

I've opened another issue superseding this discussion: #362