Add support for SSL context and SNI

[jamesmyatt]

Use SSLContext objects, if available, since this provides more options and features including SNI (#11). Also, allows default system context from ssl.create_default_context() to be used, improves encapsulation and supports dependency injection.

Update: Removed support for SSL without SSLContext (#115)

[jamesmyatt]

I've rebased this on the latest develop branch to get the updated tests, etc.

[anthonyrisinger]

How can we move this forward? Is more work or review needed? SNI is the greatest!

[PierreF]

Just spotted a small unexpected behavious: tls_insecure_set() does not work when using tls_set_context() with a ssl.create_default_context().

This is caused by SSLContext.check_hostname that already performs the match_hostname during handshake. This flag is due on a SSLContext produced by create_default_context().

Note: when using tls_set() and tls_insecure_set() it works since in this case the SSLContext is created using ssl.SSLContext() and not create_default_context().

I think either tls_insecure_set() should update the check_hostname flag on the ssl context or add a note in documentation that tls_insecure_set should not be used with tls_set_context.

Step to reproduce:

# Certificate created with: openssl req -x509 -nodes -newkey rsa:1024 -keyout server.key -out server.pem -days 365 -subj '/CN=localhost' -batch
# mosquitto config:
# listener 1885
# cafile server.pem
# certfile server.pem
# keyfile server.key

import paho.mqtt.client
import ssl

cl2 = paho.mqtt.client.Client()
ssl_ctx = ssl.create_default_context(cafile='server.pem')
ssl_ctx.check_hostname = False  # Without this line, I does not work
cl2.tls_set_context(ssl_ctx)
cl2.tls_insecure_set(True)

# certificate was generated for CN=localhost, not "CN=127.0.0.1"
cl2.connect('127.0.0.1', 1885)

[jamesmyatt]

@PierreF , that makes sense. I think it's just a matter of re-arranging some of the logic. I'll look at it in more detail later.

[jamesmyatt]

OK. So I think it's pretty complicated to get all of the logic completely consistent. I'm going open an issue to propose removing SSL support in Python versions that don't include SSLContext (i.e. require Python 2.7.9+ or Python 3.2+). It would remove quite a lot of branches and make the intended behaviour clearer.

[ralight]

That sounds fine.

[jamesmyatt]

@PierreF , your issue should be fixed now. Note that you must call tls_insecure_set after tls_set now.

[PierreF]

ssl.CertificateError get caught by the except ValueError. Which means that if certificate validation failed (and server_hostname is supported) we will try to re-wrap the socket (which fail).

We probably should add a except ssl.CertificateError:\n raise juste before.

[jamesmyatt]

:( Thanks for checking this out!

[jamesmyatt]

Should be fixed now.

Looking at the ssl package docs, ssl.CertificateError should be the only error raised by the ssl code that isn't derived from OSError.

[Lokesh-K-Haralakatta]

When can we get the SNI support released with paho.mqtt.python pkg? We need this feature to support client side certificates in our connections to IoT Platform.

[jamesmyatt]

@Lokesh-K-Haralakatta , might be better to ask at on the Paho-dev mailing list: https://dev.eclipse.org/mailman/listinfo/paho-dev