-
-
Notifications
You must be signed in to change notification settings - Fork 31.7k
Support TLS SNI extension in ssl module #49889
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
With TLS it is possible to have the client use an extension (defined in OpenSSL 0.9.8f onwards have optional support for this; OpenSSL needs to This patch, against Python 2.6.1, adds to the standard ssl module the I have tested this on FreeBSD 7.0/amd64 with OpenSSL 0.9.8k when talking I am not a Python core dev and not too enthusiastic at the thought of ============= import socket
import ssl
import sys
def dump_https_page(hostname, uri='/'):
sock = socket.socket(socket.AF_INET)
s = ssl.SSLSocket(sock=sock,
ca_certs='/etc/ssl/certs',
server_hostname=hostname)
print 'have socket'
s.connect((hostname, 443))
print 'connected' print >>s, 'GET %s HTTP/1.0\r\nHost: %s\r\nConnection: close\r\n\r\n' % ( t = s.read()
while t:
print t,
t = s.read()
if __name__ == '__main__':
for x in sys.argv[1:]:
dump_https_page(hostname=x) |
Note: this previous work is client-side only, as noted in the body of |
patch against TRUNK (2.7) with self tests and doco. Essentially the same Contains some spacing cleanups that where highlighted by vim. |
py3k version |
current self tests cannot fully test the existence of the SNI extension This client script run with argument sni.velox.ch will show the "Great! |
The small deficiency with these patches is that the specified |
(Sorry for dropping this, lost available time) I see your point. OTOH, use of SNI needs to be something that can be Unless s.connect() gains a keep_original_hostname=False option (?), this Then there's the principle of least surprise -- while it would be nice So I tend towards favouring "make use of the newer, less well tested, In which case, if the default is to change, the API should be sorted |
Hey Phil,
Options for client side SNI:
Small rational to enable SNI by default:
just my current thoughts |
wrapssl(server_hostname=True/False/String) looks good to me. Your arguments for enabling by default are compelling, for P3k. |
Too late for 2.7 now, but looks like a good idea. |
The patch probably needs refreshing now that first SSL contexts are in. I wonder whether a combined boolean/string flag is really the best solution. I think we could instead enable SNI by default and add an optional "server_hostname" to set the hostname to SSLContext.wrap_socket(), so that people can explicitly set the hostname; and otherwise take it, if possible, from the argument given to connect(). We can also add an "enable_sni" attribute to SSLContext (True by default) to allow selective disabling. This attribute would raise an exception if SNI support isn't available, which would be a way to test for it. |
Here's another possible approach: ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
ctx.set_tlsext_host_name("foo.bar")
skt = ctx.wrap_socket(socket.socket())
skt.connect("bar.baz") This makes it obvious what the SNI hostname is and what the TCP address to connect to is, and they can easily be different. |
I quite like your proposed alternative here. Not sure when/if I'll get to implement this.
|
Well, the hostname should be specific to a connection, so I'm not sure it makes sense to set it on the context. |
It's almost always wrong for Python to enforce a particular *policy*, particularly in a very low level API (which is what the ssl module should be). Python's main job is to make it *possible* to do things. It's the application developer's job to decide what things should be done. It would be entirely appropriate, though, for a higher-level interface (for example, the httplib module) to take care of this itself and not require users to explicitly specify things separately.
That doesn't make sense to me. For example, consider the case where you're talking to a web service. The hostname lookup might result in 10 A records, which you then drop into a connection pool. Your application doesn't care which server you talk to (and maybe it talks to serveral, or all, of them). But it does want to specify the same hostname for each.
Nope, I checked before making the suggestion. There's an SSL_CTX_ version of this API (in addition to the SSL_ version). |
Sorry, I just checked again, and it seems you're right. Perhaps I saw SSL_CTX_set_tlsext_servername_callback and got the two confused. |
Ok, I find this argument rather convincing. Also, enabling implicit SNI with the connect() argument could make user code stop working if he decides to pass the IP instead, without him being able to diagnose precisly what happens. As you said, httplib/urllib should probably enable client-side SNI by default. |
Here is a patch for py3k, including http.client and urllib support. |
Committed with docs in r85793. |
I see the patch has been applied python3 in r85793, but is there any chance there will also be patches for python 2.6 or 2.7? And if so, what release of python (any version) might this patch be included in? |
No, Python 2 only receives bug fixes. |
And python3? Any idea which version the patch will be included there? On 11 August 2011 18:49, Antoine Pitrou <report@bugs.python.org> wrote:
|
It was included in Python 3.2. |
Python 2.7 is still used in production. Given the scarcity of IPv4-addresses — and with CDNs (think: Amazon, Akamai, EdgeCast…) starting to offer HTTP+SSL — the need for SNI arises in order to avoid pitfalls such as shared certificates. The lack of ubiquitous support for TLS SNI could cause delays in HTTPS-everywhere–deployments. Therefore, in the light of the latest revelations about mass surveillance, I'd like even to argue that this is a matter of security and privacy. |
Mark, thanks for the patch. However, unless exceptional situations, we don't |
Antoine, thank you for the heads-up. As long as I've reminded distribution maintainers of this issue and this or a similar patch (always send a server_hostname with TLS, if one is missing) will be integrated (please do!) I've accomplished my goal. BTW, today I've encountered a similar certificate. Semper aliquid haeret: subjectAltName=DNS:cdn.cloudtop.org,DNS:barely-legal-spam.com,DNS:*.banging-ham.com,DNS:jimmyforcongress2014.com ;-) |
Is this really not going into Python2 series? It's not a Python feature or a language feature, it's a matter of exporting OpenSSL feature. Furthermore it's a matter of security, same as support for session tickets is a matter of performance. SNI was first introduced in 2004, RFC in 2006, libcurl supported it from 2008, you had a patch since 2009 and still it's not in? Are you guys intentionally trying to cripple Python2? What do you think is likelier outcome, faster Python3 adoption or Python labelled insecure? |
It's a feature regardless (from our POV), and Python 2.x has been in bug fix mode for a long time now. Please understand that this is how our release process works. |
This is *not* a feature request, it's a bug fix in the underlying protocols. Client sides that do not send SNI are actively hurting the Web and the Internet by constraining the deployment of TLS. The closest analogy would be if Python's HTTP client side didn't emit a Host header, and the excuse were "But we only advertise ourselves as HTTP/1.0." The biggest difference being that this has additional security impact. The pain of lack of support for SNI is completely borne by the server-side, not the client (here, Python). As such, this is not a feature for Python client-side developers, but an interop / scaling / security issue for the Web and Internet overall. |
Are you kidding me? I can't believe SNI isn't being backported to python 2.x. This is ridiculous in my opinion. The bug fix needs to be back ported. |
I'd be happy to add a disclaimer to the Python 2.7 docs directing users to use the requests module instead (https://pypi.python.org/pypi/requests). People *really* *really* *really* should be using requests on the client side when doing HTTP/HTTPS in Python 2.x - the standard library support is now too old to have kept up with the evolution of web security and standard. |
To be clear, to get SNI with requests on 2.x you need requests, pyopenssl, ndg-httpsclient, and pyasn1 (which also pulls in cryptography, six, cffi, and pycparser). So that's 8 dependencies to get SNI on Python 2.x. At least it's doable but it's kind of really unfriendly :/ Also the error message you get when you need SNI and it's not available is basically the most obtuse thing ever. You get told that the SSL verification failed for <some other domain> that isn't what you asked for but when you go to it in your Browser it'll all work kosher with no indication it was using SNI. It's generally a good idea to install those extra things anyways because the SSL lib on Python 2.7 has other actual security issues which those address (IIRC it still has TLS compression on, I think it's default cipher list is rather poor, doesn't support TLS 1.2, etc). |
+dstufft is absolutely right. SNI needs to be enabled on lower level than "user" python code. if it is, requests and most other http client libs get it for free without dependencies. |
Nick: rather than direct users to use requests, we should direct them to use Python 3, which has had SNI support for 3+ years now. If client programs choose to remain on Python 2, it's *their* fault, not Python's. |
Antoine, was Python 2.x a mistake? I don't think so. SNI is not a language feature, it's not even a python extension feature. |
Really, can you stop arguing about this? |
I'm currently discussing some options with Donald and Christian. While it's |
I'm missing some context to appreciate your message, Nick, but please note that SNI is not in itself a security feature. It just enables interoperability with TLS virtual hosts (aka. hosting several TLS-enabled domains behind a single IP and port). |
It's somewhat of a grey area of security feature. It's not directly a security feature but if you don't have SNI and you hit a site that requires it then your error message is going to be something like what people run into with PyPI[1] which is "Cannot verify pypi.python.org, does not match hostname *.a.ssl.fastly.net". At this point most people go "What?" and assume the site is at fault and disable verification. Even more frustrating is this is going to work fine in their browser. The answer of how to actually verify this is without SNI is (once you even figure out the problem is SNI, which is non obvious) verify against what's actually in the CN of the cert, and send a Host header for what site you actually want. So while it is not strictly a security feature, it is fairly important for reasonably securely connecting to a site that requires SNI for the lay person. [1] PyPI's problem is no SNI but that some clients don't support SAN certificates, but the error message is exactly the same. |
Understood, but that's no different from trying to connect with an old Windows or MSIE version (which I'm sure will also fail on some websites). Client-side SNI support has been added in Python 3.2, and 3.4 is now out. People who migrated their code to Python 3 have been enjoying SNI support for years now, and they're gradually getting more TLS features at every new feature release. |
Please discuss the Python 2 documentation in a new issue, this one is now closed and so hidden from the list of bugs. |
Hopefully pep-466 resolves this for 2.x series. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: