get_url: bad handshake failure, with ansible 2.3.1.0, python 2.7.12

[dflock]

ISSUE TYPE

• Bug Report

COMPONENT NAME

get_url

ANSIBLE VERSION

ansible 2.3.1.0
  config file = 
  configured module search path = Default w/o overrides
  python version = 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]

CONFIGURATION

[defaults]
log_path = ./ansible.log
pipelining = True
host_key_checking = False
command_warnings = True
retry_files_enabled = False

OS / ENVIRONMENT

OS: Ubuntu 16.04.2 LTS (Xenial Xerus)
Python: 2.7.12
OpenSSL: 1.0.2g 1 Mar 2016

Possibly relevant python modules:

• pyOpenSSL==17.0.0
• ndg-httpsclient==0.4.2
• pyasn1==0.2.3
• pyasn1-modules==0.0.7
• idna==2.5
• pyOpenSSL==17.0.0
• urllib3==1.21.1
• requests==2.17.3

SUMMARY

Trying to download a vagrant binary (from https://releases.hashicorp.com/vagrant/) using the get_url module fails with the following error (wrapped by me):

Failed to validate the SSL certificate for releases.hashicorp.com:443.
Make sure your managed systems have a valid CA certificate installed.
You can use validate_certs=False if you do not need to confirm the servers identity
but this is unsafe and not recommended.

Paths checked for this platform:

/etc/ssl/certs,
/etc/pki/ca-trust/extracted/pem,
/etc/pki/tls/certs,
/usr/share/ca-certificates/cacert.org,
/etc/ansible.

The exception msg was:

(\"bad handshake: Error([
  ('SSL routines', 'ssl3_read_bytes', 'tlsv1 alert protocol version')
],)

STEPS TO REPRODUCE

$ ansible localhost -m get_url -a 'url=https://releases.hashicorp.com/vagrant/ dest=/tmp/download'

EXPECTED RESULTS

I should work. If you run it with validate_certs=false, it does work:

$ ansible localhost -m get_url -a 'url=https://releases.hashicorp.com/vagrant/ dest=/tmp/download validate_certs=false'

 [WARNING]: Host file not found: /etc/ansible/hosts

 [WARNING]: provided hosts list is empty, only localhost is available

localhost | SUCCESS => {
    "changed": false, 
    "dest": "/tmp/download", 
    "gid": 1000, 
    "group": "duncan", 
    "mode": "0664", 
    "msg": "file already exists", 
    "owner": "duncan", 
    "size": 3123, 
    "state": "file", 
    "uid": 1000, 
    "url": "https://releases.hashicorp.com/vagrant/"
}

ACTUAL RESULTS

$ ansible localhost -m get_url -a 'url=https://releases.hashicorp.com/vagrant/ dest=/tmp/download' -vvvv

No config file found; using defaults
 [WARNING]: Host file not found: /etc/ansible/hosts

 [WARNING]: provided hosts list is empty, only localhost is available

Loading callback plugin minimal of type stdout, v2.0 from /usr/local/lib/python2.7/dist-packages/ansible/plugins/callback/__init__.pyc
META: ran handlers
Using module file /usr/local/lib/python2.7/dist-packages/ansible/modules/network/basics/get_url.py
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: duncan
<127.0.0.1> EXEC /bin/sh -c 'echo ~ && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/duncan/.ansible/tmp/ansible-tmp-1496782045.65-131654390911415 `" && echo ansible-tmp-1496782045.65-131654390911415="` echo /home/duncan/.ansible/tmp/ansible-tmp-1496782045.65-131654390911415 `" ) && sleep 0'
<127.0.0.1> PUT /tmp/tmpzQtKr2 TO /home/duncan/.ansible/tmp/ansible-tmp-1496782045.65-131654390911415/get_url.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/duncan/.ansible/tmp/ansible-tmp-1496782045.65-131654390911415/ /home/duncan/.ansible/tmp/ansible-tmp-1496782045.65-131654390911415/get_url.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python /home/duncan/.ansible/tmp/ansible-tmp-1496782045.65-131654390911415/get_url.py; rm -rf "/home/duncan/.ansible/tmp/ansible-tmp-1496782045.65-131654390911415/" > /dev/null 2>&1 && sleep 0'
localhost | FAILED! => {
    "changed": false, 
    "failed": true, 
    "invocation": {
        "module_args": {
            "attributes": null, 
            "backup": false, 
            "checksum": "", 
            "content": null, 
            "delimiter": null, 
            "dest": "/tmp/download", 
            "directory_mode": null, 
            "follow": false, 
            "force": false, 
            "force_basic_auth": false, 
            "group": null, 
            "headers": null, 
            "http_agent": "ansible-httpget", 
            "mode": null, 
            "owner": null, 
            "regexp": null, 
            "remote_src": null, 
            "selevel": null, 
            "serole": null, 
            "setype": null, 
            "seuser": null, 
            "sha256sum": "", 
            "src": null, 
            "timeout": 10, 
            "tmp_dest": "", 
            "unsafe_writes": null, 
            "url": "https://releases.hashicorp.com/vagrant/", 
            "url_password": null, 
            "url_username": null, 
            "use_proxy": true, 
            "validate_certs": true
        }
    }, 
    "msg": "Failed to validate the SSL certificate for releases.hashicorp.com:443. Make sure your managed systems have a valid CA certificate installed. You can use validate_certs=False if you do not need to confirm the servers identity but this is unsafe and not recommended. Paths checked for this platform: /etc/ssl/certs, /etc/pki/ca-trust/extracted/pem, /etc/pki/tls/certs, /usr/share/ca-certificates/cacert.org, /etc/ansible. The exception msg was: (\"bad handshake: Error([('SSL routines', 'ssl3_read_bytes', 'tlsv1 alert protocol version')],)\",)."
}

[dflock]

This is what openssl says about releases.hashicorp.com:443:

$ openssl s_client -connect releases.hashicorp.com:443

CONNECTED(00000003)
depth=2 C = BE, O = GlobalSign nv-sa, OU = Root CA, CN = GlobalSign Root CA
verify return:1
depth=1 C = BE, O = GlobalSign nv-sa, CN = GlobalSign CloudSSL CA - SHA256 - G3
verify return:1
depth=0 C = US, ST = California, L = San Francisco, O = "Fastly, Inc", CN = s.ssl.fastly.net
verify return:1
---
Certificate chain
 0 s:/C=US/ST=California/L=San Francisco/O=Fastly, Inc/CN=s.ssl.fastly.net
   i:/C=BE/O=GlobalSign nv-sa/CN=GlobalSign CloudSSL CA - SHA256 - G3
 1 s:/C=BE/O=GlobalSign nv-sa/CN=GlobalSign CloudSSL CA - SHA256 - G3
   i:/C=BE/O=GlobalSign nv-sa/OU=Root CA/CN=GlobalSign Root CA
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIHvTCCBqWgAwIBAgIMZu1ZtYWr84THrOYbMA0GCSqGSIb3DQEBCwUAMFcxCzAJ
BgNVBAYTAkJFMRkwFwYDVQQKExBHbG9iYWxTaWduIG52LXNhMS0wKwYDVQQDEyRH
bG9iYWxTaWduIENsb3VkU1NMIENBIC0gU0hBMjU2IC0gRzMwHhcNMTcwNDI1MTkz
NzAzWhcNMTcwNjI5MTc0NjQ1WjBrMQswCQYDVQQGEwJVUzETMBEGA1UECAwKQ2Fs
aWZvcm5pYTEWMBQGA1UEBwwNU2FuIEZyYW5jaXNjbzEUMBIGA1UECgwLRmFzdGx5
LCBJbmMxGTAXBgNVBAMMEHMuc3NsLmZhc3RseS5uZXQwggEiMA0GCSqGSIb3DQEB
AQUAA4IBDwAwggEKAoIBAQCqj0oXBg+SH9lQ2OD/ZPVzSPUKE/xn2dpQYaWlf8/W
n7TpCeaFm0/bdTCYc5O/B4p/q49kjH/WEMtszV27hEPX3zI20LGAFx5gNQiFVkLE
5WPKLASuQJZYnPTmPEdflKFrq7pPD5FLl2ZSOe4RUdc8AATeLv3SJPq305TGiRP5
eTbFf+/iTVp7ggNuOb2G+YN3mxUqZCKOcFYvPdKF4NZOn1HB8xx+PrVOWQch+tjT
Djc0OfYD0oFngpSbA0kKV0GN1Ddv93PDEBCrjJV8JAKDcNrLmXuFw1Rqo5WnzzP7
Me9WLN599/y0ynO3/Jd34JlrnsCTGheUNjnbmM5F1S3tAgMBAAGjggRzMIIEbzAO
BgNVHQ8BAf8EBAMCBaAwgYoGCCsGAQUFBwEBBH4wfDBCBggrBgEFBQcwAoY2aHR0
cDovL3NlY3VyZS5nbG9iYWxzaWduLmNvbS9jYWNlcnQvY2xvdWRzc2xzaGEyZzMu
Y3J0MDYGCCsGAQUFBzABhipodHRwOi8vb2NzcDIuZ2xvYmFsc2lnbi5jb20vY2xv
dWRzc2xzaGEyZzMwVgYDVR0gBE8wTTBBBgkrBgEEAaAyARQwNDAyBggrBgEFBQcC
ARYmaHR0cHM6Ly93d3cuZ2xvYmFsc2lnbi5jb20vcmVwb3NpdG9yeS8wCAYGZ4EM
AQICMAkGA1UdEwQCMAAwggMMBgNVHREEggMDMIIC/4IQcy5zc2wuZmFzdGx5Lm5l
dIIMKi5lbGFzdGljLmNvggoqLmZvdW5kLm5vgg8qLmZveHRvbnMuY28udWuCFCou
Z2F6ZWxsZXN0YWdpbmcuY29tgg8qLmhhc2hpY29ycC5jb22CECouaW52YWx1YWJs
ZS5jb22CEiouc2hpZnRwcmV2aWV3LmNvbYIMKi5zb2FzdGEuY29tghVhcGkuc2hp
ZnRjb21tZXJjZS5jb22CCWNvbnN1bC5pb4IYY29udGVudC5zcGFjZWNyYWZ0ZWQu
Y29tghJkb2NzLnZhZ3JhbnR1cC5jb22CCmVsYXN0aWMuY2+CCGZvdW5kLm5vgg1m
b3h0b25zLmNvLnVrghJnYXplbGxlc3RhZ2luZy5jb22CEmdpdmUuZnVuZGVyYm9s
dC5pb4INaGFzaGljb25mLmNvbYIMaGFzaGljb25mLmV1gg1oYXNoaWNvcnAuY29t
gg9ub21hZHByb2plY3QuaW+CDm90dG9wcm9qZWN0LmlvgglwYWNrZXIuaW+CG3Nl
Y3VyZS10ZXJtaW51cy5wYW50aGVvbi5pb4IHc2VyZi5pb4Iic3RhZ2luZy1zZWN1
cmUuZG9sbGFyc2hhdmVjbHViLmNvbYIMdGVycmFmb3JtLmlvgg12YWdyYW50dXAu
Y29tgg92YXVsdHByb2plY3QuaW+CDXd3dy5jb25zdWwuaW+CEXd3dy5mcmVlbG90
dG8uY29tghF3d3cuaGFzaGljb25mLmNvbYIQd3d3Lmhhc2hpY29uZi5ldYIOd3d3
Lm1hZXN0cm8uaW+CE3d3dy5ub21hZHByb2plY3QuaW+CEnd3dy5vdHRvcHJvamVj
dC5pb4INd3d3LnBhY2tlci5pb4ILd3d3LnNlcmYuaW+CEHd3dy50ZXJyYWZvcm0u
aW+CEXd3dy52YWdyYW50dXAuY29tghN3d3cudmF1bHRwcm9qZWN0LmlvgiJ4dTRz
dXV0Nm9veWFlcGhvLndlc3RmaWVsZGxhYnMuY29tMB0GA1UdJQQWMBQGCCsGAQUF
BwMBBggrBgEFBQcDAjAdBgNVHQ4EFgQURQV0YPBxvZRZqJTcu6himTYwV1AwHwYD
VR0jBBgwFoAUqSuH4c4kRzsbv8+FNwJVnQ2UWOYwDQYJKoZIhvcNAQELBQADggEB
AIqVgpk8q2HknwXnzdXsD9C0tTU4gChbJ9W7kw4Lt7lS6HwwrW5lSskHIQMt29Qy
ZJHnQI/e4ezZCoki28bkdthodmt6uUpc83Jfpp+x4s0zHYmsfBaIMsX5+BZ9NCsR
5wLoBDUj020u6T5C/KEV2qAA6gG5vXun7jP3NJNYb0Ka+7fEOGyWPkBjJfyVeUO0
rGeZSi7bRnoOMllXLZ4rd0N/0Fq7oeiQjDROit+0UL9ybJwu4lvYskcjKvs/FJsT
+CuCU58I6IoezyLwLUm6npGTCAnq474yo+AjcqKLBsz5WYYKO5I1oVPF55Fafvds
HyKrsFeXDNghDFoBeIBI/NA=
-----END CERTIFICATE-----
subject=/C=US/ST=California/L=San Francisco/O=Fastly, Inc/CN=s.ssl.fastly.net
issuer=/C=BE/O=GlobalSign nv-sa/CN=GlobalSign CloudSSL CA - SHA256 - G3
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 3809 bytes and written 431 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: 233A1BDC9B84C8334964EFC93C5FC13546A6C389EBA15C7462FAC2D333464E28
    Session-ID-ctx: 
    Master-Key: 607780613AA7821EE4187E3D2ED35F6AD29EA5119158632781097D88F2AA6BE34CA6EB0D92AFE0F4D6F30C9B313A0D85
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 1200 (seconds)
    TLS session ticket:
    0000 - e1 8b a2 16 d3 c0 73 b3-e7 fd 57 a9 2d 9a 02 a3   ......s...W.-...
    0010 - 47 91 3f c1 1c 08 1f 5c-42 63 40 59 f1 f3 f5 8b   G.?....\Bc@Y....
    0020 - 68 57 09 91 0c 25 5a af-3a 4d 30 ea fb f1 bf 2f   hW...%Z.:M0..../
    0030 - eb c7 db a8 e2 67 55 72-07 ad 2a ce c0 08 87 df   .....gUr..*.....
    0040 - 14 8a ee 9f 6a 19 d7 c8-f5 c7 e9 14 48 0a 4e 9f   ....j.......H.N.
    0050 - db b2 dd 03 f5 29 9c 69-88 1a 92 68 55 20 85 32   .....).i...hU .2
    0060 - cf d1 d3 05 dd f1 88 f0-ec d8 e7 8f 74 2b 59 c9   ............t+Y.
    0070 - 30 1f ba f0 b6 5f c7 0c-28 fb 0f b0 4e 33 2f b9   0...._..(...N3/.
    0080 - 79 22 f5 75 85 80 c9 1f-f6 c1 5a 1f c9 d6 3f 23   y".u......Z...?#
    0090 - ea 9d 1f 2e b8 45 91 dc-5e 15 cb 15 fd dd 88 f0   .....E..^.......

    Start Time: 1496780474
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

[dflock]

I do have /etc/ssl/certs - with 533 certs, but I don't have any of the other folders mentioned in the error msg:

• /etc/pki/ca-trust/extracted/pem
• /etc/pki/tls/certs
• /usr/share/ca-certificates/cacert.org
• /etc/ansible

[ansibot]

cc @jpmens
click here for bot help

[dflock]

It works fine using requests:

Python 2.7.12 (default, Nov 19 2016, 06:48:10) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.get('https://releases.hashicorp.com/vagrant/')
<Response [200]>

using the latest version of requests:

$ pip freeze | grep requests
requests==2.17.3

It also works fine using curl:

$ curl -I https://releases.hashicorp.com/vagrant/

HTTP/1.1 200 OK
Cache-Control: max-age=3600
Content-Disposition: inline
Last-Modified: Thu, 25 May 2017 16:03:26 GMT
ETag: "80c92508985a629cb2f6ebb090241c26"
Content-Type: text/html; charset=utf-8
Via: 1.1 varnish
Fastly-Debug-Digest: 3d7c178238392b3e01e6866020851c0fa859e305fca4cc60da94a58e83ec761d
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Frame-Options: sameorigin
Content-Length: 3123
Accept-Ranges: bytes
Date: Tue, 06 Jun 2017 21:23:58 GMT
Via: 1.1 varnish
Age: 595
Connection: keep-alive
X-Served-By: cache-iad2148-IAD, cache-sea1029-SEA
X-Cache: HIT, HIT
X-Cache-Hits: 1, 1
X-Timer: S1496784239.922364,VS0,VE4
Vary: Accept-Encoding

[sivel]

I've done a little research here, and it is due to that host requiring TLS v1.2.

A few things to note. We are not specifically checking for TLS v1.2 since some hosts don't support it. create_default_context seems to be creating a context that is incompatible, and I'm not sure what series of changes will result in it working.

A proof of concept fix that I have test successfully is:

diff --git a/lib/ansible/module_utils/urls.py b/lib/ansible/module_utils/urls.py
index 74cb1a1..e2322e6 100644
--- a/lib/ansible/module_utils/urls.py
+++ b/lib/ansible/module_utils/urls.py
@@ -171,7 +171,10 @@ except ImportError:
 
 if HAS_SSL:
     # If we can't find extra tls methods, ssl.PROTOCOL_TLSv1 is sufficient
-    PROTOCOL = ssl.PROTOCOL_TLSv1
+    try:
+        PROTOCOL = ssl.PROTOCOL_TLSv1_2
+    except AtrributeError:
+        PROTOCOL = ssl.PROTOCOL_TLSv1
 if not HAS_SSLCONTEXT and HAS_SSL:
     try:
         import ctypes
@@ -698,7 +701,7 @@ class SSLValidationHandler(urllib_request.BaseHandler):
         if HAS_URLLIB3_PYOPENSSLCONTEXT:
             context = PyOpenSSLContext(PROTOCOL)
         else:
-            context = create_default_context()
+            context = SSLContext(PROTOCOL)
         if to_add_ca_cert_path:
             context.load_verify_locations(to_add_ca_cert_path)
         return context

But as create_default_context is supposed to be more secure, allowing the ssl module to choose the best settings, this will need investigated more closely.

[dflock]

Looks like it:

$ curl -sSfv https://releases.hashicorp.com/vagrant/

*   Trying 151.101.193.183...
* Connected to releases.hashicorp.com (151.101.193.183) port 443 (#0)
* found 173 certificates in /etc/ssl/certs/ca-certificates.crt
* found 704 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
* 	 server certificate verification OK
* 	 server certificate status verification SKIPPED
* 	 common name: s.ssl.fastly.net (matched)
* 	 server certificate expiration date OK
* 	 server certificate activation date OK
* 	 certificate public key: RSA
* 	 certificate version: #3
* 	 subject: C=US,ST=California,L=San Francisco,O=Fastly\, Inc,CN=s.ssl.fastly.net
* 	 start date: Tue, 25 Apr 2017 19:37:03 GMT
* 	 expire date: Thu, 29 Jun 2017 17:46:45 GMT
* 	 issuer: C=BE,O=GlobalSign nv-sa,CN=GlobalSign CloudSSL CA - SHA256 - G3
* 	 compression: NULL
* ALPN, server accepted to use http/1.1
> GET /vagrant/ HTTP/1.1
> Host: releases.hashicorp.com
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 200 OK

Appears similar to #23642, maybe - although I don't think the resolution/specifics apply here?

[sivel]

It's similar but no the same. In the other issue using a version of python with SNI support resolved the problem.

Here, that doesn't seem to affect it. It's a different set of requirements somewhere that I haven't had the time to figure out yet.

[sivel]

My initial assessment of simply picking the TLSv1.2 is incorrect. Using SSLContext directly, by default sets context.verify_mode = ssl.CERT_NONE which doesn't do verification.

I'll see if there is something in urllib3 in how they construct their context that is somehow different that allows this to work.

[sivel]

It seems on my host, I did not have a cacert bundle capable of validating this host. Installing the cacert bundle from https://curl.haxx.se/docs/caextract.html resolved my issue.

Can you verify that you have the most up to date ca-certificates package, and potentially, try utilizing the cacert bundle from the curl docs I mention above?

[dflock]

Interesting.

I do have the latest version of ca-certificates:

$ sudo apt install ca-certificates

Reading package lists... Done
Building dependency tree       
Reading state information... Done
ca-certificates is already the newest version (20160104ubuntu1).

...but, adding that extra .pem file does indeed fix the problem for me, too!:

$ cd /etc/ssl/certs/
$ sudo curl --remote-name --time-cond cacert.pem https://curl.haxx.se/ca/cacert.pem

$ ansible localhost -m get_url -a 'url=https://releases.hashicorp.com/vagrant/ dest=/tmp/download'
 [WARNING]: Host file not found: /etc/ansible/hosts

 [WARNING]: provided hosts list is empty, only localhost is available

localhost | SUCCESS => {
    "changed": false, 
    "dest": "/tmp/download", 
    "gid": 1000, 
    "group": "duncan", 
    "mode": "0664", 
    "msg": "file already exists", 
    "owner": "duncan", 
    "size": 3123, 
    "state": "file", 
    "uid": 1000, 
    "url": "https://releases.hashicorp.com/vagrant/"
}

I don't know enough about how this works to understand why it worked fine without this extra cert bundle with curl, requests, openssl - but not ansible - and then adding it fixes it for ansible?

[sivel]

requests uses certifi which provides it's own bundle.

I don't have immediate insight into what bundle or how curl was able to work, but I'll take a look, in case it helps resolve this problem in another way.

[sivel]

I'm having a difficult time re-producing this on an Ubuntu 16.04 host. A fresh install, with updated ca-certificates, works without any issue at all for me.

In my initial tests, I was using a Mac, that had accepted a certificate for GlobalSign into the login keychain that was interfering with the ability to verify the certificate. Explicitly specifying the cacert bundle from curl resolved that issue, as well as removing the offending cacert from the host.

On an Ubuntu host, we would look in the following locations for cacerts:

• /etc/ssl/certs
• /etc/pki/ca-trust/extracted/pem
• /etc/pki/tls/certs
• /usr/share/ca-certificates/cacert.org
• /etc/ansible

On a default Ubuntu install, only /etc/ssl/certs exists. It might be worth looking through that directory for anything not symlinked to /usr/share/ca-certificates/

[MiLk]

Same issue here, with same versions.

Ansible version

ansible 2.3.1.0
  config file =
  configured module search path = Default w/o overrides
  python version = 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]

OS / ENVIRONMENT

OS: Ubuntu 16.04.2 LTS (Xenial Xerus)
Python: 2.7.12
OpenSSL: 1.0.2g 1 Mar 2016

Error message

# ansible localhost -m get_url -a 'url=https://releases.hashicorp.com/vagrant/ dest=/tmp/download'

localhost | FAILED! => {
    "changed": false,
    "failed": true,
    "msg": "Failed to validate the SSL certificate for releases.hashicorp.com:443. Make sure your managed systems have a valid CA certificate installed. You can use validate_certs=False if you do not need to confirm the servers identity but this is unsafe and not recommended. Paths checked for this platform: /etc/ssl/certs, /etc/pki/ca-trust/extracted/pem, /etc/pki/tls/certs, /usr/share/ca-certificates/cacert.org, /etc/ansible. The exception msg was: (\"bad handshake: Error([('SSL routines', 'ssl3_read_bytes', 'tlsv1 alert protocol version')],)\",)."
}

[aurelienmaury]

Hi, I ran into the same bug I guess, and after testing on Debian8, Debian9 and CentOS7, compared libraries versions and so on. I found a little patch that solves the problem. Its my first dive into ansible code, so please measure impacts of this:

From b12c1b32653e76fbfeff216c5862888c304d22a6 Mon Sep 17 00:00:00 2001
From: "A. Maury" <aurelien.maury@wescale.fr>
Date: Thu, 29 Jun 2017 13:38:49 +0200
Subject: [PATCH] raised ssl context PROTOCOL for urls.py

---
 lib/ansible/module_utils/urls.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/ansible/module_utils/urls.py b/lib/ansible/module_utils/urls.py
index 9de91d2..ffcb945 100644
--- a/lib/ansible/module_utils/urls.py
+++ b/lib/ansible/module_utils/urls.py
@@ -170,8 +170,8 @@ except ImportError:
 # Exclude insecure ssl protocols if possible

 if HAS_SSL:
-    # If we can't find extra tls methods, ssl.PROTOCOL_TLSv1 is sufficient
-    PROTOCOL = ssl.PROTOCOL_TLSv1
+    # If we can't find extra tls methods, ssl.PROTOCOL_TLSv1_2 is sufficient
+    PROTOCOL = ssl.PROTOCOL_TLSv1_2
 if not HAS_SSLCONTEXT and HAS_SSL:
     try:
         import ctypes
--
2.10.1 (Apple Git-78)

If that is valid, either integrate it as-is or tell me if I need to make a formal pull-request. That would be great if that could make its path into the next 2.3.2.0 release.

[LaurentDumont]

Also ran into this bug with the vagrant download URL, which is using the Fastly CDN.

fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "Failed to validate the SSL certificate for releases.hashicorp.com:443. Make sure your managed systems have a valid CA certificate installed. You can use validate_certs=False if you do not need to confirm the servers identity but this is unsafe and not recommended. Paths checked for this platform: /etc/ssl/certs, /etc/pki/ca-trust/extracted/pem, /etc/pki/tls/certs, /usr/share/ca-certificates/cacert.org, /etc/ansible. The exception msg was: (\"bad handshake: Error([('SSL routines', 'ssl3_read_bytes', 'tlsv1 alert protocol version')],)\",)."}

I've tried using the patch from @aurelienmaury but it didn't seem to fix the problematic behavior.

System
Debian 9
Ansible 2.3.1.0
Python 2.7.13

[GrandPubba]

I believe I am running into this as well. Updating my cacerts as @sivel and @MiLk suggest did not fix the problem for me. What I noticed is that this was working on some of my nodes and not others. After a bit of work I determined that before installing urlib3==1.22 and pyOpenSSL==17.2.0 the following test works:

 ansible localhost -m get_url -a 'url=https://releases.hashicorp.com/vagrant/ dest=/tmp/download'

After installing these two packages the error appears.

[gnosek]

FWIW, I reinvented the same patch as @aurelienmaury when I hit the issue on Ubuntu 16.04 and it fixes the error for me. 14.04 worked fine recently but I didn't test it after running into the issue on 16.04.

[mtmiller]

This issue affects me and is easily reproducible on Debian, Fedora, and Ubuntu with ansible 2.3, 2.4, and master. The ansible target system must have both

• urllib3 >= 1.17
• pyOpenSSL

installed. The target remote server must only support TLSv1.2 and not fall back to TLSv1.

I ran into this with

$ ansible --version
ansible 2.5.0 (devel 6d16739926) last updated 2017/10/12 12:59:50 (GMT -700)
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/mike/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/mike/src/ansible/lib/ansible
  executable location = /home/mike/src/ansible/bin/ansible
  python version = 2.7.14 (default, Sep 29 2017, 12:52:50) [GCC 7.2.0]
$ ansible localhost -m get_url -a "url=https://repos.sonar.digitalocean.com/sonar-agent.asc dest=/tmp/out"
localhost | FAILED! => {
    "changed": false, 
    "failed": true, 
    "msg": "Failed to validate the SSL certificate for repos.sonar.digitalocean.com:443. Make sure your managed systems have a valid CA certificate installed. You can use validate_certs=False if you do not need to confirm the servers identity but this is unsafe and not recommended. Paths checked for this platform: /etc/ssl/certs, /etc/pki/ca-trust/extracted/pem, /etc/pki/tls/certs, /usr/share/ca-certificates/cacert.org, /etc/ansible. The exception msg was: (\"bad handshake: Error([('SSL routines', 'ssl3_read_bytes', 'tlsv1 alert protocol version')],)\",)."
}

The server repos.sonar.digitalocean.com and @dflock's example of releases.hashicorp.com are both servers that are not providing TLSv1 fallback, as verified with openssl:

$ openssl s_client -tls1 -connect releases.hashicorp.com:443
CONNECTED(00000003)
139736727127296:error:1409442E:SSL routines:ssl3_read_bytes:tlsv1 alert protocol version:../ssl/record/rec_layer_s3.c:1399:SSL alert number 70
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 102 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1
    Cipher    : 0000
    Session-ID: 
    Session-ID-ctx: 
    Master-Key: 
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1507839487
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
---

[BenCoffeed]

I had this issue as well with Hashicorp's servers. I followed @mtmiller's advice and checked the versions of urllib. I ran my playbook against 3 identical 16.04 servers and it only failed on one. The only difference I could fined on the failing server was that it had urllib3 version 1.22 while the rest were on urllib3 version 1.13.1. Manually forcing the problematic server to downgrade to urllib 1.13.1 resolved the issue. (sudo pip install urllib3==1.13.1 for you Googler's. )

[xenithorb]

I can confirm @mtmiller's assessment as being the correct one. I discovered this some moments ago and wrote it out in other tickets that were older but also resembled the problem (a link to spare you from the wall of text again.):

#18528 (comment)

---

In addition, in my case https://download.rocket.chat/stable is being hosted on CloudFront of all places, so I assume anyone with urllib3 and ansible >=2.3 is hosed.

[sivel]

@xenithorb although the issues are similar, the OPs problem here was different and not related. It was due to not having a correct cacert to validate the host.

A new issue should be opened.

[xenithorb]

d'oh, ok

[mtmiller]

Apologies, was not clear to me that this issue had a known cause and fix already and that I should have reported a new one.

[xenithorb]

@mtmiller @sivel - my new ticket: #31998