tornado rejects valid SSL certificate

[centromere]

When trying to download files using archive.extracted, I get the following error:

2015-09-30 08:30:53,649 [tornado.general  ][WARNING ][9181] SSL Error on 74 ('54.192.118.235', 443): [Errno 1] _ssl.c:510: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

This is the state in question:

consul-archive:
  archive.extracted:
    - name: /usr/local/bin/
    - source: https://dl.bintray.com/mitchellh/consul/0.5.2_linux_amd64.zip
    - source_hash: sha256=171cf4074bfca3b1e46112105738985783f19c47f4408377241b868affa9d445
    - archive_format: zip
    - if_missing: /usr/local/bin/consul
    - require:
      - pkg: unzip-package

Version report:

Salt Version:
           Salt: 2015.8.0

Dependency Versions:
         Jinja2: 2.7.2
       M2Crypto: Not Installed
           Mako: 0.9.1
         PyYAML: 3.10
          PyZMQ: 14.4.0
         Python: 2.7.6 (default, Jun 22 2015, 17:58:13)
           RAET: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.4
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 1.5
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
        libnacl: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.3.0
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
   python-gnupg: Not Installed
          smmap: Not Installed
        timelib: Not Installed

System Versions:
           dist: Ubuntu 14.04 trusty
        machine: x86_64
        release: 3.13.0-65-generic
         system: Ubuntu 14.04 trusty

[centromere]

The same issue happens with github as well:

2015-09-30 10:14:01,150 [tornado.general  ][WARNING ][9233] SSL Error on 57 ('54.231.0.233', 443): [Errno 1] _ssl.c:510: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

State:

consul-template-archive:
  archive.extracted:
    - name: /tmp/
    - source: http://github.com/hashicorp/consul-template/releases/download/v0.10.0/consul-template_0.10.0_linux_amd64.tar.gz
    - source_hash: sha256=ef298a2ae54cf51dbfc4108194299a9055b252ff9b917e7dd40c72fa30820096
    - archive_format: tar
    - tar_options: z
    - if_missing: /usr/local/bin/consul-template

(Github forces HTTPS)

[centromere]

This can be replicated by opening python and executing the following code:

import salt.utils.http
salt.utils.http.query('https://dl.bintray.com/mitchellh/consul/0.5.2_linux_amd64.zip', decode=False)

[Ch3LL]

@centromere thanks for the report. I was not able to reproduce this:

[ch3ll@thecakeisalie salt]$ python
Python 2.7.10 (default, Jul  5 2015, 14:15:43) 
[GCC 5.1.1 20150618 (Red Hat 5.1.1-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import salt.utils.http
>>> salt.utils.http.query('https://dl.bintray.com/mitchellh/consul/0.5.2_linux_amd64.zip', decode=False)
{}
>>> 
[ch3ll@thecakeisalie salt]$ git describe
v2015.8.0

[centromere]

@Ch3LL Do you have the pip package certifi installed? If so, can you tell me the version? Also, could you please run this command?:
sha256sum /usr/local/lib/python2.7/dist-packages/certifi/cacert.pem

[Ch3LL]

@centromere I was actually able to replicate this issue when I was following the instructions at https://docs.saltstack.com/en/latest/topics/development/hacking.html to setup a development setup for salt. But I cannot replicate the issue when I install the package version of salt for ubuntu 14.0 from the saltstack repository. How did you initially install salt? My guess is there is some dependency issue or a package needs to be updated.

My version of certifi: certifi (2015.9.6.2)
My output of sha256sum command

sha256sum /root/vsalt/lib/python2.6/site-packages/certifi/cacert.pem 
c18d747b3a34af964f7207dcb688b743d29167bf7fb696ff30f0c6b205a23b01  /root/vsalt/lib/python2.6/site-packages/certifi/cacert.pem

[centromere]

@Ch3LL salt-minion was installed on the nodes via salt-cloud (EC2), which uses salt-bootstrap. salt-master/salt-cloud was installed via package management, using this repository:

deb http://repo.saltstack.com/apt/ubuntu/ubuntu14 trusty main

Provisioner:

ec2:
  id: 'use-instance-role-credentials'
  key: 'use-instance-role-credentials'
  keyname: 'salt master'
  private_key: /etc/salt/salt-ssh.key
  location: us-west-2
  ssh_interface: private_ips
  ssh_username: ubuntu
  script_args: -P
  minion:
    master: salt-master.local
  driver: ec2

From salt-bootstrap.sh:

__ScriptVersion="2015.08.06"

Your cacert.pem is identical to mine (just wanted to rule that out).

[Ch3LL]

@centromere thanks for the additional information. I'll see if I can replicate this and will update this issue.

[eburghar]

affected by the same issue. using the certifi cacert.pem with openssl directly works as expected.

using saltstack 2015.8.0 on ubuntu 14.04.3

openssl s_client -CAfile /usr/local/lib/python2.7/dist-packages/certifi/cacert.pem -connect github.com:443

---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: 574ABDE700DAD076F50BA63EDA7DA249C72D4D787F3C71CBA4E3F0804370379A
    Session-ID-ctx: 
    Master-Key: BD9897A7264560FBFA8A8B5EA727EB925CBAA1CB3A9C9BEB39357F9CC8119680326B965C064CB37AB663EEC3BFE3DB9E
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1445419302
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

[bahadir]

I'm having same issue. I was also working on Consul when I hit this problem.

@centromere Your feedback is really appreciated: https://github.com/saltstack-formulas/consul-formula

[bahadir]

Installing certifi==2015.04.28 fixed my problem: certifi/python-certifi#26

[Ch3LL]

@centromere sorry for the late reply. I was able to replicate your issue on ubuntu 14, salt 2015.8.0. One thing to note is that once i upgraded to 2015.8.1 it started working. Can you try upgrading to 2015.8.1 and see if you are seeing the error still?

[cachedout]

@centromere Did you see the latest comment from @Ch3LL here?

[centromere]

@Ch3LL Hi, I will do so and get back to you on this.

[Ch3LL]

@centromere great thanks i appreciate it

[cachedout]

Because this is fixed in our testing and we haven't had a reply here, I am closing this. If this can be reproduced on the latest release of Salt, please leave a comment and we will re-open it. Thanks.