Guzzle SSL Certificate Verification Failure on CircleCI #3

Closed
evilantnie opened this Issue May 27, 2014 · 35 comments

Projects

None yet
@evilantnie

I'm trying to push coverage results from our CircleCI builds, but the test-reporter step fails at the end with the following error:

[Guzzle\Http\Exception\CurlException]                                        
  [curl] 60: server certificate verification failed. CAfile: /home/ubuntu/Sto  
  ck/www/protected/composer_vendor/guzzle/guzzle/src/Guzzle/Http/Resources/ca  
  cert.pem CRLfile: none [url] https://codeclimate.com/test_reports    

I've tried upgrading to the latest guzzle version in our composer.json, as well as ensuring that the box has running update-ca-certs on the box, but I still get this error. Any ideas?

@evilantnie

I've attempted to upgrade the project to Guzzle 4.0 which has an updated pem file (https://github.com/guzzle/guzzle/blob/master/src/cacert.pem) which fixed the certificate issue. However, the ApiClient code isn't entirely compatible with the newer guzzle client and is failing w/ another error: "Unknown error posting Test coverage data."

At this point I'm giving up on leveraging code climate for php coverage until the feature is better supported.

@pbrisbin
Member
pbrisbin commented Jun 2, 2014

I was able to upload a coverage file successfully in a test project. This test project is using guzzle/guzzle3@92d9934 which is the release commit for 3.9.1. What version is installed in your case?

@rbait
rbait commented Jun 20, 2014

I'm getting the same exception on Codeship's CI. Installed version of guzzle is guzzle/guzzle3@92d9934

@munkie
munkie commented Jun 22, 2014

I have the same exception on Travis-CI. I came up with workaround creating report json with test-rporter --stdout > codeclimate.json and then sending it using console curl:

CODECLIMATE_REPO_TOKEN=<token> bin/test-reporter --stdout > codeclimate.json
curl -X POST -d @codeclimate.json -H 'Content-Type: application/json' -H 'User-Agent: Code Climate (PHP Test Reporter v1.0.1-dev)' https://codeclimate.com/test_reports
@rbait
rbait commented Jun 22, 2014

Nice, works like a charm 👍

@pbrisbin pbrisbin added a commit that referenced this issue Jun 25, 2014
@pbrisbin pbrisbin Upgrade to guzzlehttp/guzzle ~4.0
This version is known to fix certificate issues in some environments.

Required changes:

- New GuzzleHttp namespace
- Catch different exception class
- Use a Stream request body

Closes #3.
936dba6
@pbrisbin
Member

@rbait @munkie I've pushed a branch (pb-upgrade-guzzle) which uses Guzzle 4.0. Could you test this version for me?

I had to add the following to my project's composer.json to try out the branch:

    "repositories": [
      {
        "type": "vcs",
        "url": "https://github.com/codeclimate/php-test-reporter"
      }
    ],
    "require-dev": {
        "codeclimate/php-test-reporter": "dev-pb-upgrade-guzzle"
    }

I'm not very familiar with PHP packaging, so there may be an easier way.

@munkie
munkie commented Jun 26, 2014

@pbrisbin I tried to build with your branch - https://travis-ci.org/crystalservice/samba/jobs/28482243
Result is the same curl error #60:

CURLE_SSL_CACERT (60)
Peer certificate cannot be authenticated with known CA certificates. 

Also job with PHP 5.3 failed because Guzzle requires php 5.4 and composer install failes to execute.

@pbrisbin
Member

@munkie Just pushed an update to use Guzzle 4.1 (not 4.0) which has an even newer cert file. Could you try that?

@sun
Contributor
sun commented Jun 30, 2014

Still an issue (travis-ci). File transfer fails due to CA cert verification error.

Given that cURL works, but Guzzle does not, the most simple fix is probably to configure/instruct Guzzle to not use its own CA file, but use the CA file of the operating system instead.

For Guzzle 3, pass the following config option into Client:

$config = ['ssl.certificate_authority' => 'system'];

For Guzzle 4, pass the following config option into Client:

$config = ['verify' => TRUE];

Alternatively, in case the final HTTP file transfer is the only usage of Guzzle in php-test-reporter, then it might be wise to even consider to

  1. drop Guzzle altogether (one less dependency), and
  2. either change the docs to use @munkie's script lines
    OR
    simply write a few lines of raw/native PHP code to perform a HTTP POST request (e.g., relying on the curl extension, which should be available everywhere), which should be more than sufficient for this use-case.
@pbrisbin
Member

Thanks for this information!

For now, I'm going to cut a new branch from master and do the (seemingly) quick fix of telling Guzzle 3 to use the system cert.

Ultimately, I like your last suggestion the best -- we really don't need that dependency. I'm not sure how quickly I can get that out the door though, not being a PHP developer myself. Hopefully that change to the client configuration will get things working in the meantime.

@pbrisbin
Member

OK, the branch pb-system-cert has been pushed and is ready to try.

/cc @munkie

@pborreli pborreli referenced this issue in FriendsOfPHP/pickle Jul 1, 2014
Closed

Added CodeClimate coverage #49

@sun
Contributor
sun commented Jul 1, 2014

Started some code to skip Guzzle. cURL reports the same error + code, but supplies a more helpful error message (emphasis mine):

Connected to codeclimate.com (199.182.122.73) port 443 (#0)
60: SSL certificate problem: self signed certificate in certificate chain
Closing connection 0

Did you install the wrong SSL certificate on codeclimate.com? Self-signed certs should not be used in production.

@pbrisbin
Member
pbrisbin commented Jul 1, 2014

/cc @noahd1 can you check on the cert used in production?

@sun thanks again for your efforts. If that were the issue though, why would it work for some and not others? Why would command-line curl work, but not Guzzle configured to use system certificates?

@sun
Contributor
sun commented Jul 1, 2014

@pbrisbin curl may be globally configured to ignore self-signed cert errors.

FWIW, the verbose curl shell output from travis-ci:

$ curl -v -X POST -d @codeclimate.json -H 'Content-Type: application/json' -H 'User-Agent: php-test-reporter' https://codeclimate.com/test_reports
* About to connect() to codeclimate.com port 443 (#0)
*   Trying 199.182.122.73... connected
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using RC4-SHA
* Server certificate:
*    subject: OU=Domain Control Validated; OU=EssentialSSL Wildcard; CN=*.codeclimate.com
*    start date: 2013-06-11 00:00:00 GMT
*    expire date: 2014-08-10 23:59:59 GMT
*    subjectAltName: codeclimate.com matched
*    issuer: C=GB; ST=Greater Manchester; L=Salford; O=COMODO CA Limited; CN=EssentialSSL CA
*    SSL certificate verify ok.
> POST /test_reports HTTP/1.1
> Host: codeclimate.com
> Accept: */*
> Content-Type: application/json
> User-Agent: php-test-reporter
> Content-Length: 3559
> Expect: 100-continue
> 

< HTTP/1.1 100 Continue
< HTTP/1.1 200 OK
< Content-Type: text/plain; charset=utf-8
< Transfer-Encoding: chunked
< Status: 200 OK
< Strict-Transport-Security: max-age=31536000
< X-Served-By: cc-app02.c45591.bbg
< X-Revision: 8af27e0c34a0570686f1e85fedacbfdb207b74ed
< X-UA-Compatible: IE=Edge,chrome=1
< ETag: "c8fa519244422aeae605c966a97252a2"
< Cache-Control: max-age=0, private, must-revalidate
< X-Request-Id: 053a7bdb85d0f4e00b86ae9da383b40b
< X-Runtime: 0.021619
< Date: Tue, 01 Jul 2014 14:13:05 GMT
< X-Rack-Cache: invalidate, pass
< X-Powered-By: Phusion Passenger
< Server: nginx + Phusion Passenger
< 

* Connection #0 to host codeclimate.com left intact
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):
@pbrisbin
Member
pbrisbin commented Jul 1, 2014

The only thing that sticks out:

Not working case (from Travis error):

CAfile: /etc/ssl/certs/ca-certificates.crt
CRLfile: none

Working case (from your curl output):

CAfile: none
CApath: /etc/ssl/certs
@pbrisbin
Member
pbrisbin commented Jul 1, 2014

We've confirmed that our cert is not self-signed and appears to be setup correctly. I'm not sure how to explain that error.

@sun is your code anywhere I could access it (or are you planning on opening a PR ;)?

In one of our other test reporters, we bundle our own cert so it probably makes sense to give up on more trouble-shooting and just do that.

I can figure out how to have Guzzle use a custom cert, but if your code is close, I'd love to incorporate that into that change.

@pbrisbin
Member
pbrisbin commented Jul 1, 2014

New branch: pb-bundled-cert ready to try.

@atulatri
atulatri commented Jul 8, 2014

Just tried pb-bundled-cert branch, I am getting same errors.

@pbrisbin
Member

Heads up to anyone currently testing pb- branches: We've determined these fixes don't work and I'm going to delete the branches soon. Please remove them from any composer.json files since an install would fail to find them.

@gunnarlium gunnarlium added a commit to aptoma/silex-extras that referenced this issue Aug 5, 2014
@gunnarlium gunnarlium [travis] Submit Code Climate with Curl c41ed1d
@arvenil arvenil added a commit to arvenil/ninja-mutex that referenced this issue Sep 30, 2014
@arvenil arvenil Workaround for bug codeclimate/php-test-reporter#3 29f2b4c
@amercier amercier added a commit to amercier/php-cli-helpers that referenced this issue Oct 25, 2014
@amercier amercier Workaround for bug codeclimate/php-test-reporter#3 0ea976a
@amercier amercier added a commit to amercier/php-cli-helpers that referenced this issue Oct 25, 2014
@amercier amercier Workaround for bug codeclimate/php-test-reporter#3 dc79742
@tyler-sommer tyler-sommer added a commit to nice-php/framework that referenced this issue Nov 17, 2014
@tyler-sommer tyler-sommer Workaround for bug codeclimate/php-test-reporter#3 5900a30
@renanivo renanivo added a commit to renanivo/authoritarian that referenced this issue Dec 8, 2014
@renanivo renanivo send coverage via Curl to Code Climate
to avoid a known Guzzle issue:
codeclimate/php-test-reporter#3
0ea8597
@enygma
enygma commented Jan 14, 2015

Okay, so there's something amiss here...when I execute this:

openssl s_client -connect codeclimate.com:443

I get the response:

CONNECTED(00000003)
depth=4 C = SE, O = AddTrust AB, OU = AddTrust External TTP Network, CN = AddTrust External CA Root
verify error:num=19:self signed certificate in certificate chain
verify return:0

Note the "self signed certificate in certificate chain" there. So, in going through some of this SSL certificate mess, I noticed something that could cause this error. I wonder if the intermediate certificates were installed correctly for the Code Climate wildcard domain. Most browsers apparently already come with the necessary intermediate certs so they don't error. @pbrisbin or @noahd1 - could you guys check on that and see if those were installed correctly?

This site doesn't show any problems though, so I'm not sure what's up here...
https://www.sslshopper.com/ssl-checker.html#hostname=https://codeclimate.com/test_reports:443

@ahmadnassri

any progress on this? seems like self-signed cert is not the way to go here ... wouldn't you want to use cert with a signing authority backing it?

@enygma
enygma commented Jan 15, 2015

@ahmadnassri it's not that Code Climate's cert is self-signed (it's signed just fine by EssentialSSL's External CA). There's just something in between that's causing issues...either the intermediate certs or something else. I'm not a SSL guru, so I'm just gathering what I can here. Anyone with more skills at identifying the issue is more that welcome to take a crack at it.

@enygma
enygma commented Jan 16, 2015

Okay, @ahmadnassri @atulatri @sun @munkie @evilantnie - can you folks do me a favor? Can you report back what this line gives you on the machines it's not working on?

php -i | grep 'cURL Information'

I'm starting to suspect different curl versions as Travis-CI still runs 7.22.0 and I seem to have 7.30.0.

@ahmadnassri

@enygma problem only on travis for me, local is: cURL Information => 7.35.0

@enygma
enygma commented Jan 20, 2015

I think this could be a curl issue...I'm going to try to build a box with 7.22 installed and see what happens.

@enygma
enygma commented Jan 20, 2015

Well, the bad news is that I compiled a VM with PHP using curl 7.22.0 and ran my test script that fails on Travis...and it worked. Back to the drawing board.

UPDATE: I tried it first with PHP 5.6.4 and it worked so I tried it with 5.4.35 and....it worked. I'm at a loss here.

@enygma
enygma commented Jan 20, 2015

After a bit more research (and more reading of the results) I noticed this in the cert chain from the above openssl command:

 1 s:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
   i:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root

The issuer and signer are the same. It looks like AddTrust has a self-signed cert for their External CA. That's where that message comes from but it doesn't seem like it should cause the issue with the Unexpected response: -60 server certificate verification failed message. hmm...

@ahmadnassri

wonder if poking any one from the travis team for their vms / settings would help give a repeatable path...

@enygma
enygma commented Jan 26, 2015

So I have a test branch of the php-test-reporter that removes the cURL dependency...I'd be interested to see if it cooperates for anyone else. It works local and on Travis-CI:

https://github.com/enygma/php-test-reporter/tree/remove-curl

I just tried it out on a sample app build and it did indeed report back the coverage data correctly.

@geshan
geshan commented Mar 29, 2015
after_script:
  - CODECLIMATE_REPO_TOKEN="..." bin/test-reporter --stdout > codeclimate.json
  - "curl -X POST -d @codeclimate.json -H 'Content-Type: application/json' -H 'User-Agent: Code Climate (PHP Test Reporter v0.1.1)' https://codeclimate.com/test_reports"

It there an easy way to run the above commands from a shell script, as the code coverage is just taken into account for master branch I generate the coverage only for master branch. But when I need to post the file I think due to "quotes" the curl does not find the file. How can I call it from a shell script?

@jonathancadepowers
Contributor

We have now resolved this issue, so it should no longer be necessary to use the cURL work-around.

Instead, you should now be able to use our standard configuration, by passing your clover.xml file to our PHP test reporter directly. This is true for all testing environments, including CIs.

If anyone else runs into a certificate issue, please let us know via our help form.

@nunofgs nunofgs referenced this issue in seegno/uphold-sdk-php Aug 10, 2015
Merged

Fix code climate SSL certificate error #95

@nunorafaelrocha

@jonathancadepowers I still get the certificate errors, but only on the travis CI. Does anyone still have this?

If this was fixed, shouldn't the workaround be removed from the README.md file?

@jonathancadepowers
Contributor

@nunorafaelrocha We have received a few new reports of this recently, so it looks like this issue is unfortunately not fully resolved for all projects, as we had originally thought. Very sorry about that. For now the cURL work-around will continue to work: https://github.com/codeclimate/php-test-reporter#known-issue-ssl-certificate-error

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment