Skip to content

Problem with SSL certificate (works with Net::HTTPS) #79

Closed
CountCulture opened this Issue Jan 10, 2012 · 10 comments

2 participants

@CountCulture

Having a problem getting HTTPClient to work on an HTTPS site, but does work with plain old Net::HTTPS

Here's an example:

client = HTTPClient.new
client.get('https://wyobiz.wy.gov/Business/FilingDetails.aspx?FilingNum=2011-000611582')

produces:

at depth 0 - 20: unable to get local issuer certificate
OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed

Whereas:

uri = URI.parse('https://wyobiz.wy.gov/Business/FilingDetails.aspx?FilingNum=2011-000611582')
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
response = http.request_get('/Business/FilingDetails.aspx?FilingNum=2011-000611582')

works fine.

Using Ruby REE 1.8.7 on Debian Squeeze (OpenSSL 0.9.8o 01 Jun 2010)

Have also tried pointing to the Debian certs folder without any luck, and explicitly passing the site's certificate:

client.ssl_config.set_trust_ca("/etc/ssl/certs") #no change
client.ssl_config.set_trust_ca(Rails.root.join('config','certificates','us_wy.crt').to_s)

Any ideas?

Many thanks,
Chris

@nahi
Owner
nahi commented Jan 11, 2012

The site seems not set CA cert files properly. As you suspected, the difference from Net::HTTP is normally from local certificate settings.

Can you show me the result of following commands on the machine?

% ruby -ropenssl -e 'p [OpenSSL::X509::DEFAULT_CERT_DIR, OpenSSL::X509::DEFAULT_CERT_FILE]'
% env | grep -i cert
@CountCulture

The first gives:

["/usr/lib/ssl/certs", "/usr/lib/ssl/cert.pem"]

However, on debian /usr/lib/ssl/certs is a symlink of /etc/ssl/certs. There isn't a /usr/lib/ssl/cert.pem file.

The second returns nothing.

@nahi
Owner
nahi commented Jan 11, 2012

Hmm. As you said, 'client.ssl_config.set_trust_ca("/etc/ssl/certs")' should work, then.

Please try:
client = HTTPClient.new
client.ssl_config.clear_cert_store
client.ssl_config.set_trust_ca("/etc/ssl/certs")
p client.get("https://wyobiz.wy.gov/Business/FilingDetails.aspx?FilingNum=2011-000611582")

I'll obtain Debian Squeeze's ca-certificate package and try it by myself (I don't have Debian Squeeze box so I'm going to try that on my Ubuntu box).
client.ssl_config

@nahi
Owner
nahi commented Jan 11, 2012

I confirmed that ca-certificates downloaded from http://packages.debian.org/squeeze/ca-certificates does not help.

I should have said this before but the net/https client of your example does not work on Ubuntu box with or without ca-certificates of Debian Squeeze.

Can you show me 'p OpenSSL::SSL::VERIFY_PEER' just before your net/https client? I mean:

uri = URI.parse('https://wyobiz.wy.gov/Business/FilingDetails.aspx?FilingNum=2011-000611582')
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
response = http.request_get('/Business/FilingDetails.aspx?FilingNum=2011-000611582')
p [response, OpenSSL::SSL::VERIFY_PEER]
@CountCulture

OK. Just realised the Net::HTTP issue is a red herring. I had an initializer that included

OpenSSL::SSL::VERIFY_PEER = OpenSSL::SSL::VERIFY_NONE

to fix a problem with omniauth and Twitter. Once that is removed, Net::HTTP exhibits the same behaviour.

With

client = HTTPClient.new
client.ssl_config.clear_cert_store
client.ssl_config.set_trust_ca("/etc/ssl/certs")
p client.get("https://wyobiz.wy.gov/Business/FilingDetails.aspx?FilingNum=2011-000611582")

I get

at depth 0 - 20: unable to get local issuer certificate
OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed

WIth (and the initializer removed)

uri = URI.parse('https://wyobiz.wy.gov/Business/FilingDetails.aspx?FilingNum=2011-000611582')
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
response = http.request_get('/Business/FilingDetails.aspx?FilingNum=2011-000611582')
p [response, OpenSSL::SSL::VERIFY_PEER]

I get

 OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed

and

[nil, 1]
@CountCulture

So this may not be an HTTPClient issue (which by the way is a terrific library).

@nahi
Owner
nahi commented Jan 11, 2012

Bingo. Can you try to download https://gist.github.com/1594306 as 'cert.pem' (It's intermediate CA cert the site requires) then

client.ssl_config.add_trust_ca("/etc/ssl/certs") # Debian ca-certificates
client.ssl_config.add_trust_ca("cert.pem") # Above cert
client.get("...")

You should be able to access the site by this way (I can.)

And you should ask the server administrators to fix the mis-configured CA certificates of their SSL server. Can you?

@CountCulture

Yes, that works. Many thanks. Really appreciate your help.

Unfortunately don't have any contact with the server admin, but this is not the first time we've had probs with badly configured certificates. Could you explain where/how I'd get intermediate certificates for similar cases (or if this site changes certificate)?

@nahi
Owner
nahi commented Jan 11, 2012

I opened the site with a browser (Chrome this time) and click the "https" part of the address bar, then finds certificate the browser used, then export it as a file. Browsers usually have enough intermediate certificates to avoid this kind of problem.

Bear in mind that the file format OpenSSL can read is PEM (base64 + ---BEGIN/END--- lines). Some browsers could save an exported file as a DER format so you need to convert it with openssl command line. (openssl x509 -outform pem -out cert.pem -inform der -in cert.der)

I'm closing this issue since I believe you have a workaround now but please feel free to reopen or comment this.

@nahi nahi closed this Jan 11, 2012
@CountCulture

That's very helpful. Many thanks. I did try using the site's certificate, but exported from Firefox, and it seemed to save just the certificate itself (not the meta data). Yes, go ahead & close it, & once again thanks for your help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.