Logstash jruby-openssl tripping on expired DST Root CA X3

[jsvd]

With the expiration of the "DST Root CA X3" certificate Logstash is observing failures in multiple locations:

• building the docker parameter conversion tool with golang fails (fix update golang image #13260)
• the geoip download manager fails to connect to the endpoint (fix replace Faraday with Manticore to get rid of jruby-openssl verification error #13273)
• s3 output Impact of LetsEncrypt certificate expiry logstash-plugins/logstash-output-s3#239
• s3 input
• rss input logstash-input-rss cant fetch various RSS feeds due to Error: certificate verify failed logstash-plugins/logstash-input-rss#32
• ??

It may be that jruby-openssl doesn't prioritize non-expired certs when building a chain (jruby/jruby-openssl#236), but further investigation is required.
Depending on the ability to sort this out in jruby-openssl we may need to work around by using manticore instead of Faraday or Net::HTTP, since manticore uses java's httpclient and sidesteps jruby-openssl

[gose]

Is there a workaround until a fix is released?

[gose]

This doesn't fix the JRuby problem, but if you're using Let's Encrypt and hitting this issue, you can get around it by specifying the preferred chain to use with --preferred-chain "ISRG Root X1" as per:

jruby/jruby-openssl#236 (comment)

Before:

$ openssl s_client -showcerts -connect <your-domain.com>:443
CONNECTED(00000005)
depth=1 O = Digital Signature Trust Co., CN = DST Root CA X3
verify error:num=10:certificate has expired
notAfter=Sep 30 14:01:15 2021 GMT
verify return:0
...

After:

$ openssl s_client -showcerts -connect <your-domain.com>:443
CONNECTED(00000005)
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = <your-domain.com>
verify return:1
...