Add verify_SSL=>1 to HTTP::Tiny in CPAN::HTTP::Client to verify https server identity

[stigtsp]

The verify_SSL flag is missing from HTTP::Tiny, and allows a network attacker to MITM the connection if it is used by the CPAN client.

Luckily, CPAN.pm seems to prefer curl (and maybe wget) over HTTP::Tiny when doing https. Those tools verify server identity by default.

Problem has been verified using mitmproxy on CPAN v2.34

MITM test with fix

cpan[1]> reload index
Reading '/root/.cpan/Metadata'
  Database was generated on Tue, 28 Feb 2023 11:17:02 GMT
Fetching with HTTP::Tiny:
https://cpan.org/authors/01mailrc.txt.gz
HTTP::Tiny failed with an internal error: SSL connection failed for cpan.org: SSL connect attempt failed error:0A000086:SSL routines::certificate verify failed

Fetching with HTTP::Tiny:
https://cpan.org/modules/02packages.details.txt.gz
HTTP::Tiny failed with an internal error: SSL connection failed for cpan.org: SSL connect attempt failed error:0A000086:SSL routines::certificate verify failed

Fetching with HTTP::Tiny:
https://cpan.org/modules/03modlist.data.gz
HTTP::Tiny failed with an internal error: SSL connection failed for cpan.org: SSL connect attempt failed error:0A000086:SSL routines::certificate verify failed

MITM test without fix

cpan[1]> reload index
Reading '/root/.cpan/Metadata'
  Database was generated on Tue, 28 Feb 2023 11:17:02 GMT
Fetching with HTTP::Tiny:
https://cpan.org/authors/01mailrc.txt.gz
Reading '/root/.cpan/sources/authors/01mailrc.txt.gz'
............................................................................DONE
Fetching with HTTP::Tiny:
https://cpan.org/modules/02packages.details.txt.gz
Reading '/root/.cpan/sources/modules/02packages.details.txt.gz'
  Database was generated on Tue, 28 Feb 2023 11:17:02 GMT
  HTTP::Date not available
............................................................................DONE
Fetching with HTTP::Tiny:
https://cpan.org/modules/03modlist.data.gz
Reading '/root/.cpan/sources/modules/03modlist.data.gz'
DONE
Writing /root/.cpan/Metadata

[stigtsp]

For more context:

• SSL environment controls, and warn about insecure connections chansen/p5-http-tiny#151
• https://hackeriet.github.io/cpan-http-tiny-overview/

[sjn]

I'm assuming there still are systems out there that don't use certificates properly (e.g. the may use self-signed certs and failed to place introduce them correctly into their setup).

Since CPAN.pm often is used for bootstrapping Perl, would it be more sensible to introduce a warning and/or environment variable override first, and perhaps change the verify_SSL attribute at some later point?

FWIW, I'm in favor of using this module as a tool to prod developers in the right direction when it comes to security issues (including certificate validation), but it seems to me that we may be asking a bit much if we expect a functioning CA chain of trust on a typical stripped-down minimal OS installation.

I'm happy if I'm proven wrong though; For example, if we can find out that most common & reasonably recent minimal OS images (e.g. container images) have /etc/ssl set up correctly, then I'd love to see verify_SSL set to 1.

Still I'm also wondering how much actually can break with this change.

Would you mind sharing some of your own thoughts on this, @stigtsp? 🙂

[stigtsp]

The concern is that HTTP::Tiny is used in some configurations to download packages from CPAN over https, without certificate verification.

Here is an example using the Perl image from Docker Hub. Note that this image has ca-certificates installed.

$ docker run --rm -it registry.hub.docker.com/library/perl bash -c "echo | cpan CPAN"
[..]
Fetching with HTTP::Tiny:
https://cpan.org/modules/02packages.details.txt.gz
[..]
Fetching with HTTP::Tiny:
https://cpan.org/modules/03modlist.data.gz
[..]
Fetching with HTTP::Tiny:
https://cpan.org/authors/id/A/AN/ANDK/CPAN-2.34.tar.gz

I'm unfamiliar with the bootstrap problem you are describing, but wouldn't it be possible to use plain http for those edge cases? It should be equally insecure as unverified https for downloading CPAN packages, imho.

[sjn]

The bootstrap problem I'm thinking of is related to a new developer's experience when trying out Perl for the first time. This is a PITA when SSL isn't set up correctly, and at this used to be the rule back in the day, when one really only had the choice to self-sign certificates or pay big money for a proper one. The situation has probably changed massively since letsencrypt became a thing, but may very well still be an issue in some situations...

[stigtsp]

I think this issue demonstrates that whenever CPAN.pm decides to use HTTP::Tiny for HTTPS requests, these are insecure and can be MITM'ed even if proper ca-certificates are installed on the system.

In addition to the official Docker perl image example, here is an example from Debian 11 with CPAN 2.34 installed (and autoconfigured):

root@debian:~# cpan Mojolicious
[..]
Fetching with HTTP::Tiny:
https://cpan.org/authors/id/S/SR/SRI/Mojolicious-9.31.tar.gz
[..]

Output from mitmproxy:

>> GET https://cpan.org/                                               
       ← 200 text/html 8.35k 71ms
   GET https://cpan.org/authors/01mailrc.txt.gz                        
       ← 200 application/x-gzip 259.29k 79ms
   GET https://cpan.org/modules/02packages.details.txt.gz              
       ← 200 application/x-gzip 2.36m 141ms
   GET https://cpan.org/modules/03modlist.data.gz                      
       ← 200 application/x-gzip 248b 50ms
   GET https://cpan.org/authors/id/S/SR/SRI/Mojolicious-9.31.tar.gz    
       ← 200 application/x-gzip 833.18k 99ms
   GET https://cpan.org/authors/id/S/SR/SRI/CHECKSUMS                  
       ← 200 113k 106ms

While wget correctly rejects a mitm'ed request against cpan.org:

root@debian:~# wget -O - https://cpan.org/
--2023-04-12 09:10:08--  https://cpan.org/
Resolving cpan.org (cpan.org)... 151.101.193.55, 151.101.65.55, 151.101.1.55, ...
Connecting to cpan.org (cpan.org)|151.101.193.55|:443... connected.
ERROR: The certificate of ‘cpan.org’ is not trusted.
ERROR: The certificate of ‘cpan.org’ doesn't have a known issuer.

[andk]

I find this a very reasonable patch, thank you. Given that we have a switch to turn off https altogether, we can afford to make https, where it is turned on, as best as we can. I'll make a 2.35-TRIAL release over the weekend.

[stigtsp]

9c98370

Thx for merging