1.1.0 causes SSL error downloading node with custom CA certs

[olivierlacan]

I've dug into this enough to know that installing any other version than Volta 1.1.0 resolves the issue.

I have a feeling Volta's removal of the OpenSSL dependency means it is now incompatible with zero-trust tools like Zscaler which require custom Certificate Authority certificates installed at the OS level (macOS Ventura 13.1 in my case).

sandbox (:|✔) $ volta -v
sandbox (:|✔) 1.0.4
sandbox (:|✔) $ volta install node
success: installed and set node@18.13.0 as default
   note: this version of Node includes npm@8.19.3, which is higher than your default version (8.13.0).
      To use the version included with Node, run `volta install npm@bundled`

After installing v1.1.0:

sandbox (:|✔) $ curl https://get.volta.sh | bash
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 10930  100 10930    0     0  28779      0 --:--:-- --:--:-- --:--:-- 29460
  Installing latest version of Volta (1.1.0)
    Checking for existing Volta installation
    Fetching archive for macOS, version 1.1.0
######################################################################## 100.0%
    Creating directory layout
  Extracting Volta binaries and launchers
    Finished installation. Updating user profile settings.
success: Setup complete. Open a new terminal to start using Volta!
sandbox (:|✔) $ volta install node@17
error: Could not download node@17.9.1
from https://nodejs.org/dist/v17.9.1/node-v17.9.1-darwin-arm64.tar.gz

Please verify your internet connection and ensure the correct version is specified.
Error details written to /Users/olivier-lacan/.volta/log/volta-error-2023-01-23_22_29_02.486.log

Indeed the swallowed error is an Invalid Peer Certificate error:

sandbox (:|✔) $ cat /Users/olivier-lacan/.volta/log/volta-error-2023-01-23_22_29_02.486.log
"volta" "install" "node@17"
Volta v1.1.0

Could not download node@17.9.1
from https://nodejs.org/dist/v17.9.1/node-v17.9.1-darwin-arm64.tar.gz

Please verify your internet connection and ensure the correct version is specified.

Error cause: Io Error: invalid peer certificate contents: invalid peer certificate: UnknownIssuer

Error cause: Io Error: invalid peer certificate contents: invalid peer certificate: UnknownIssuer

ZScaler is fully trusted in macOS Keychain on my system:

If Volta pinged my local openssl it should be getting the appropriate CA certs as my local (Homebrew-installed) openssl shows:

olivier-lacan (:|✔) $ openssl s_client -showcerts -servername nodejs.org -connect nodejs.org:443
CONNECTED(00000005)
depth=3 C = US, ST = California, L = San Jose, O = Zscaler Inc., OU = Zscaler Inc., CN = Zscaler Root CA, emailAddress = support@zscaler.com
verify return:1
depth=2 C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = Zscaler Intermediate Root CA (zscalertwo.net), emailAddress = support@zscaler.com
verify return:1
depth=1 C = US, ST = California, O = Zscaler Inc., OU = Zscaler Inc., CN = "Zscaler Intermediate Root CA (zscalertwo.net) (t) "
verify return:1
depth=0 CN = *.nodejs.org
verify return:1
write W BLOCK
---
Certificate chain
 0 s:/CN=*.nodejs.org
   i:/C=US/ST=California/O=Zscaler Inc./OU=Zscaler Inc./CN=Zscaler Intermediate Root CA (zscalertwo.net) (t)

Right now I'm not sure how to get volta to use the appropriate CA chain to make SSL requests under v1.1.0 so I have to downgrade and tell my teammates to avoid upgrading volta.

Could be related to #1035 but no one specified their versions in this issue thread so I can't be sure.

[trallnag]

To my understanding the latest version uses an integrated certificate store and there is no way to override it. A solution has already been merged, but there has been no release

[trallnag]

See this PR: #1375

[charlespierce]

@olivierlacan Thanks for reporting! As @trallnag said, this is very likely caused by Volta 1.1.0 using an internal certificate store rather than the system-native one. The newly-released Volta 1.1.1 includes the above fix to make Volta properly use the system-native certificates, so it should work for this case.

If it doesn't, please let us know so we can dig in deeper to figure out why we aren't getting the appropriate certificates!

[olivierlacan]

@charlespierce Thanks for the release. It fixed the issue for me on the install side, regardless of whether I had the Zscaler CA in my local OpenSSL bundle, so it seems like volta is appropriately deferring to OS-level certs.

sandbox (:|✔) $ volta --version
1.1.1
sandbox (:|✔) $ volta install node@15
success: installed and set node@15.14.0 as default
sandbox (:|✔) $ volta install node@14
  Fetching node@14.21.2  [=======================================>] 100%
success: installed and set node@14.21.2 as default

However there is an interesting 10+ second pause prior to the download progress bar actually moving which seems worth noting since I don't recall seeing downloads ever hang for that long. I wonder if the TLS handshake process is somehow delaying the download in some way. Below is a screencast demonstrating the issue. Occasionally downloads seem to pause midstream as well, and randomly resume a few seconds later. So maybe that rules out the handshake issue.

volta-1-1-1-node-download-hangs.mp4

I did check if I had any noticeable packet loss and couldn't isolate that as a variable:

PING nodejs.org (104.20.22.46): 56 data bytes
64 bytes from 104.20.22.46: icmp_seq=0 ttl=59 time=18.513 ms
64 bytes from 104.20.22.46: icmp_seq=1 ttl=59 time=31.938 ms
64 bytes from 104.20.22.46: icmp_seq=2 ttl=59 time=16.158 ms
64 bytes from 104.20.22.46: icmp_seq=3 ttl=59 time=15.720 ms
64 bytes from 104.20.22.46: icmp_seq=4 ttl=59 time=19.209 ms
64 bytes from 104.20.22.46: icmp_seq=5 ttl=59 time=13.683 ms
64 bytes from 104.20.22.46: icmp_seq=6 ttl=59 time=24.649 ms
64 bytes from 104.20.22.46: icmp_seq=7 ttl=59 time=21.855 ms
64 bytes from 104.20.22.46: icmp_seq=8 ttl=59 time=11.752 ms
64 bytes from 104.20.22.46: icmp_seq=9 ttl=59 time=19.013 ms
64 bytes from 104.20.22.46: icmp_seq=10 ttl=59 time=26.156 ms
^C
--- nodejs.org ping statistics ---
11 packets transmitted, 11 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 11.752/19.877/31.938/5.642 ms

This pausing behavior may be entirely unrelated and due to my system or my network so I'm closing this issue since it's effectively fixed. But it might be worth getting some macOS users to try and reproduce the hangs. If there are verbose debug flags I can pass to Volta without messing with the source so it can display verbose cURL output, let me know and I'll try with those.

[trallnag]

This sounds like some kind of enterprise security product that slows down the stream artificially to have more time for scanning. I guess this Zscaler certificate is part of a product that does TLS inspection?