On First run: Error: FetchError: certificate has expired

[peterlopen]

right after starting (mac os, catalina)
Checking for update
Error: FetchError: request to https://desktop.version.rancher.io/v1/checkupgrade failed, reason: certificate has expired
at ClientRequest. (/Applications/Rancher Desktop.app/Contents/Resources/app.asar/node_modules/node-fetch/lib/index.js:1461:11)
at ClientRequest.emit (events.js:315:20)
at TLSSocket.socketErrorListener (_http_client.js:469:9)
at TLSSocket.emit (events.js:315:20)
at emitErrorNT (internal/streams/destroy.js:106:8)
at emitErrorCloseNT (internal/streams/destroy.js:74:3)
at processTicksAndRejections (internal/process/task_queues.js:80:21)
selecting different k8s version did not helped.

There was an unknown error starting Kubernetes: FetchError: request to https://github-releases.githubusercontent.com/135516270/f179172f-7874-4e00-9be0-0e4531935272?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20211015%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211015T085144Z&X-Amz-Expires=300&X-Amz-Signature=b4fde258af6089f87ac507e6a373894334818643a75294e0a2ba7b7dd5b3584f&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=135516270&response-content-disposition=attachment%3B%20filename%3Dk3s&response-content-type=application%2Foctet-stream failed, reason: certificate has expired

[mook-as]

Hmm, given that both desktop.version.rancher.io and github-releases.githubusercontent.com had expired certificates (controlled by completely unrelated entities), the most likely scenarios I can think of are:

• There's an HTTPS proxy in the way with an expired certificate.
• The machine clock was set far in the future.

I'm not sure there's anything we can do in either of the cases (other than allowing for airgapped startup, which is a different topic). However, given a thumbs up from a different user on this issue (about a week after this was initially filed, if I'm reading the reactions data correctly), this isn't just a one-off configuration error.

Unfortunately I suspect it's been much too long and there's no way to get more context about what the issue may have been…

[peterlopen]

hi @mook-as , I tried today with version 0.6.1 and same problem:

kr, peter

[mook-as]

Thanks, Peter! Could you attach the logs to the issue? (Under the Troubleshooting tab, there's a Show Logs button.)

Also, would you be able to describe your set up a bit to help eliminate possible issues?

• Going by your original comment, I assume you're still on macOS / Catalina?
• Do you have any system-wide proxies set? (In System Preferences.app, open Network, click Advanced near the bottom right corner, and check the Proxies tab, for both Web Proxy (HTTP) and Secure Web Proxy (HTTPS). I believe this is per-connection.)
• As an additional check, see if you can open https://desktop.version.rancher.io/v1/checkupgrade?randomstuffhere in a Safari window? Please note that it's expected to be completely blank; it's only a problem if there's an error message.

Looking at the message, I find it interesting that the second error is about https://github-releases.githubusercontent.com — that means that:

1. it had managed get far enough to find the versions of k3s available, complete with the list of file names (we do a quick check to make sure the release has all the files we expect before trying to use it)
2. It actually started the download (the URLs are of the form https://github.com/k3s-io/k3s/releases/download/…) and got redirected.

Hmm, I guess those are the harder-to-cache URLs: the version check is a HTTP POST, and the other one is explicitly a short-lived URL (because of how GitHub works). That makes it more likely to be a proxy problem, so the additional check above might not show the issue (because that would be a GET instead). I've edited it to add parameters that are never used; when trying to re-test, please change it to other randomness.

[lfleal]

I'm having the same issue with both 0.5.0 and 0.6.1, I'm attaching logs. I also tested the URL in the log file with curl and that works fine. No complaints about expired certs. macos version 12.0.1

k8s.log
update.log
background.log

[mook-as]

Hmm, that one basically shows cert expired for https://api.github.com/ everywhere (but appears to have gotten past https://desktop.version.rancher.io/). That smells like it may be the possibilities I listed above, or a stale DNS entry (but curl rules that out).

Note that curl by default wouldn't use the system proxy settings (you would need to specify --proxy manually), so that's still on the table.

[lfleal]

I don't have a proxy and the URL is accessible with Firefox and Safari.

[lfleal]

Hmm, that one basically shows cert expired for https://api.github.com/ everywhere (but appears to have gotten past https://desktop.version.rancher.io/). That smells like it may be the possibilities I listed above, or a stale DNS entry (but curl rules that out).

Note that curl by default wouldn't use the system proxy settings (you would need to specify --proxy manually), so that's still on the table.

I reinstalled 0.6.1 and captured logs. I saw the following:
FetchError: request to https://api.github.com/repos/rancher-sandbox/rancher-desktop/releases/tags/v0.6.1

Including screenshot of the above in Safari.

background.log
k8s.log
update.log

[mook-as]

Hi! I've created #1487 with a build that provides extra logging about the certificates in this path. Would you be able to grab that and try it (with the debug mode on in the Troubleshooting tab)? It should generate a fetch.log in the logs directory.

(The mac builds are still running now, but Windows & Linux are done, so I don't expect any issues building.)

Thanks!

[peterlopen]

hi, sure. where I can find image from that build?
I was thinking whether problem might be somehow related to MITM corporate certificate...
thank you.
kind regards,
peter

[jandubois]

where I can find image from that build?

Scroll down to the artefacts list on the CI summary: https://github.com/rancher-sandbox/rancher-desktop/actions/runs/1792217979

[jandubois]

Note that the builds from the CI system are not signed, so you will get a popup about this.

On macOS you need to remove the quarantine bits after installing it:

$ xattr -d com.apple.quarantine /Applications/Rancher\ Desktop.app/

On Apple M1 systems you would also have to do some ad-hoc code signing, but I don't have the commands for that handy right now. Let me know if I need to look for them!

[peterlopen]

hi, actually, this seems to work for me:

never saw that one before.
I enabled debug mode, change version of k8s, here is fetch.log
fetch.log

regards

[peterlopen]

FYI: download of k8s version 1.23.2 got stuck in the middle (2 times) and I could not do anything else just manually kill rancher and qemu processes)

[jandubois]

It doesn't make sense to me that the server cert is expired, given that this has affected multiple domains and multiple users.

I suspect the problem is that we are loading all CA certs from the keychain into the app without validating them first. So when the server cert is "validated" by a CA or intermediate cert, the code may throw the "certificate is expired" error instead of continuing to search for a matching non-expired CA.

If this theory is correct, then the issue should be fixed by #1295

@mook-as Does that sound plausible?

[mook-as]

My understanding was that CA expiry dates don't matter (that is, if it's in the explicitly trusted list of roots, then when the CA cert itself is expired doesn't matter). However, I'm not very confident in that; we can certainly see if implementing #1295 helps with things.

[jandubois]

I don't know. Looking at electron/electron#31212 I see that an expired root cert has caused incorrect expiration errors before. However, that particular issue was fixed and backported to Electron 16 beta, so should not be the cause here.

Still, I have no other idea what might be causing the problem, and excluding expired (or untrusted) certs from the validation seems like a prudent choice regardless.

[mook-as]

This should be fixed in the next version (planned to be 1.1.0); please let us know if it wasn't once that's released. (Of if you want to try it early, grab a build from CI.)