Deno.createHttpClient CA issues "invalid peer certificate: UnknownIssuer"

[balopat]

Making calls to an HTTPS endpoint turns out to be very hard right now from supabase Edge functions, if the service uses a self-signed certificate.
Unfortunately this is really required in order to kick off for example a GKE job on Google Cloud. Any help would be greatly appreciated!

If anything this issue should be +1 evidence for Deno 2 becoming available ASAP in Supabase.

1. Everything works using Deno 2 (of course only locally).
2. Deno 1.45.2 has logging issues, so the true error is obscured due to Error messages from fetch became unfriendly after reimplementation with hyper denoland/deno#24835

Supabase is using a Deno 1.45.2 version under the hood right now.
This has the issue of hiding some details of HTTP requests

Error details: {
  message: "error sending request for url (https://<GKE URL>/version)",
  name: "TypeError",
  stack: "TypeError: error sending request for url (https://<GKE URL>/version)\n" +
    "    at async mainFetch (ext:deno_fetch/26_fetch.js:170:12)\n" +
    "    at async fetch (ext:deno_fetch/26_fetch.js:391:7)\n" +
    "    at async KubeConfigRestClient.performRequest (file:///<proj dir>/vendor/jsr.io/@cloudydeno/kubernetes-client/0.7.3/transports/via-kubeconfig.ts:162:18)\n" +
    "    at async K8sClient.connect (file:///<proj dir>/supabase/functions/shared/lib/k8s-client.ts:150:29)\n" +
    "    at async file:///<proj dir>/supabase/functions/planqtn_job/index.ts:198:7\n" +
    "    at async ext:deno_http/00_serve.ts:364:18",
  cause: undefined

3. Using Deno 1.44.4 outside of the edge runtime, I can finally debug the issue - which is "invalid peer certificate: UnknownIssuer" (again this does not happen with Deno 2 with the same exact code / parameters / CA certificate)

Error details: {
  message: "error sending request for url (https://<GKE URL>/version): error trying to connect: invalid peer certificate: UnknownIssuer",
  name: "TypeError",
  stack: "TypeError: error sending request for url (https://<GKE URL>/version): error trying to connect: invalid peer certificate: UnknownIssuer\n" +
    "    at async mainFetch (ext:deno_fetch/26_fetch.js:170:12)\n" +
    "    at async fetch (ext:deno_fetch/26_fetch.js:391:7)\n" +
    "    at async KubeConfigRestClient.performRequest (file:///<proj dir>/vendor/jsr.io/@cloudydeno/kubernetes-client/0.7.3/transports/via-kubeconfig.ts:162:18)\n" +
    "    at async K8sClient.connect (file:///<proj dir>/supabase/functions/shared/lib/k8s-client.ts:150:29)\n" +
    "    at async file:///<proj dir>/supabase/functions/planqtn_job/index.ts:198:7\n" +
    "    at async ext:deno_http/00_serve.ts:364:18",

4. DENO_CERT works on 1.44.4 and 1.45.2 - I found this idea based on here - Error "invalid peer certificate: UnknownIssuer" on Deno but not on Node denoland/deno#20362
5. DENO_CERT doesn't seem to work when I pass it in as an env var for my local Supabase instance (I checked that my pem file is in the right place in the Docker container, cat $DENO_CERT works) - and it doesn't work for the hosted version (I bundled the crt file into my function as a static file, I could read it with Deno.readfile, but DENO_CERT does not pick it up).

--unsafely-ignore-certificate-errors ?

At this point I'm a bit desperate and I'd be willing to turn off cert verification until Deno 2 comes out. But I don't think it's supported by Supabase. I managed to vendor in the kubernetes-client and maybe that would be a direction - but I don't see any way to force "--unsafely-ignore-certificate-errors" through the createHttpClient

Please comment if you have any ideas.

[GaryAustin1]

As far as Deno 2.
https://supabase.com/docs/guides/functions/deno2

[balopat]

Yeah, I know, however "Deno 2 will soon become the default choice for creating new functions. For now, Deno 2 is available in preview mode for local development." - I wonder when that will be?

[GaryAustin1]

My bad. Missed the context of your question. I have no insight into Supabase's schedule for this.
You might comment on this thread https://github.com/orgs/supabase/discussions/34054 with a Supabase dev on it.

[balopat]

Thanks, I added a comment. 🤞 that it'll be out soon.

[balopat]

FWIW I went with Cloud Run which has stable managed certificates, and no need for self-signed certs. GKE API server endpoints are probably not meant for direct calling from the outside, the certs are rotated regularly, etc. If someone wants to have access to a GKE server from the outside a dedicated endpoint with a stable public IP + managed certificate would be a potential option. And now it seems Deno 2.1 is in preview mode for prod too: https://github.com/orgs/supabase/discussions/36814