http2 error: stream error received: unspecific protocol error detected

[Jonny1987]

Bug report

• I confirm this is a bug with Supabase, not with my own application.
• I confirm I have searched the Docs, GitHub Discussions, and Discord.

Describe the bug

I get the following error in my edge function:

Error: request to https://www.googleapis.com/oauth2/v4/token failed, reason: error sending request for url (https://www.googleapis.com/oauth2/v4/token): http2 error: stream error received: unspecific protocol error detected
    at Gaxios._request (file:///tmp/sb-compile-edge-runtime/node_modules/localhost/gaxios/6.5.0/build/src/gaxios.js:143:19)
    at eventLoopTick (ext:core/01_core.js:168:7)
    at async GoogleToken._GoogleToken_requestToken (file:///tmp/sb-compile-edge-runtime/node_modules/localhost/gtoken/7.1.0/build/src/index.js:241:19)
    at async GoogleToken._GoogleToken_getTokenAsync (file:///tmp/sb-compile-edge-runtime/node_modules/localhost/gtoken/7.1.0/build/src/index.js:160:16)
    at async JWT.refreshTokenNoCache (file:///tmp/sb-compile-edge-runtime/node_modules/localhost/google-auth-library/9.8.0/build/src/auth/jwtclient.js:173:23)
    at async JWT.authorizeAsync (file:///tmp/sb-compile-edge-runtime/node_modules/localhost/google-auth-library/9.8.0/build/src/auth/jwtclient.js:154:24) {
  config: {
    method: "POST",
    url: "https://www.googleapis.com/oauth2/v4/token",
    data: {
      grant_type: "<<REDACTED> - See `errorRedactor` option in `gaxios` for configuration>.",
      assertion: "<<REDACTED> - See `errorRedactor` option in `gaxios` for configuration>."
    },
    headers: {
      "Content-Type": "application/x-www-form-urlencoded",
      "User-Agent": "google-api-nodejs-client/9.8.0",
      "x-goog-api-client": "gl-node/20.11.1",
      Accept: "application/json"
    },
    responseType: "json",
    retryConfig: {
      httpMethodsToRetry: [ "POST" ],
      currentRetryAttempt: 2,
      retry: 3,
      noResponseRetries: 2,
      statusCodesToRetry: [ [ 100, 199 ], [ 408, 408 ], [ 429, 429 ], [ 500, 599 ] ]
    },
    paramsSerializer: [Function: paramsSerializer],
    body: "<<REDACTED> - See `errorRedactor` option in `gaxios` for configuration>.",
    validateStatus: [Function: validateStatus],
    errorRedactor: [Function: defaultErrorRedactor]
  },
  response: undefined,
  error: FetchError: request to https://www.googleapis.com/oauth2/v4/token failed, reason: error sending request for url (https://www.googleapis.com/oauth2/v4/token): http2 error: stream error received: unspecific protocol error detected
    at HttpsClientRequest.<anonymous> (file:///tmp/sb-compile-edge-runtime/node_modules/localhost/node-fetch/2.7.0/lib/index.js:1501:11)
    at HttpsClientRequest.emit (ext:deno_node/_stream.mjs:1851:9)
    at node:http:467:16
    at eventLoopTick (ext:core/01_core.js:168:7) {
    message: "request to https://www.googleapis.com/oauth2/v4/token failed, reason: error sending request for url "... 116 more characters,
    type: "system",
    errno: undefined,
    code: undefined
  },
  [Symbol("gaxios-gaxios-error")]: "6.5.0"
}

To Reproduce

Create an edge function which contains the following code (service-account.json which downloaded from firebase when creating a new private key):

import { JWT } from "npm:google-auth-library@9";

const getAccessToken = (
  { clientEmail, privateKey }: { clientEmail: string; privateKey: string },
): Promise<string> => {
  return new Promise((resolve, reject) => {
    const jwtClient = new JWT({
      email: clientEmail,
      key: privateKey,
      scopes: ["https://www.googleapis.com/auth/firebase.messaging"],
    });
    jwtClient.authorize((err, token) => {
      if (err) {
        reject(err);
        return;
      }
      resolve(token!.access_token!);
    });
  });
};

  const { default: serviceAccount } = await import("../service-account.json", {
    with: { type: "json" },
  });

  const accessToken = await getAccessToken({
    clientEmail: serviceAccount.client_email,
    privateKey: serviceAccount.private_key,
  });

Expected behavior

No error

Screenshots

If applicable, add screenshots to help explain your problem.

System information

Additional context

No change to my edge function has occured and the error started sometime between 10th June and 18th June (my edge function was not run between these dates)

I have tried downgrading google-auth-library (versions 9.10 down to 9.0) but error still occured.

I have also run this code locally using the latest version of deno (1.44.2) and deno version 1.43.0 (same as current supabase edge-runtime) and no error occurs

[kurtguenther]

I am experiencing the same error. I have also not touched my supabase edge function and it is failing now (in my case a call to auth for google firebase fcm)

edit: If this helps for context, this is being run as a supabase branch for the first time

[laktek]

Do you get the same error when you run it locally with Supabase CLI? (latest stable release 1.176.10)

[Uglybeard]

I am experiencing too the exact same error. Edge functions were working until yesterday and today, without any changes, they stopped working giving this error as output.

I also leave here a message that was written by a user on the Supabase discord group, just about this error:

We have the same problem. Seems to be related to an update of the Deno runtime. I had an old version of the edge runtime locally (Version supabase-edge-runtime-1.54.2 with Deno 1.40.3) and with that it works as expected. Unfortunately, one can't change the runtime version in the Supabase Studio 😄

[laktek]

We'll investigate and post a fix for it.

@Uglybeard Do you also get the error with the same package? (ie. google-auth-library)

[Uglybeard]

I haven't checked what package is causing the error and unfortunately I can't reproduce it at this time, but I assume so since a code snippet very similar to the one posted by the author is also present in my code.

I followed the step-by-step procedure to implement the push notification in Supabase via FCM which contains the code posted by the author inside the file supabase/functions/push/index.ts and until yesterday everything was working perfectly, today I changed nothing and suddenly the Edge function stopped working giving the same error in the output.

[Jonny1987]

Do you get the same error when you run it locally with Supabase CLI? (latest stable release 1.176.10)

Yes it happens locally on 1.176.10 too

[Jonny1987]

In this old issue (denoland/deno#16923) which looks the same, someone mentioned using axios to make a request causes the same error and I can confirm this happen when running in an edge function, but not when running a script directly with deno (just like with my original code that using google-auth-library).

My edge function (which causes the error):

import axios from "npm:axios";
Deno.serve(async (req) => {
  await axios.get("https://google.com")
    .then(function (response) {
      // handle success
      console.log("success");
    }).catch((err) => {
      console.log("error", err.cause);
    });
  return new Response(
    JSON.stringify({}),
    {
      headers: { "Content-Type": "application/json" },
    },
  );
});

But running this script with deno run works (tried with deno version 1.43.0 and most recent version 1.44.2):

import axios from "npm:axios";
await axios.get("https://google.com")
.then(function (response) {
  // handle success
  console.log("success");
}).catch((err) => {
  console.log("error", err.cause);
});

[github-actions]

🎉 This issue has been resolved in version 1.54.6 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

[laktek]

We will deploy this release to all regions in the next 4-6 hours.