Intermittent error when verifying tokens ("Public key retrieved from google was not found or could not be parsed")

[joeljuca]

Hello Nick!

Thanks for writing and sharing ExFirebaseAuth! I'm using it on my project, and when I try to parse a Firebase token with ExFirebaseAuth.Token.verify_token/1, I get the following error:

{:error,
 "Public key retrieved from google was not found or could not be parsed"}

This is an intermittent scenario. It works sometimes, but once in a while I get this error. I'm not sure if it's an issue with my application, my build, or the library itself (I'm doing using anything other than decoding Firebase tokens, and I've properly configured the library as per docs says).

What could I do to fix, debug, or handle this kind of error?

[Nickforall]

Hi Joel, are you using the latest version? I have recently patched some issues that may be related to this.

[joeljuca]

Nick,

I believe I am. My mix.lock file shows the following line for ex_firebase_auth:

"ex_firebase_auth": {:hex, :ex_firebase_auth, "0.4.0", "2c855adc5af63910dbe912c260fe896f6bc5c967d32e76a4f9f5d2e3d003ad00", [:mix], [{:finch, "~> 0.6.3", [hex: :finch, repo: "hexpm", optional: false]}, {:jason, "~> 1.2.2", [hex: :jason, repo: "hexpm", optional: false]}, {:jose, "~> 1.10", [hex: :jose, repo: "hexpm", optional: false]}], "hexpm", "5f3a1af2826556f6c9c9456f7ca8a22204a2050a399183b87db653b16b480dd3"},

[joeljuca]

@Nickforall any clue on why it might be happening? I'm trying to debug your code (mostly the ExFirebaseAuth.KeySource.Google module), but since I'm not much experienced in Elixir I'm kinda lost on if/where could I start.

It seems the keys are being stored on the ETS table. I was able to check it, and it's there. I've checked the fetching and storing logic, and it seems fine.

Any suggestion on how to tackle it will be appreciated.

[Nickforall]

Ah thanks for looking that up. Version looks good indeed.

If the keys are present it's an error likely in ExFirebaseAuth.Token.get_public_key. Are you able to share the response of looking up the key in ETS.

Running

:ets.lookup(ExFirebaseAuth.KeyStore, "key identifier") |> IO.inspect()

Where key identifier is the kid field set in the header of the failing JWT token. You can find that field by pasting the JWT token on this site https://jwt.io/

[joeljuca]

I noticed something interesting: the keys returned from the Google API (endpoint: https://www.googleapis.com/robot/v1/metadata/x509/securetoken@system.gserviceaccount.com) are two, but there's one one in :ets.

Keys being returned from Google:

• 1de8067a(...)
• 2c8e2b29(...)

But then I can see only one of these keys in ETS (the 2c8e2b29 one):

:ets.first(ExFirebaseAuth.KeyStore) ==  :ets.last(ExFirebaseAuth.KeyStore) # => true

I'm not sure if it could be the problem, but it sounds like it's trying to use a key from ETS that hasn't been saved on GenServer properly (I'm checking it right now). Does these two keys need to be persisted on GS?

[Nickforall]

Both keys appear to be stored for me

iex(1)> :ets.lookup(ExFirebaseAuth.KeyStore, "1de8067a8298a4e3344b4dbded25f2fb4f40f3ce")
[
  {"1de8067a8298a4e3344b4dbded25f2fb4f40f3ce",
   %JOSE.JWK{
     fields: %{},
     keys: :undefined,
     kty: {:jose_jwk_kty_rsa,
      {:RSAPublicKey,
       25508669593564474024552777674588971472920440915151214730917034891541643605274620759316528055591468136661137520152594515799266531014792352467051812558759718253701274916961570450051977940078876540866807034755900397893743767729821776034347281804226663347871309953420303138970497142145548103282535712474656170759061616747904818234768797587000431430823479696758424263808719865263113404572079543540174336760263005282594729455496333252868461897332779892096102967768149760946636181515453210181352755808415199796061764682755851736443579504731293496183947727903952024163376295430559216168903882510680225192394422189350042523529,
       65537}}
   }}
]
iex(2)> :ets.lookup(ExFirebaseAuth.KeyStore, "2c8e2b296c6f3284ec30b8865d7293e627fa2b8f")
[
  {"2c8e2b296c6f3284ec30b8865d7293e627fa2b8f",
   %JOSE.JWK{
     fields: %{},
     keys: :undefined,
     kty: {:jose_jwk_kty_rsa,
      {:RSAPublicKey,
       23201029414087117862652297303138015744716970221023194014737088335173488623053669445118950777896686375145280397736719801585386103191715603537943277936222540841961380814749652592621245476142671898218399048549247964898530843412888690324221782782333188312388982278764033871484591743614113262601891652921518639600098187889918029300715734069201637798801480542357681640333674591855805519565142407445271198676123850744648196824392359019957123169918414953914088989160670319893455739807902909711974347754244271427745437696805081847397258439433970805476231323309864317884645909958048949905458702056091649093251520793764276120603,
       65537}}
   }}
]
iex(3)>

[joeljuca]

Sorry, I just checked it again, both keys seem to be in ETS now. Any, I'm afraid don't have the skills to debug it myself.

I'm not sure what else I could provide you to help tackle the issue, but I'm up for providing you more information that could help you solve it. Thank you for writing and sharing it in first place!

[Nickforall]

I’ll take the time to dive a bit deeper. Whenever you find a failing token can you test whether the kid of that token is in ETS or not? Thanks for your help already!

…

On Mon, 19 Apr 2021 at 19:11, Joel Jucá ***@***.***> wrote: Sorry, I just checked it again, both keys seem to be in ETS now. Any, I'm afraid don't have the skills to debug it myself. I'm not sure what else I could provide you to help tackle the issue, but I'm up for providing you more information that could help you solve it. Thank you for writing and sharing it in first place! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABGBWMXSFMGLR7PS3INQE5TTJRP5VANCNFSM43EV6HPQ> .

[Nickforall]

There may be another cause of this. Firebase recommends implementing a refresh time with the cache-control headers that are returned in the response of the keys endpoint.

There may be a tiny timeframe where ExFirebaseAuth is not yet refreshed while our cache of the keys must be invalidated according to those headers. But I think this is highly unlikely because the default refresh time is 5 minutes.

[lucasavila00]

I have been having this issue too. It's very rare, I couldn't find a pattern to reproduce it consistently.

But I think this is highly unlikely because the default refresh time is 5 minutes.

I see on other places it seems that the header must be respected: https://discuss.dgraph.io/t/authentication-using-jwkurl/10455/9

https://github.com/firebase/firebase-admin-go/blob/master/auth/token_verifier.go

[Nickforall]

I have not seen this error, and we've been running on production for quite some months now, so closing this as stale. Feel free to reopen if it reoccurs.

[akashvibhute]

Getting the same error consistently for last 30 minutes. First time using this library.

Version - {:ex_firebase_auth, "~> 0.5.1"}

ets table contains only one entry with following value -

iex(1)> :ets.first(ExFirebaseAuth.KeyStore)
"6a4f87ff5d93fa6ea03e5c6e88eea0acd2a232a9"
iex(2)> :ets.last(ExFirebaseAuth.KeyStore)
"6a4f87ff5d93fa6ea03e5c6e88eea0acd2a232a9"
iex(3)> :ets.lookup(ExFirebaseAuth.KeyStore, "6a4f87ff5d93fa6ea03e5c6e88eea0acd2a232a9")
[
  {"6a4f87ff5d93fa6ea03e5c6e88eea0acd2a232a9",
   %JOSE.JWK{
     fields: %{},
     keys: :undefined,
     kty: {:jose_jwk_kty_rsa,
      {:RSAPublicKey,
       26683466267166000117937282941287368499188964267704350320016815060755186299190334552341359030182847862883650390995706932990206917504303874597159717626156857776530120286608824298039026662712309062585296391513994028930201376912224740158628136408547035523323040966599847393117788904610391693639428199426775371439187431710883575502972086796633627213970033547315970319396410708150818633921757026450722384908347116277890243025314192440036812929920627983552722832658264926504359043778585873454116991087894748041249472749584754467425019137143154494316545575161017698890108258608397227982398909111303817013085192016995952648177,
       65537}}
   }}
]

Verify token returns

{:error, "Public key retrieved from google was not found or could not be parsed"}