Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RS256 Token Validation & Decoding using Public Key Not Working "ValueError: Could not deserialize key data." #359

Closed
tyrelkostyk opened this issue Jul 12, 2018 · 19 comments

Comments

@tyrelkostyk
Copy link

tyrelkostyk commented Jul 12, 2018

I'm trying to validate Google's ID Tokens for user authentication on a web app. The id token can be decoded fine if I disable verification, but won't verify when I pass it the RSA256 Public Key. The Public Key in question is Base64urlUInt-Encoded (RFC 7518 Specification).

The Entire Public Key Response

  {
   "kty": "RSA",
   "alg": "RS256",
   "use": "sig",
   "kid": "8c9eb968f73744eaed421e48010142bce51a067f",
   "n": "uweJ3hFY9wqZ6ZG-iSNhQwHtKCGl8G_jcQgGPjOrS-Rum3dyDjicqkAyfS8XDn480KD_TZ5m-lqBjqfimePu2_cH4URDPIwsqSzJI2_piEhaqnXRptIe5YB5imAL6iETKaOPjw284Fc7EdHK-ekHMn3AXjsy9AIErwAVw4-4ZXXwHbyQXJy1DyUB4ZzxiEvw_qkQmLdltmrNkLOw-Xh-C9UkTZ9NA58bYPBnxLwnAu_ggw_g_-hCAs6OvXZbAfFHhIGBLyjtdDLVrfXo1112QREB9d5sEds0bKZtJcD9afl4E7Ht6G-g3jNP2clAu6-6B-cIe4-j8Ph1uJDPkAmDfw",
   "e": "AQAB"
  }

According to the Specification I linked above, the "n" parameter is the Modulus, and the "e" parameter is the exponent. I've tried absolutely every combination of decoding these to common Base64 format, but no matter what I do, pyJWT doesn't like it.

Expected Result

A Verified, decoded JSON data packet.

Actual Result

Traceback (most recent call last):
  File "/anaconda3/lib/python3.6/site-packages/jwt/algorithms.py", line 205, in prepare_key
    key = load_pem_private_key(key, password=None, backend=default_backend())
  File "/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/primitives/serialization.py", line 20, in load_pem_private_key
    return backend.load_pem_private_key(data, password)
  File "/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1015, in load_pem_private_key
    password,
  File "/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1234, in _load_key
    self._handle_key_loading_error()
  File "/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1292, in _handle_key_loading_error
    raise ValueError("Could not deserialize key data.")
ValueError: Could not deserialize key data.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/anaconda3/lib/python3.6/site-packages/jwt/api_jwt.py", line 93, in decode
    jwt, key=key, algorithms=algorithms, options=options, **kwargs
  File "/anaconda3/lib/python3.6/site-packages/jwt/api_jws.py", line 157, in decode
    key, algorithms)
  File "/anaconda3/lib/python3.6/site-packages/jwt/api_jws.py", line 221, in _verify_signature
    key = alg_obj.prepare_key(key)
  File "/anaconda3/lib/python3.6/site-packages/jwt/algorithms.py", line 207, in prepare_key
    key = load_pem_public_key(key, backend=default_backend())
  File "/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/primitives/serialization.py", line 24, in load_pem_public_key
    return backend.load_pem_public_key(data)
  File "/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1041, in load_pem_public_key
    self._handle_key_loading_error()
  File "/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1292, in _handle_key_loading_error
    raise ValueError("Could not deserialize key data.")
ValueError: Could not deserialize key data.

Reproduction Steps

import jwt

IDjwt = <my id_token here>  # Decoding this with verify=False works correctly, so the problem isn't with the ID Token

GoogPubKey = b'uweJ3hFY9wqZ6ZG-iSNhQwHtKCGl8G_jcQgGPjOrS-Rum3dyDjicqkAyfS8XDn480KD_TZ5m-lqBjqfimePu2_cH4URDPIwsqSzJI2_piEhaqnXRptIe5YB5imAL6iETKaOPjw284Fc7EdHK-ekHMn3AXjsy9AIErwAVw4-4ZXXwHbyQXJy1DyUB4ZzxiEvw_qkQmLdltmrNkLOw-Xh-C9UkTZ9NA58bYPBnxLwnAu_ggw_g_-hCAs6OvXZbAfFHhIGBLyjtdDLVrfXo1112QREB9d5sEds0bKZtJcD9afl4E7Ht6G-g3jNP2clAu6-6B-cIe4-j8Ph1uJDPkAmDfw'

#Convert to Base64 (replace '-', '_' with '+', '/', respectively, and pad with '=' to make multiple of 4)
GoogPubKey = GoogPubKey.replace(b'-', b'+')
GoogPubKey = GoogPubKey.replace(b'_', b'/')
GoogPubKey += b'=='
len(GoogPubKey) % 4  # 0
GoogPubKey = b'-----BEGIN PUBLIC KEY-----\n' + GoogPubKey + b'\n-----END PUBLIC KEY-----'

decoded = jwt.decode(IDjwt, GoogPubKey, algorithms='RS256')

Again, I've tried over a dozen different ways of using this RSA Public Key Google supplies, but nothing works (Using Base64url like they provide, using or not using the 'BEGIN PUBLIC KEY' prefix/suffix, type bytes or str, adding the "AQAB" prefix in multiple places, nothing works)

Any help would be greatly appreciated!! Thank you

System Information

$ python -m jwt.help
{
  "cryptography": {
    "version": "2.1.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.4"
  },
  "platform": {
    "release": "17.4.0",
    "system": "Darwin"
  },
  "pyjwt": {
    "version": "1.6.4"
  }
}
@tyrelkostyk tyrelkostyk changed the title RSA256 Token Validation & Decoding using Public Key Not Working "ValueError: Could not deserialize key data." RS256 Token Validation & Decoding using Public Key Not Working "ValueError: Could not deserialize key data." Jul 12, 2018
@thomasliub
Copy link

thomasliub commented Jul 19, 2018

The same issue.

@jpadilla
Copy link
Owner

Try something like this:

import jwt
from jwt.algorithms import RSAAlgorithm

IDjwt = <my id_token here>  # Decoding this with verify=False works correctly, so the problem isn't with the ID Token

key_json = '{"kty": "RSA","alg": "RS256","use": "sig","kid": "4129db2ea1860d2e871ee48506287fb05b04ca3f","n": "sxorUSxfZZjQL1mDr1rtbNGJE9lbVMiBmNZFqLhnQaefTfqMO3YgSlb_cptw5wS2Dn4phGNzjBaO1Hg5572mEqsmPl5z9MmybIOuqWXxYyIiCGWH3hoR2VPJ-1bN-SdszHb4ZWadXCCYqnHS216nrvHZK8vJyQ7XCchw43O00LC5Iwi2eKspQEj8YDQSZFsd7Mp2ULhKXVPyKeLH06aenBZZFwgjw8bow7MXS4uUkg4NOeH2iHNxclOYycg6Z87QrTVzHGBo9r-6s1XRTFh-rqcZC8RnR62wkPqB2AEHctOof_ZtaaDTZ1Xw7db8dRhhCnFkpiK_1d8c9N2Vm7Frxw","e": "AQAB"}'

public_key = RSAAlgorithm.from_jwk(key_json)

decoded = jwt.decode(IDjwt, public_key, algorithms='RS256')

@mtairu
Copy link

mtairu commented Aug 8, 2018

@styk-tv
Copy link

styk-tv commented Sep 10, 2018

Hi. Arrived here after searching for the past 6 hours on the solution. My eyes are bleeding. My token is minted in Keycloak as asymmetric RS256.

Initial form retrieved from keycloak > realm > keys > rsa (and btw converted cert also matches public key), kid in the token matches kid in keycloak. its definitely the right key.

MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAq33wpseciIME/pakwslqucEAC6f/T9lN1OYaNhFN3cs/50KhWuPu8918JZFECvtby835CIyIEngKWLFr+VPbe5GW94dujvlaZJOj0eGst3t2gd6TOeu5FwzsAJWHNP725fu5SwGlN2J81fmYSYAWG1QNK3Bu5Fn5KD0gCN1MRD8gjC+hXHte904fdwRxZdLfQinaEyW1xwlItsJ1U9/Ve6hZbE4HMZeyeGPrJna//xWbNi9xCize32L+pepyeXWGmcTgq7++p9bXu6xtm/8Pmt5KkuLS+sE1Lrj19sZffjeJoy5q6tTXr8CAJT5qU+P9km4WAdKkb+2IlWMmGtHyQwIDAQAB

Above key is raw, without headers. I found out they must be present for the key to be valid as the Crypto is making it as far as https://github.com/pyca/cryptography/blob/master/src/cryptography/hazmat/backends/openssl/backend.py#L1029-L1052 and then kills conversion with error producing

  File "/usr/local/lib/python3.6/site-packages/jwt/algorithms.py", line 207, in prepare_key
    key = load_pem_public_key(key, backend=default_backend())
  File "/usr/local/lib/python3.6/site-packages/cryptography/hazmat/primitives/serialization.py", line 24, in load_pem_public_key
    return backend.load_pem_public_key(data)
  File "/usr/local/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1040, in load_pem_public_key
    self._handle_key_loading_error()
  File "/usr/local/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1291, in _handle_key_loading_error
    raise ValueError("Could not deserialize key data.")
ValueError: Could not deserialize key data.

I found that headers are quite important for decoding as decisions as to what decoder to use is based on the header. -----BEGIN PUBLIC KEY----- vs -----BEGIN RSA PUBLIC KEY----- ref: https://tls.mbed.org/kb/cryptography/asn1-key-structures-in-der-and-pem and also new lines are important based on historical issue #257

So tried all combinations, exactly like @tyrelkostyk "no header, begin public, begin rsa public and no go.

For completion sending a token as well (to match against pasted key above)

eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJoSjZYWnF2cWVuVjVIay1qWnV5TUN5bGNPT3FnTzNpWlYtMnVBRTVhMDYwIn0.eyJqdGkiOiI4YjA3YzUyZi0wMWYzLTRjY2QtOTg2Yi04ODk2OWYzZTViZmIiLCJleHAiOjE1MzY2MTM2OTYsIm5iZiI6MCwiaWF0IjoxNTM2NjEzNjM2LCJpc3MiOiJodHRwczovL3NpZ25pbi5kZW1vLm1hZGNvcmUuY2xvdWQvYXV0aC9yZWFsbXMvbWFzdGVyIiwiYXVkIjoibG9jYWxob3N0Iiwic3ViIjoiMjUzNDk1ZmUtZGY2MC00NDE5LWExMzQtZWZhZTFiMGE5OWQwIiwidHlwIjoiQmVhcmVyIiwiYXpwIjoibG9jYWxob3N0IiwiYXV0aF90aW1lIjowLCJzZXNzaW9uX3N0YXRlIjoiYjg3Mzg5MDItYjY3ZS00MmE5LTlhZjAtYjU4ZTlhZDE1NWVjIiwiYWNyIjoiMSIsImFsbG93ZWQtb3JpZ2lucyI6WyIqIl0sInJlYWxtX2FjY2VzcyI6eyJyb2xlcyI6WyJvZmZsaW5lX2FjY2VzcyIsInVtYV9hdXRob3JpemF0aW9uIl19LCJyZXNvdXJjZV9hY2Nlc3MiOnsiYWNjb3VudCI6eyJyb2xlcyI6WyJtYW5hZ2UtYWNjb3VudCIsIm1hbmFnZS1hY2NvdW50LWxpbmtzIiwidmlldy1wcm9maWxlIl19fSwic2NvcGUiOiJwcm9maWxlIGVtYWlsIiwiZW1haWxfdmVyaWZpZWQiOnRydWUsIm5hbWUiOiJQZXRlciBTdHlrIiwicHJlZmVycmVkX3VzZXJuYW1lIjoicG9sZmlsbSIsImdpdmVuX25hbWUiOiJQZXRlciIsImZhbWlseV9uYW1lIjoiU3R5ayIsImVtYWlsIjoicG9sZmlsbUBnbWFpbC5jb20ifQ.XPjYxLBlzo5tiaqEg3mwJwTpMaDzYleoeAUwc69ZrSqu-WtqIp6uSAPk_hsTnftTaCw9KSuCrvAi9csDjDswKST1rwUDpfSBOQ_lna_lxHMJFKgnVATpMmmoybyiAdn99kvISfrZ18nW8ECdBvOdJbPK-iI_WKq1NdxoilPPwOpsEAUT-hynK-astFoNQZn5mJegrAdU9LKho5D7Hs0A2I5i3nryyKJ2_Cak0HN8UB8BOBbiA_z0TWyeqY41VQFKqp-pNFMHltcQ3DeiSC1HK9bMjcy2AQ75ABy21xMlUtQJls_e000PgFClCpacayTCtj-udSHA5C23fRf3Qjsr5Q

Token is expired but the idea obviously to parse load public key instead before any other verifications. Two things: After this exercise i am certain I know far less than I thought I did. AND that this issue is most likely related to the formating of the public key. Decoding of token works fine obviously with verify=false

@styk-tv
Copy link

styk-tv commented Sep 10, 2018

screen shot 2018-09-11 at 00 35 42

jwt.io validates the key with -----BEGIN RSA PUBLIC KEY-----/-----END RSA PUBLIC KEY----- header styles hence in my case corresponds to PKCS#1

screen shot 2018-09-11 at 00 42 45

and going in, selected line represents last working line, everything fails on that check,
you can see in debugger pub key made it all the way here.

screen shot 2018-09-11 at 00 53 21

This could still be a formatting issue as far as I suspect since online verifier at jwt.io was javascript based hence implemented differently.

@styk-tv
Copy link

styk-tv commented Sep 11, 2018

@jpadilla Out of 1000's of messages i've read today, your hint WORKED.

@tyrelkostyk give up on passing public key yourself. Instead realy on OIDC well known config to discover exactly how you can construct RSA from JWK. It is there you don't have to guess it, at least it worked for me. Here is how I retrieved it.

Find your well-known config endpoint that lists all other endpoints. In case of keycloak its here (obviously realm will be different)

screen shot 2018-09-11 at 01 41 05

..then load "certs" endpoint

screen shot 2018-09-11 at 01 43 53

and then... use @jpadilla example above but with new token and detected jwk construction set

import jwt
from jwt.algorithms import RSAAlgorithm
IDjwt = 'eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJoSjZYWnF2cWVuVjVIay1qWnV5TUN5bGNPT3FnTzNpWlYtMnVBRTVhMDYwIn0.eyJqdGkiOiI3M2Y0NDk2NC03M2UxLTRmZTgtYmRlZi1mNGJiY2FkNDQ5NjUiLCJleHAiOjE1MzY2MjY2NzksIm5iZiI6MCwiaWF0IjoxNTM2NjI2MDc5LCJpc3MiOiJodHRwczovL3NpZ25pbi5kZW1vLm1hZGNvcmUuY2xvdWQvYXV0aC9yZWFsbXMvbWFzdGVyIiwiYXVkIjoibG9jYWxob3N0Iiwic3ViIjoiMjUzNDk1ZmUtZGY2MC00NDE5LWExMzQtZWZhZTFiMGE5OWQwIiwidHlwIjoiQmVhcmVyIiwiYXpwIjoibG9jYWxob3N0IiwiYXV0aF90aW1lIjowLCJzZXNzaW9uX3N0YXRlIjoiMWZmYTgzZGItZDg3MS00YjQzLTk5YjgtOTMzNDQxNTg1OTExIiwiYWNyIjoiMSIsImFsbG93ZWQtb3JpZ2lucyI6WyIqIl0sInJlYWxtX2FjY2VzcyI6eyJyb2xlcyI6WyJvZmZsaW5lX2FjY2VzcyIsInVtYV9hdXRob3JpemF0aW9uIl19LCJyZXNvdXJjZV9hY2Nlc3MiOnsiYWNjb3VudCI6eyJyb2xlcyI6WyJtYW5hZ2UtYWNjb3VudCIsIm1hbmFnZS1hY2NvdW50LWxpbmtzIiwidmlldy1wcm9maWxlIl19fSwic2NvcGUiOiJwcm9maWxlIGVtYWlsIiwiZW1haWxfdmVyaWZpZWQiOnRydWUsIm5hbWUiOiJQZXRlciBTdHlrIiwicHJlZmVycmVkX3VzZXJuYW1lIjoicG9sZmlsbSIsImdpdmVuX25hbWUiOiJQZXRlciIsImZhbWlseV9uYW1lIjoiU3R5ayIsImVtYWlsIjoicG9sZmlsbUBnbWFpbC5jb20ifQ.gaYPTNBLVYQJTRU5UHJ3GpVNKOrpekBIHmso6ZWLvaUJu4lgXf5wY1fdCqtsubTl2IlCy-zL81LABqua3He1M8qdAFvhKiTqyjm5SvzT40sjIDEnzCfUjYTvJIor8gcVscyNlqAuph0LNJb-aGu_tkaXjjLBl0DXqbyVyUtQ80ai9-ReCsmLobVLWeoyfi7hu9-elP6pOxaueIh5kT-MIux63xvvhSXBwQTxN9Dv7mNfSd1MCUUGtUJE1Fsb8dnIzpGjQh23Mw3p7SLq_ox_IhFP_mzRh3H8ye0gwVjILOOFUrCvFcylBS6TGhCZtkZP6luqRFbkCM9flQjQkON6lw'
key_json = '{"kid":"hJ6XZqvqenV5Hk-jZuyMCylcOOqgO3iZV-2uAE5a060","kty":"RSA","alg":"RS256","use":"sig","n":"q33wpseciIME_pakwslqucEAC6f_T9lN1OYaNhFN3cs_50KhWuPu8918JZFECvtby835CIyIEngKWLFr-VPbe5GW94dujvlaZJOj0eGst3t2gd6TOeu5FwzsAJWHNP725fu5SwGlN2J81fmYSYAWG1QNK3Bu5Fn5KD0gCN1MRD8gjC-hXHte904fdwRxZdLfQinaEyW1xwlItsJ1U9_Ve6hZbE4HMZeyeGPrJna__xWbNi9xCize32L-pepyeXWGmcTgq7--p9bXu6xtm_8Pmt5KkuLS-sE1Lrj19sZffjeJoy5q6tTXr8CAJT5qU-P9km4WAdKkb-2IlWMmGtHyQw","e":"AQAB"}'
public_key = RSAAlgorithm.from_jwk(key_json)
decoded = jwt.decode(IDjwt, public_key, algorithms='RS256')

TADA..

Thank you!!!

@styk-tv
Copy link

styk-tv commented Sep 12, 2018

FYI:

Openidc well.known endpoint for Azure:
doc: https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-protocols-oidc
https://login.microsoftonline.com/{tenant}/v2.0/.well-known/openid-configuration

@tyrelkostyk and for Google:
doc: https://developers.google.com/identity/protocols/OpenIDConnect#discovery
https://accounts.google.com/.well-known/openid-configuration

@sander-su
Copy link

Just for reference, I'm also using keycloak and for me the issue was line breaks
(windows vs linux) and
adding linebreaks (after 76 characters) in the keystring (besides adding the header and footer)

@vjsimha1
Copy link

Thank you @jpadilla! I was stuck for a long time but then came across this post and now I am able to successfully verify JWT token issued by AWS Cognito User Pool using the same guidelines.

Thanks once again!

@donspaulding
Copy link

@vjsimha1 Indeed, this is the solution I came up with for decoding AWS Cognito tokens:

import json
import jwt
import requests
from jwt.algorithms import RSAAlgorithm

def validate_cognito_token(id_token, cognito_region, cognito_user_pool_id, cognito_app_client_id):
    jwks = requests.get('https://cognito-idp.{aws_region}.amazonaws.com/{user_pool_id}/.well-known/jwks.json'.format(aws_region=cognito_region, user_pool_id=cognito_user_pool_id)).json()
    keys = {k['kid']: RSAAlgorithm.from_jwk(json.dumps(k)) for k in jwks['keys']}
    header = jwt.get_unverified_header(id_token)
    key_id = header['kid']
    algorithm = header['alg']
    pub_key = keys[key_id]
    # Next line raises errors if the audience isn't right or if the token is expired or has other errors.
    valid_token_data = jwt.decode(id_token, pub_key, audience=cognito_app_client_id, algorithm=algorithm)
    return valid_token_data

Does this look similar to your solution?

@styk-tv
Copy link

styk-tv commented Jan 23, 2019

@donspaulding @vjsimha1

I documented my process in this PR vimalloc/flask-jwt-extended#222
Take a look at the doc, that setup works with Keycloak but should be very similar to Cognito. As a matter of fact we should start a catalog of all the public OIDC autoconfig url's in a gist somewhere.

There is no point of making the call to JWK endpoint and then retrieve all elements independently since they can be fed all at once into the RSAAlgorithm and then that public key is ready to go for verification of signature. Its a two lines of code if you use something like this: https://github.com/vimalloc/flask-jwt-extended/blob/master/flask_jwt_extended/tokens.py#L115

@rudmac
Copy link

rudmac commented Feb 1, 2019

same problem when trying to validate firebase tokens with the public key https://www.googleapis.com/robot/v1/metadata/x509/securetoken@system.gserviceaccount.com

works only with verify=False

`
@app.post("/auth_firebase")
async def route_firebase_login_access_token(form_data: OAuth2FirebaseTokenRequestForm = Depends()):
token = form_data.token

header = jwt.get_unverified_header(token)

kid = header['kid']
alg = header['alg']
public_key = ""

with urllib.request.urlopen("https://www.googleapis.com/robot/v1/metadata/x509/securetoken@system.gserviceaccount.com") as url:
    data = json.loads(url.read().decode())
    public_key = data[kid]

payload = jwt.decode(
    token, 
    algorithms=alg, 
    verify=False
)

# buggy:
#payload = jwt.decode(
#    token, 
#    public_key, 
#    algorithms=alg
#)

return {
    "payload": payload
}

`
the token test:
http://jwt.io

@beaugogh
Copy link

beaugogh commented Jul 16, 2019

import jwt
from jwt.algorithms import RSAAlgorithm

IDjwt = <my id_token here>  # Decoding this with verify=False works correctly, so the problem isn't with the ID Token

key_json = '{"kty": "RSA","alg": "RS256","use": "sig","kid": "4129db2ea1860d2e871ee48506287fb05b04ca3f","n": "sxorUSxfZZjQL1mDr1rtbNGJE9lbVMiBmNZFqLhnQaefTfqMO3YgSlb_cptw5wS2Dn4phGNzjBaO1Hg5572mEqsmPl5z9MmybIOuqWXxYyIiCGWH3hoR2VPJ-1bN-SdszHb4ZWadXCCYqnHS216nrvHZK8vJyQ7XCchw43O00LC5Iwi2eKspQEj8YDQSZFsd7Mp2ULhKXVPyKeLH06aenBZZFwgjw8bow7MXS4uUkg4NOeH2iHNxclOYycg6Z87QrTVzHGBo9r-6s1XRTFh-rqcZC8RnR62wkPqB2AEHctOof_ZtaaDTZ1Xw7db8dRhhCnFkpiK_1d8c9N2Vm7Frxw","e": "AQAB"}'

public_key = RSAAlgorithm.from_jwk(key_json)

decoded = jwt.decode(IDjwt, public_key, algorithms='RS256')

just fyi, a simpler approach also worked for me;

import jwt
IDjwt = <my id_token here>
decoded = jwt.decode(IDjwt, '', verify=False)

@rSrkn
Copy link

rSrkn commented Sep 17, 2019

Just for reference, I'm also using keycloak and for me the issue was line breaks
(windows vs linux) and
adding linebreaks (after 76 characters) in the keystring (besides adding the header and footer)

We also faced the same problem.
Our tests on mac had not problems but jenkins tests (slave docker) had 'Could not deserialize key data' error.
After applying above solution (adding line breaks ...) problem solved.

@jordanmkoncz
Copy link

jordanmkoncz commented Dec 19, 2019

Just a note that I faced a similar issue and found a solution. In my case, I was using a signing certificate downloaded from Auth0 in PEM format, which I had incorrectly assumed was a public key. According to https://cryptography.io/en/latest/hazmat/primitives/asymmetric/serialization/#pem:

A PEM block which starts with -----BEGIN CERTIFICATE----- is not a public or private key, it’s an X.509 Certificate. You can load it using load_pem_x509_certificate() and extract the public key with Certificate.public_key.

My PEM file did start with -----BEGIN CERTIFICATE-----, so I had to implement what is suggested by the cryptograpy library. Here's some sample code for doing this:

import jwt
from cryptography import x509
from cryptography.hazmat.backends import default_backend
from django.conf import settings


def get_public_key_from_certificate(certificate_content: bytes) -> bytes:
    """
    Given a public signing certificate (e.g. downloaded from Auth0), load the certificate and then return the public key for it.

    :param certificate_content: The content of a public signing certificate in PEM format.
    :return: The public key.
    """
    certificate = x509.load_pem_x509_certificate(
        certificate_content, backend=default_backend()
    )

    return certificate.public_key()


def decode_access_token(access_token: str) -> dict:
    """
    Verify a JWT access token generated by Auth0, and return its decoded content.
    """
    public_key = get_public_key_from_certificate(settings.AUTH0_PUBLIC_CERTIFICATE)

    return jwt.decode(
        access_token,
        public_key,
        audience=settings.AUTH0_API_IDENTIFIER,
        algorithms=["RS256"],
        verify=True,
    )

Note that in this code, settings.AUTH0_PUBLIC_CERTIFICATE is a bytes value containing the content of the PEM key.

Now that I'm actually passing a public key into jwt.decode, everything works correctly.

See also: https://community.auth0.com/t/token-validation-with-python/21589/2.

@EduardSchwarzkopf
Copy link

@jpadilla Out of 1000's of messages i've read today, your hint WORKED.

@tyrelkostyk give up on passing public key yourself. Instead realy on OIDC well known config to discover exactly how you can construct RSA from JWK. It is there you don't have to guess it, at least it worked for me. Here is how I retrieved it.

Find your well-known config endpoint that lists all other endpoints. In case of keycloak its here (obviously realm will be different)

screen shot 2018-09-11 at 01 41 05

..then load "certs" endpoint

screen shot 2018-09-11 at 01 43 53

and then... use @jpadilla example above but with new token and detected jwk construction set

import jwt
from jwt.algorithms import RSAAlgorithm
IDjwt = 'eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJoSjZYWnF2cWVuVjVIay1qWnV5TUN5bGNPT3FnTzNpWlYtMnVBRTVhMDYwIn0.eyJqdGkiOiI3M2Y0NDk2NC03M2UxLTRmZTgtYmRlZi1mNGJiY2FkNDQ5NjUiLCJleHAiOjE1MzY2MjY2NzksIm5iZiI6MCwiaWF0IjoxNTM2NjI2MDc5LCJpc3MiOiJodHRwczovL3NpZ25pbi5kZW1vLm1hZGNvcmUuY2xvdWQvYXV0aC9yZWFsbXMvbWFzdGVyIiwiYXVkIjoibG9jYWxob3N0Iiwic3ViIjoiMjUzNDk1ZmUtZGY2MC00NDE5LWExMzQtZWZhZTFiMGE5OWQwIiwidHlwIjoiQmVhcmVyIiwiYXpwIjoibG9jYWxob3N0IiwiYXV0aF90aW1lIjowLCJzZXNzaW9uX3N0YXRlIjoiMWZmYTgzZGItZDg3MS00YjQzLTk5YjgtOTMzNDQxNTg1OTExIiwiYWNyIjoiMSIsImFsbG93ZWQtb3JpZ2lucyI6WyIqIl0sInJlYWxtX2FjY2VzcyI6eyJyb2xlcyI6WyJvZmZsaW5lX2FjY2VzcyIsInVtYV9hdXRob3JpemF0aW9uIl19LCJyZXNvdXJjZV9hY2Nlc3MiOnsiYWNjb3VudCI6eyJyb2xlcyI6WyJtYW5hZ2UtYWNjb3VudCIsIm1hbmFnZS1hY2NvdW50LWxpbmtzIiwidmlldy1wcm9maWxlIl19fSwic2NvcGUiOiJwcm9maWxlIGVtYWlsIiwiZW1haWxfdmVyaWZpZWQiOnRydWUsIm5hbWUiOiJQZXRlciBTdHlrIiwicHJlZmVycmVkX3VzZXJuYW1lIjoicG9sZmlsbSIsImdpdmVuX25hbWUiOiJQZXRlciIsImZhbWlseV9uYW1lIjoiU3R5ayIsImVtYWlsIjoicG9sZmlsbUBnbWFpbC5jb20ifQ.gaYPTNBLVYQJTRU5UHJ3GpVNKOrpekBIHmso6ZWLvaUJu4lgXf5wY1fdCqtsubTl2IlCy-zL81LABqua3He1M8qdAFvhKiTqyjm5SvzT40sjIDEnzCfUjYTvJIor8gcVscyNlqAuph0LNJb-aGu_tkaXjjLBl0DXqbyVyUtQ80ai9-ReCsmLobVLWeoyfi7hu9-elP6pOxaueIh5kT-MIux63xvvhSXBwQTxN9Dv7mNfSd1MCUUGtUJE1Fsb8dnIzpGjQh23Mw3p7SLq_ox_IhFP_mzRh3H8ye0gwVjILOOFUrCvFcylBS6TGhCZtkZP6luqRFbkCM9flQjQkON6lw'
key_json = '{"kid":"hJ6XZqvqenV5Hk-jZuyMCylcOOqgO3iZV-2uAE5a060","kty":"RSA","alg":"RS256","use":"sig","n":"q33wpseciIME_pakwslqucEAC6f_T9lN1OYaNhFN3cs_50KhWuPu8918JZFECvtby835CIyIEngKWLFr-VPbe5GW94dujvlaZJOj0eGst3t2gd6TOeu5FwzsAJWHNP725fu5SwGlN2J81fmYSYAWG1QNK3Bu5Fn5KD0gCN1MRD8gjC-hXHte904fdwRxZdLfQinaEyW1xwlItsJ1U9_Ve6hZbE4HMZeyeGPrJna__xWbNi9xCize32L-pepyeXWGmcTgq7--p9bXu6xtm_8Pmt5KkuLS-sE1Lrj19sZffjeJoy5q6tTXr8CAJT5qU-P9km4WAdKkb-2IlWMmGtHyQw","e":"AQAB"}'
public_key = RSAAlgorithm.from_jwk(key_json)
decoded = jwt.decode(IDjwt, public_key, algorithms='RS256')

TADA..

Thank you!!!

This was exactly what i needed. i also struggled with keycloak. i had to make a small change to the jwt.decode though to make it working on my side:
decoded = jwt.decode(token, public_key, algorithms="RS256", audience="account")

Thanks alot for this solution!

@vikramarsid
Copy link

I could get this working without a public/private key using options.

decoded_details = jwt.decode(jwt_id_token, options={"verify_signature": False})
python -m jwt.help
{
  "cryptography": {
    "version": "3.2.1"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.5"
  },
  "platform": {
    "release": "19.6.0",
    "system": "Darwin"
  },
  "pyjwt": {
    "version": "2.0.0a1"
  }
}

Note: "verify_signature": False is not recommended for production use.

@scottwn
Copy link

scottwn commented Jul 22, 2021

Working with IBM Cloud tokens, @jpadilla's suggestion above cleared the could not deserialize error for me; however I'm now getting Expecting a PEM-formatted key:

>>> r = requests.get('https://iam.cloud.ibm.com/identity/keys')
>>> key = r.json()['keys'][1]
>>> public = jwt.algorithms.RSAAlgorithm.from_jwk(key)
>>> decoded = jwt.decode(token,key,algorithms=key['alg'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/scott/.pyenv/versions/3.9.5/lib/python3.9/site-packages/jwt/api_jwt.py", line 119, in decode
    decoded = self.decode_complete(jwt, key, algorithms, options, **kwargs)
  File "/home/scott/.pyenv/versions/3.9.5/lib/python3.9/site-packages/jwt/api_jwt.py", line 90, in decode_complete
    decoded = api_jws.decode_complete(
  File "/home/scott/.pyenv/versions/3.9.5/lib/python3.9/site-packages/jwt/api_jws.py", line 149, in decode_complete
    self._verify_signature(signing_input, header, signature, key, algorithms)
  File "/home/scott/.pyenv/versions/3.9.5/lib/python3.9/site-packages/jwt/api_jws.py", line 233, in _verify_signature
    key = alg_obj.prepare_key(key)
  File "/home/scott/.pyenv/versions/3.9.5/lib/python3.9/site-packages/jwt/algorithms.py", line 260, in prepare_key
    raise TypeError("Expecting a PEM-formatted key.")
TypeError: Expecting a PEM-formatted key.
>>> key = r.json()['keys'][2]
>>> public = jwt.algorithms.RSAAlgorithm.from_jwk(key)
>>> decoded = jwt.decode(token,key,algorithms=key['alg'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/scott/.pyenv/versions/3.9.5/lib/python3.9/site-packages/jwt/api_jwt.py", line 119, in decode
    decoded = self.decode_complete(jwt, key, algorithms, options, **kwargs)
  File "/home/scott/.pyenv/versions/3.9.5/lib/python3.9/site-packages/jwt/api_jwt.py", line 90, in decode_complete
    decoded = api_jws.decode_complete(
  File "/home/scott/.pyenv/versions/3.9.5/lib/python3.9/site-packages/jwt/api_jws.py", line 149, in decode_complete
    self._verify_signature(signing_input, header, signature, key, algorithms)
  File "/home/scott/.pyenv/versions/3.9.5/lib/python3.9/site-packages/jwt/api_jws.py", line 233, in _verify_signature
    key = alg_obj.prepare_key(key)
  File "/home/scott/.pyenv/versions/3.9.5/lib/python3.9/site-packages/jwt/algorithms.py", line 260, in prepare_key
    raise TypeError("Expecting a PEM-formatted key.")
TypeError: Expecting a PEM-formatted key.

@dajiaji
Copy link
Contributor

dajiaji commented Jul 22, 2021

@scottwn At least, you should use public instead of key.

decoded = jwt.decode(token, public, algorithms=key['alg'])

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests