Add message about lack of X.509 certificate support in documentation

[anjorinjnr]

I get this error when using algorithms='RS256' on google app engine.

Full stack trace

Traceback (most recent call last):
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/python/request_handler.py", line 226, in handle_interactive_request
    exec(compiled_code, self._command_globals)
  File "<string>", line 12, in <module>
  File "lib/jose/jwt.py", line 121, in decode
    payload = jws.verify(token, key, algorithms, verify=verify_signature)
  File "lib/jose/jws.py", line 75, in verify
    _verify_signature(signing_input, header, signature, key, algorithms)
  File "lib/jose/jws.py", line 218, in _verify_signature
    key = jwk.construct(key, alg)
  File "lib/jose/jwk.py", line 65, in construct
    return RSAKey(key_data, algorithm)
  File "lib/jose/jwk.py", line 201, in __init__
    raise JWKError(e)
JWKError: RSA key format is not supported

[mpdavis]

Can you provide an example JWT and key that you are getting the error for?

[anjorinjnr]

jwt:
eyJhbGciOiJSUzI1NiIsImtpZCI6ImY0YjBhNWM3M2FkODVhNWRhMDlmMGU3Zjc2NDYzNjMxMzM5ZTBiYmYifQ.eyJpc3MiOiJodHRwczovL3NlY3VyZXRva2VuLmdvb2dsZS5jb20vd2Vkb3RyYW5zZmVyLTIwMTYiLCJhdWQiOiJ3ZWRvdHJhbnNmZXItMjAxNiIsImF1dGhfdGltZSI6MTQ2NzM0NjI3MCwidXNlcl9pZCI6IjRjemVXVllIekNNVnN0WEZOYldHVXBKYmJTZzEiLCJzdWIiOiI0Y3plV1ZZSHpDTVZzdFhGTmJXR1VwSmJiU2cxIiwiaWF0IjoxNDY3MzQ2MjcwLCJleHAiOjE0NjczNDk4NzAsImVtYWlsIjoic2V1bkBjbXUuY29tIiwiZW1haWxfdmVyaWZpZWQiOmZhbHNlLCJmaXJlYmFzZSI6eyJpZGVudGl0aWVzIjp7InBhc3N3b3JkIjpbInNldW5AY211LmNvbSJdLCJlbWFpbCI6WyJzZXVuQGNtdS5jb20iXX19fQ.U-fYjx8rMm5tYV24r0uEcNQtIe3UKULxsHecLdGzTbi1v-VKzKDk_QPL26SPDoU8JUMY3nJQ1hOE9AapBrQck8NVUZSKFMD49XdtsyoN2kKdinpFR1hSxIE0L2dRStS7OZ8sGiX866lNa52Cr6TXSsnMD6N2P0OtVE5EeD1Nf-AiJ-gsaLrP4tBnmj1MNYhEYVHb6sAUrT3nEI9gWmeKcPWPfn76FGTdGWZ2mjdaeAG4RbuFL4cHdOISA_0HVLGJxuNyEHAHybDX8mVdNW_F4yzL3H-SmPFY5Kv3tCdBzpzhUKfNOnFFmf2ggFOJnDsqMp-TZaIPk6ce_ltqhQ0dnQ

key:

-----BEGIN CERTIFICATE-----
MIIDHDCCAgSgAwIBAgIIWDhBeVUilCcwDQYJKoZIhvcNAQEFBQAwMTEvMC0GA1UE
AxMmc2VjdXJldG9rZW4uc3lzdGVtLmdzZXJ2aWNlYWNjb3VudC5jb20wHhcNMTYw
NzAxMDA0NTI2WhcNMTYwNzA0MDExNTI2WjAxMS8wLQYDVQQDEyZzZWN1cmV0b2tl
bi5zeXN0ZW0uZ3NlcnZpY2VhY2NvdW50LmNvbTCCASIwDQYJKoZIhvcNAQEBBQAD
ggEPADCCAQoCggEBALRWaRmoi5EFyj5TBrUGKFI6uBJ4x9wSHq9tlRL1qmnwzdNb
lDoeoh6Gw3H54IqM0XqjZZwgV5KXOQDOaoUpMBRH93x7Ma7NjhiDtpQr0JSbFIQL
sIay/VxQ9gfa/I83HViEAbF1FXjhBKniwFKUv26mU30upZfsDQkHM8OLc/iXRvhA
Yn7S732Oefdv0kJ9t3h+WOGKGVkYfDaAGn5Uyzx+9oyyLY33borKOBBzphSQlZCr
L569zTXvvLgvdStrsPGaiRGj64DGXD6LCg6acLJcMUvlVUO6THHJHVgp8pzlrPQG
3B1rZk61lZqJyjK/nTi2tY9GPLfdxOfDAMjNoz8CAwEAAaM4MDYwDAYDVR0TAQH/
BAIwADAOBgNVHQ8BAf8EBAMCB4AwFgYDVR0lAQH/BAwwCgYIKwYBBQUHAwIwDQYJ
KoZIhvcNAQEFBQADggEBAIlFwO3C+X2+na0nLjR+zQYGHzZYqFe4V67P6ugFJxun
xP8pyDCYAGer1mkDcIyDacdQ3natNp0xv61a0yk5tSmDYZbXZRTFdLkf/GzH+VmH
EMl5W4TvxjAe/x2opm3QUaPC+jVlvndcP99FF5ULFp7/PwSTp8uzyrd/fhSFaxhq
uIW4syNzDSpDItzUsiKCtsKGYX/qvd/cNP8cXlPd5rWTM4Sic9Baf2nXuHaZRkBr
SJYcxdh8xbGsY1tC8TIgWot6GXtldNvXDLqRUwb2t6Rr3Tqhbc0CcHndTCuHXf0i
0s9jU/UCrNhhmaD0rZLHQ2tuN6W/xpOHKtO0a8Lys7c=
-----END CERTIFICATE-----


Thanks.

[mpdavis]

It appears that you are trying to verify the token with an X.509 certificate, as opposed to a public key. Unfortunately, PyCrypto doesn't support X.509 using certificates.

The certificate contains the public key and it can be converted with openssl.

> openssl x509 -pubkey -noout < cert.pem
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtFZpGaiLkQXKPlMGtQYo
Ujq4EnjH3BIer22VEvWqafDN01uUOh6iHobDcfngiozReqNlnCBXkpc5AM5qhSkw
FEf3fHsxrs2OGIO2lCvQlJsUhAuwhrL9XFD2B9r8jzcdWIQBsXUVeOEEqeLAUpS/
bqZTfS6ll+wNCQczw4tz+JdG+EBiftLvfY5592/SQn23eH5Y4YoZWRh8NoAaflTL
PH72jLItjfduiso4EHOmFJCVkKsvnr3NNe+8uC91K2uw8ZqJEaPrgMZcPosKDppw
slwxS+VVQ7pMcckdWCnynOWs9AbcHWtmTrWVmonKMr+dOLa1j0Y8t93E58MAyM2j
PwIDAQAB
-----END PUBLIC KEY-----

Using that public key allows me to verify the given token (ignoring that it is expired)

>>> from jose import jwt
>>> key = """-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtFZpGaiLkQXKPlMGtQYo
Ujq4EnjH3BIer22VEvWqafDN01uUOh6iHobDcfngiozReqNlnCBXkpc5AM5qhSkw
FEf3fHsxrs2OGIO2lCvQlJsUhAuwhrL9XFD2B9r8jzcdWIQBsXUVeOEEqeLAUpS/
bqZTfS6ll+wNCQczw4tz+JdG+EBiftLvfY5592/SQn23eH5Y4YoZWRh8NoAaflTL
PH72jLItjfduiso4EHOmFJCVkKsvnr3NNe+8uC91K2uw8ZqJEaPrgMZcPosKDppw
slwxS+VVQ7pMcckdWCnynOWs9AbcHWtmTrWVmonKMr+dOLa1j0Y8t93E58MAyM2j
PwIDAQAB
-----END PUBLIC KEY-----"""
>>> token = "eyJhbGciOiJSUzI1NiIsImtpZCI6ImY0YjBhNWM3M2FkODVhNWRhMDlmMGU3Zjc2NDYzNjMxMzM5ZTBiYmYifQ.eyJpc3MiOiJodHRwczovL3NlY3VyZXRva2VuLmdvb2dsZS5jb20vd2Vkb3RyYW5zZmVyLTIwMTYiLCJhdWQiOiJ3ZWRvdHJhbnNmZXItMjAxNiIsImF1dGhfdGltZSI6MTQ2NzM0NjI3MCwidXNlcl9pZCI6IjRjemVXVllIekNNVnN0WEZOYldHVXBKYmJTZzEiLCJzdWIiOiI0Y3plV1ZZSHpDTVZzdFhGTmJXR1VwSmJiU2cxIiwiaWF0IjoxNDY3MzQ2MjcwLCJleHAiOjE0NjczNDk4NzAsImVtYWlsIjoic2V1bkBjbXUuY29tIiwiZW1haWxfdmVyaWZpZWQiOmZhbHNlLCJmaXJlYmFzZSI6eyJpZGVudGl0aWVzIjp7InBhc3N3b3JkIjpbInNldW5AY211LmNvbSJdLCJlbWFpbCI6WyJzZXVuQGNtdS5jb20iXX19fQ.U-fYjx8rMm5tYV24r0uEcNQtIe3UKULxsHecLdGzTbi1v-VKzKDk_QPL26SPDoU8JUMY3nJQ1hOE9AapBrQck8NVUZSKFMD49XdtsyoN2kKdinpFR1hSxIE0L2dRStS7OZ8sGiX866lNa52Cr6TXSsnMD6N2P0OtVE5EeD1Nf-AiJ-gsaLrP4tBnmj1MNYhEYVHb6sAUrT3nEI9gWmeKcPWPfn76FGTdGWZ2mjdaeAG4RbuFL4cHdOISA_0HVLGJxuNyEHAHybDX8mVdNW_F4yzL3H-SmPFY5Kv3tCdBzpzhUKfNOnFFmf2ggFOJnDsqMp-TZaIPk6ce_ltqhQ0dnQ"

>>> print jwt.decode(token, key, algorithms='RS256', audience='wedotransfer-2016', options={'verify_exp': False})

{u'user_id': u'4czeWVYHzCMVstXFNbWGUpJbbSg1', u'sub': u'4czeWVYHzCMVstXFNbWGUpJbbSg1', u'iss': u'https://securetoken.google.com/wedotransfer-2016', u'email_verified': False, u'firebase': {u'identities': {u'password': [u'seun@cmu.com'], u'email': [u'seun@cmu.com']}}, u'exp': 1467349870, u'auth_time': 1467346270, u'iat': 1467346270, u'email': u'seun@cmu.com', u'aud': u'wedotransfer-2016'}

[anjorinjnr]

Thanks.

[TNGPS]

It may be good to point this out in the docs somewhere, since I had the same problem

We try to decode an Firebase provided JWT. The publc keys are published here in the form ob je json object of X509 certificates ...

this stack overfow post seems to be helpfull:

http://stackoverflow.com/questions/12911373/how-do-i-use-a-x509-certificate-with-pycrypto

[mpdavis]

I think it makes sense to at least include this in the docs.

[danielfaust]

Thanks @mpdavis for the library and @TNGPS for the hint to the stackoverflow question.

While migrating from "Gitkit" to "Firebase+(python-jose)":

  # firebase
  # target_audience = "firebase-project-id"
  # certificate_url = 'https://www.googleapis.com/robot/v1/metadata/x509/securetoken@system.gserviceaccount.com'

  # gitkit
  target_audience = "123456789-abcdef.apps.googleusercontent.com" # (from developer console, OAuth 2.0 client IDs)
  certificate_url = 'https://www.googleapis.com/identitytoolkit/v3/relyingparty/publicKeys'

  response = urllib.urlopen(certificate_url)
  certs = response.read()
  certs = json.loads(certs)
  print "CERTS", certs
  print ''
  print ''

  # -------------- verify via oauth2client
  from oauth2client import crypt
  crypt.MAX_TOKEN_LIFETIME_SECS = 30 * 86400 # according to https://github.com/google/identity-toolkit-python-client/blob/master/identitytoolkit/gitkitclient.py
  print "VALID TOKEN", crypt.verify_signed_jwt_with_certs(idtoken, certs, target_audience)  
  print ''
  print ''

  # -------------- verify via python-jose
  from jose import jwt
  unverified_header = jwt.get_unverified_header(idtoken)
  print "UNVERIFIED HEADER", unverified_header
  print ''
  print ''
  unverified_claims = jwt.get_unverified_claims(idtoken)
  print "UNVERIFIED CLAIMS", unverified_claims
  print ''
  print ''
  from ssl import PEM_cert_to_DER_cert
  from Crypto.Util.asn1 import DerSequence
  pem = certs[unverified_header['kid']]
  der = PEM_cert_to_DER_cert(pem)
  cert = DerSequence()
  cert.decode(der)
  tbsCertificate = DerSequence()
  tbsCertificate.decode(cert[0])
  rsa_public_key = tbsCertificate[6]
  print "VALID TOKEN", jwt.decode(idtoken, rsa_public_key, algorithms=unverified_header['alg'], audience=target_audience)

[mpdavis]

@danielfaust It looks like you have given my enough info to be able to convert Firebase's certificates to a public key that PyCrypto will accept.

It won't work for arbitrary ASN.1 formatted certificates, but even just supporting Firebase certificates appears to be beneficial to consumers of this library.

I'll see what I can do to add support.

[mpdavis]

@anjorinjnr @TNGPS @danielfaust

I added support for Firebase certs in https://github.com/mpdavis/python-jose/releases/tag/1.3.0

Let me know if you run into any issues with it.

[danielfaust]

Thanks, this is very nice. I was checking the code and noticed the effort you put into the different ways the downloaded certs can be fed into the function. A nice surprise, as I expected it just to be able to get fed the certificate string into decode and not also the object or the raw string as passed from the server.

While testing them through, I did get an error.

    gitkit_decoded = jwt.decode(gitkit_jwt, certs, algorithms='RS256', audience=target_audience)
  File "/usr/lib/python2.7/site-packages/jose/jwt.py", line 132, in decode
    payload = jws.verify(token, key, algorithms, verify=verify_signature)
  File "/usr/lib/python2.7/site-packages/jose/jws.py", line 75, in verify
    _verify_signature(signing_input, header, signature, key, algorithms)
  File "/usr/lib/python2.7/site-packages/jose/jws.py", line 259, in _verify_signature
    if not _sig_matches_keys(keys, signing_input, signature, alg):
  File "/usr/lib/python2.7/site-packages/jose/jws.py", line 211, in _sig_matches_keys
    if key.verify(signing_input, signature):
  File "/usr/lib/python2.7/site-packages/jose/jwk.py", line 244, in verify
    raise JWKError(e)
JWKError: Plaintext too large

This is because PKCS1_v1_5.new(self.prepared_key).verify(self.hash_alg.new(msg), sig) library is failing for some reason when the certs from Gitkit located at https://www.googleapis.com/identitytoolkit/v3/relyingparty/publicKeys are used, be it by passing them as a dict of certs (json.loads(certs)) or directly as the raw certs string.

I can pass the specific cert json.loads(certs)[unverified_header['kid']] to the decode function, then it works just fine, so this appears not to be related to the token.

Gitkit has not yet been deprecated, and no deprecation announcement has been done. Google explicitly says that they would announce it if they would deprecate it, but that there are currently no plans to do so. Currently it's just as valuable for authentication as Firebase.

I had no problems using Firebase.

Update: This occurs randomly... let's hope that Firebase isn't affected as well.

Update2: This is a very strange error. I have been trying this for 5 minutes now, and it only happened once or twice in the beginning. This is not caused by a malformed cert server response as this raises a JWKError: RSA key format is not supported. I also noticed that you are not using the 'kid' from the header to select the certificate but just iterating over the dict (which may be okay to do).

It appears to be working. I would open a new issue if this comes back and I've got some clue on why this occurred.

[mpdavis]

I'll take a look. Thanks for the triage.

On Thu, Sep 1, 2016 at 6:46 PM Daniel Faust notifications@github.com
wrote:

Thanks, this is very nice. I was checking the code and noticed the effort
you put into the different ways the downloaded certs can be fed into the
function. A nice surprise, as I expected it just to be able to get fed the
certificate string into decode and not also the object or the raw string
as passed from the server.

While testing them through, I did get an error.

gitkit_decoded = jwt.decode(gitkit_jwt, certs, algorithms='RS256', audience=target_audience)

File "/usr/lib/python2.7/site-packages/jose/jwt.py", line 132, in decode
payload = jws.verify(token, key, algorithms, verify=verify_signature)
File "/usr/lib/python2.7/site-packages/jose/jws.py", line 75, in verify
_verify_signature(signing_input, header, signature, key, algorithms)
File "/usr/lib/python2.7/site-packages/jose/jws.py", line 259, in _verify_signature
if not _sig_matches_keys(keys, signing_input, signature, alg):
File "/usr/lib/python2.7/site-packages/jose/jws.py", line 211, in _sig_matches_keys
if key.verify(signing_input, signature):
File "/usr/lib/python2.7/site-packages/jose/jwk.py", line 244, in verify
raise JWKError(e)
JWKError: Plaintext too large

This is because PKCS1_v1_5.new(self.prepared_key).verify(self.hash_alg.new(msg),
sig) library is failing for some reason when the certs from Gitkit
located at
https://www.googleapis.com/identitytoolkit/v3/relyingparty/publicKeys are
used, be it by passing them as a dict of certs (json.loads(certs)) or
directly as the raw certs string.

I can pass the specific cert json.loads(certs)[unverified_header['kid']]
to the decode function, then it works just fine, so this appears not to
be related to the token.

Gitkit has not yet been deprecated, and no deprecation announcement has
been done. Google explicitly says that they would announce it if they would
deprecate it, but that there are currently no plans to do so. Currently
it's just as valuable for authentication as Firebase.

I had no problems using Firebase.

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
#27 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AASlDg8y1BY5MY2f6MGeFYjwDw07o9qlks5qly6dgaJpZM4JC0Lu
.

Michael Davis

[mpdavis]

@danielfaust

The Plaintext too large error occurs when the plaintext message is larger that the particular cert can verify.

message (string) - The message to encrypt, also known as plaintext. It can be of variable length, but not longer than the RSA modulus (in bytes) minus 2, minus twice the hash output size.

Some of the certs in the list have a smaller modulus and would throw that error if tried. As you saw, I am iterating over the list of certs instead of picking one out with the kid. This is an attempt to support cert endpoints that use some other format.

As for why it was intermittently failing, python dictionaries don't have a set order. If the correct cert is used first, everything will work. If a cert with a smaller modulus is used, it will blow up.

I released a fix in https://github.com/mpdavis/python-jose/releases/tag/1.3.1

Again, thank you for the triage and help working through this. Let me know if that works for you.

[danielfaust]

@mpdavis

Thanks again for the effort, and for publishing it so fast on pypi. Good to know what caused the randomly occurring exception.

I checked the code and the solution looks totally ok to me, based on the reason for failure you mention.

I also checked how the oauth2client library is handling the kid parameter, since oauth2client is used by Google's identity-toolkit-python-client library. They also ignore the kid parameter and iterate over the certs.