Python 3 + PyCrypto: crypt.verify_signed_jwt_with_certs raises "Invalid token signature" due to string/bytes bug

[thomasboyt]

I was getting a mysterious "Invalid token signature" error trying to validate Google Sign-In-generated tokens using this method on Python 3.4.0 with PyCrypto. After confirming with jwt.io and the tokeninfo endpoint that the token was, indeed, valid, I dove into the method source. After realizing that all exceptions were swallowed in the PyCrypto wrapper's verify method, I found out that the call:

SHA256.new(message)

was raising:

TypeError: Unicode-objects must be encoded before hashing

This is because message is a string, and not bytes. This isn't surprising when you see where message is initially set:
https://github.com/google/oauth2client/blob/c5eb9f14d46d89fb33c4409a0d32fad4b919e136/oauth2client/crypt.py#L413.

I applied a quick fix to that line:

signed = ('%s.%s' % (segments[0], segments[1])).encode('utf-8')

And the method validated my token correctly.

In addition to this fix, I'd recommend removing that try/except block - I'm not sure why you'd want to swallow exceptions there, other than to make it harder for poor developers like me to debug issues ;)

[dhermes]

@thomasboyt Can you send a code snippet that invoked this failure? Also, what version of the library are you using:

import oauth2client
print oauth2client.__version__

[thomasboyt]

>>> print(oauth2client.__version__)
1.4.11

The code snippet invoking this failure was essentially identical to the one on https://developers.google.com/identity/sign-in/web/backend-auth#using-a-google-api-client-library

[dhermes]

It seems the relevant bits here then are:

from oauth2client import client, crypt
idinfo = client.verify_id_token(token, CLIENT_ID)

along with the fact that you are using PyCrypto (instead of pyOpenSSL) and Python 3.

[dhermes]

I am verifiying the failure in https://gist.github.com/dhermes/f77b972cbd5934b5f403 (with an expired token for my personal GMail account). My hunch is that the problem is with '%s.%s' when signing the first two segments of the JWT.

[dhermes]

I tested on Py2.7 and Py3.4 with OpenSSL and PyCrypto. Of the four possible cases, it only fails on Python 3.4 with PyCrypto.

[dhermes]

With the most recent revision of the gist (making sure signed part was bytes, as suggested), all four scenarios are working.

[dhermes]

@thomasboyt Thanks a lot for this. I saw your tweets and tried to help out a little bit.

Question, why are you using the special combination of PyCrypto and Python 3.4? Typically we see people using PyCrypto without OpenSSL when they are on App Engine. But Python 3.4 means this isn't the case.

[dhermes]

@thomasboyt Sorry it took so long to get this fixed. I'm not sure when the next release will be cut.

Care to offer why you are using the special combination of PyCrypto and Python 3.4?

[thomasboyt]

Thanks for the PR; I've got a hotfixed fork I'm using but will switch back once the next release is cut :)

As for why PyCrypto: I'm using http://www.paramiko.org/ in my codebase and that depends on PyCrypto, so it's what I had installed by default. I could use PyOpenSSL if it were a requirement, but I figured that since it supported either, I'd just stick with PyCrypto.

[dhermes]

Thanks for the info! We certainly want to support all code-paths, Py3.4+PyCrypto just seemed to be a rare one.

[PAStheLoD]

It could be rare, but we also happened to spend quite a few hours figuring this out on Py3.4 with PyCrypto. :)

[dhermes]

Sorry for the trouble. Thanks for the report!