XOAUTH2 google apps for domains

[mjdavies]

Morning all

Can you confirm that this gem works currently with how Google have changed the way you use XOAUTH2?

I can't get this to work, with a valid access token, keeps telling me Invalid Credentials.

I'm currently in a conversation with a google representative about this, any news I get I'll pass on.

Matt

[nfo]

Hi Matt,

can you share the page where they announced/documented this change ?

[mjdavies]

Hi Nicolas

This is going to be a bit long winded, but every time I've done anything with google that has been the case.

I'm going to paste in a list of actions I did in order to try and get a valid access token to use with your gem. Underneath that will be the response from google. Im more than likely doing something stupid, so if you can see it, please let me know. BTW, we've used your gem for years with XOAUTH1 and our google apps for Education service, worked like a charm.

My step by step guide

1. Go to google developers console, and sign into it as a Super Admins
   user account in our Google Apps Domain.
   https://console.developers.google.com/project
2. Create a project.
   The name is irrelevant and use the suggested Project ID.
   Use the default value of App Engine Location as US data center.
3. Create the secrets and ID's for authentication
   Click APIs $ auth -> Credentials
   Click Create new Client ID
   Select Installed Application
   Click Configure Consent Screen
   Select your email address and give it a product name
   Click Save
   Re-select Installed Apllication on the following screen and Other from the
   installed Application Type
   Click Create Client ID

You should now have a Client ID, a Client secret and 2 redirect URIs

1. Enable the google mail api in the project
   Click API's, then search for mail. Once yu've found the google mail api
   enable it.
2. Retrieve an access token
   I used jotto's script to retreive the access token
   https://gist.github.com/jotto/2932998
   Replace the client_id and client_secret with your values from step 3, and
   select a scope. I'm going to use the read only gmail for apps one.
   https://www.googleapis.com/auth/gmail.readonly
   Run the script once you've made your changes
   ruby google_oauth2_access_token.rb

Follow the instructions on the screen, and you should now have an access
code, paste that into the terminal window where the script is running, and
it will thorw back at you the access token, as well as a refresh token.

Token is:
ya29.YwG5POrZpc2ddddLCNMYugsFwgJlSQlSiLkf7405U_j8gILABrlBao4Uhf6rOYX10tNi9dEjZ5X
GP6Z4zEA
Refresh token is: 1/sJcCBNzdb9b8sf168nR-h_04laMDHk68dPfsYW1Pq-0u8TYM

1. Enable the project to connect to our google apps domain.
   In the google apps domain manager,
   https://admin.google.com/AdminHome?pli=1&fral=1

Click Security -> Show more -> Advanced -> Manage API Client access
Paste the Client ID from step 3 into the Client Name box
Paste the Scope you used in step 5 into the Scopes box, then click
authorize.

Googles response

"The steps you use are very good and exactly what you should be doing, other then the third party script you are using for generating of the token. Although I review it and it seems fine to me, I can't advise you to continue using it, however, this is not where I saw a problem, but keep in mind that the script is not supported by us. At the very end, you explained how you manually add the scopes within the Admin Console > Security, which is correct if you were using the old authentication methods from oAuth 1.0, however, this authentication method was deprecated and currently, we are using 2.0, which has different ways of functioning.

You can find how to use 2.0 OAuth in this official article: https://developers.google.com/identity/protocols/OAuth2UserAgent"

[nfo]

I should double-check cdaea3f then.
Did you try XOAUTH2 on a standard gmail account instead of a Google Apps domain ? This could save me some time.
I'll take a look asap (today hopefully).

[mjdavies]

Hi Nicolas

I'll try and do it with a normal gmail account now, it's unlikely to work though as the system that is doing this call is highly tied in with google apps, so it may not be possible, I will try though.

I'm not totally convinced that the google support is correct here, and I think it may be a problem with the account that I create the project with in the developer console. It is a super admin in the google apps domain though.

[mjdavies]

Bit more information

I followed the instructions on this page
https://developers.google.com/identity/protocols/OAuth2UserAgent
to create an Oauth2 access token, which was linked up to my google developers console project, so got the ID's and that and access token was fine, for gmail.read.only scope.

I then threw this at it

curl -H "Authorization: Bearer ya29.ZAGeGYrMRCcJ_1abz5LoH9lsZbILm2XNdLY7hODUqtmZcwdq8aagOs05d3uYdBJzrSKyiwWRRTGY-Nh4Q" https://www.googleapis.com/gmail/v1/users/me/threads

That was fine

But if I changed the 'me' to a google apps users email address, it failed.

"domain": "global",
"reason": "forbidden",
"message": "Delegation denied for mjdavies@glam.ac.uk"

Bear in mind I'm a fully fledged admin in that domain.

Found this about this topic

http://stackoverflow.com/questions/26135310/gmail-api-returns-403-error-code-and-delegation-denied-for-user-email

That does say that using imap is ok though. I have a deep seated fear of working with google, the documentation drives me insane.

[nfo]

Matt,
your question goes beyond my knowledge of Google authentication. The ticket subject misled me. It's not about Google Apps domains, it about Google Apps domains AND delegation.
One sure thing is that using XOAUTH2 on Google apps domain works perfectly fine by using the access token.

[nfo]

Well I'm not sure anymore. I guess the problem happens because you try to use scopes. In any case, I never tried that. Thus you should try a library in another language to know if gmail_xoauth really has a bug.
In fact .... I'm not even sure your question is about gmail_xoauth, you did not show me any code calling the gem.

[mjdavies]

Hi Nicolas, thanks for getting back to me.

I'm pretty sure that the gem is fine, but it's the process that I'm following trying to setup the access that I've messed up, or maybe I'm misreading some documentation or using the wrong token.

I'm going to write up a step by step account of what I'm doing, and include the code I'm using to try and authenticate using my google admin account, no delegation involved then, and then get back to you.

Sorry for the confusion of putting some work I did on the gmail API up here, I realise this isn't using that but it was the delegation issue that made me think it could be connected to my account, not the gem.

If you've got a blow by blow account of how to setup the connection between project in the google developers console, and the Google Apps domain, especially the options you take when creating this connection, that would be absolutely fabulous.

[mjdavies]

Hi Nicolas

I'm following google's docs here in this process.

1. Create a project in my google developer console, using the installed app option. I'm signed into google as a domain admin for our google apps domain.
2. Using this page, https://code.google.com/p/google-mail-oauth2-tools/wiki/OAuth2DotPyRunThrough, send off the client_id and client_secret to obtain an Access Token. During that process I give the application authority to access my email by a consent screen.
3. Stick that access_token into this code

require 'gmail_xoauth'
@access_token = "access_token"
@email = "my email address"
imap = Net::IMAP.new('imap.gmail.com', 993, usessl = true, certs = nil, verify = false)
imap.authenticate('XOAUTH2', @email, @access_token)

That works, happy days!!

If I change the email address to a students email address that is in our google apps for domains, it doesn't work.

4 The next thing I need to do is to tell our Google Apps that this project is allowed to access all users email data.
The way that google advise you to do that is to sign into Google Domain Admin Console as an admin, Click on Security, CLick Show More, Click Advanced, Click Manage API CLient Access, Enter the Client ID of the project created in step 1 in the Client Name box, then enter the Scope for IMAP access, https://mail.google.com/.

Once that is done, I still cannot access the other users mail count, invalid credentials error.

Recently a google representative told me that step 4 was for Oauth1 authentication, and not to do that any more as it's changed. If you click on learn more on that page, it takes you to here which also states that it's deprecated. https://developers.google.com/accounts/docs/RegistrationForWebAppsAuto?csw=1

I'll post this up in stack overflow as well, see if anyone else can shed any light on it, but it seems to me the bit I'm missing is telling our google apps domain to trust the project I set up in 1.

I'm now looking through this documentation, see if it will help.

https://developers.google.com/identity/protocols/OAuth_ref

[mjdavies]

I've asked for some help in the community Nicolas.

http://stackoverflow.com/questions/29967175/migrating-to-xoauth2-and-accessing-other-users-unread-message-count-in-google-ap

[mjdavies]

Hi Nicolas

After some research and gnashing of teeth I've finally been able to get https://github.com/google/oauth2client to access a students gmail data, using a service account.

Here's the code

from oauth2client.client import SignedJwtAssertionCredentials

client_email = 'serviceaccount@developer.gserviceaccount.com'
with open("testserviceaccount.p12") as f:
  private_key = f.read()

credentials = SignedJwtAssertionCredentials(client_email, private_key,
    'https://www.googleapis.com/auth/gmail.readonly', sub='student email address')


from httplib2 import Http

http_auth = credentials.authorize(Http())

from apiclient.discovery import build
service = build('gmail', 'v1', http=http_auth )
messages = service.users().messages().list(userId='student email address').execute()

print messages

See how I provide the public key, and also the client_email address from the Service Account that I have given delegation rights to all our users in our google apps domain.

Given that information, could you tell me how to use gmail_xoauth to accomplish the same result?

As I'm not inputting an access token here, and your code wants an access token, I'm really struggling to work it out.

Like I said earlier, it's probably me not understanding something, so any help and advice would be great.

Thanks

Matt

[nfo]

Hi Matt,

your example uses an HTTP API, but gmail_xoauth only supports IMAP and SMTP. I don't even know if what you're trying to accomplish is possible with the IMAP extended protocol provided by Google.
If you can provide an example using IMAP in any language, I'll be glad to implement it in gmail_xoauth. Implementing it using HTTP, I'm not so sure, it's not in the scope of this gem.

[mjdavies]

Hi Nicolas

It looks like it is possible.

https://code.google.com/p/java-gmail-imap/wiki/GmailAndXOAUTH2

It would be great if you could port that into this gem.

Matt

[mjdavies]

Hi Nicolas

Is this something your still considering doing or should we start looking elsewhere?

Thanks again

Matt

[matthewford]

@mjdavies did you manage to get this gem working with imap access for domain-wide delegation of authority?