Google will revoke Gmvault access March 31, 2019

[matthewhelmke]

FYI, I just received this email. I've been using Gmvault for a few years assuming it was abandoned, since there have been no commits since 2016, but thought I would create this issue anyway, just in case...

Hello,

Although you don’t need to take any action, we wanted to let you know that the following third-party apps will no longer be able to access some data in your Google Account, including your Gmail content. This change will go into effect starting March 31, 2019.

Gmvault

We are making this change as part of ongoing efforts to make sure your data is protected and private. These apps haven’t yet complied with our updated data privacy requirements announced on October 8, 2018.

You can always view, manage and remove apps you’ve given access to your account by visiting your Google Account.

Thanks,
The Google Accounts team

[tofurky]

nice. remove our ability to archive our emails to help protect it 😂

[joshstrange]

I’m hoping this can worked around by replacing an api token somewhere in the source and/or requiring users to register their own app with google. It’s not ideal but it’s easy enough and free for our level of usage I would guess.

I looked a little in the code but I’m on mobile so I’ll have to look later when I’m on my laptop.

[tofurky]

see

gmvault/src/gmv/credential_utils.py

Lines 38 to 58 in ee915c2

	def generate_permission_url():
	"""Generates the URL for authorizing access.
	This uses the "OAuth2 for Installed Applications" flow described at
	https://developers.google.com/accounts/docs/OAuth2InstalledApp
	Args:
	client_id: Client ID obtained by registering your app.
	scope: scope for access token, e.g. 'https://mail.google.com'
	Returns:
	A URL that the user should visit in their browser.
	"""
	params = {}
	params['client_id'] = gmvault_utils.get_conf_defaults().get("GoogleOauth2", "gmvault_client_id", "1070918343777-0eecradokiu8i77qfo8e3stbi0mkrtog.apps.googleusercontent.com")
	params['redirect_uri'] = gmvault_utils.get_conf_defaults().get("GoogleOauth2", "redirect_uri", 'urn:ietf:wg:oauth:2.0:oob')
	params['scope'] = gmvault_utils.get_conf_defaults().get("GoogleOauth2","scope",'https://mail.google.com/')
	params['response_type'] = 'code'
	account_base_url = gmvault_utils.get_conf_defaults().get("GoogleOauth2", "google_accounts_base_url", 'https://accounts.google.com')
	return '%s/%s?%s' % (account_base_url, 'o/oauth2/auth', gmvault_utils.format_url_params(params))

[joshstrange]

@tofurky Wow, I was in the file and even scrolled over but didn’t scroll far enough to see the “default values”. I even dug in gmvaul_utils a little after that before I gave up on my phone lol.

That gives me hope that we can work around this, potentially, if I’m scanning that code right, with just a config/defaults file?

[johanatan]

I am interested in whatever solution is ultimately reached here.

[fortran77]

Would somebody please find a procedure for users to get their own token, and post the steps required? Then each person doesn't have to figure it out separately.

[joshstrange]

I started generating new credentials and this was interesting. I'm guessing this the option that gmvault uses:

Option 3: Manual copy/paste

Important: The custom URI scheme and loopback IP address options now provide more reliable, secure, and user-friendly ways to obtain user authorization. This option may be discontinued in the future and should only be used if the choices above are not viable.

I'm still reading but:

Option 2: Loopback IP address (macOS, Linux, Windows desktop)

To receive the authorization code using this URL, your application must be listening on the local web server. That is possible on many, but not all, platforms. However, if your platform supports it, this is the recommended mechanism for obtaining the authorization code.

When your app receives the authorization response, for best usability it should respond by displaying an HTML page that instructs the user to close the browser and return to your app.

This looks like a realistic option for us if option 3 is ever removed. I think it will be but I also think we have more time. Right now we can focus on getting to to work past March 31st.

[gboudreau]

The client_id and secret are in $HOME/.gmvault/gmvault_defaults.conf
Swapping them there should do the trick without needing to modify the code.

[joshstrange]

Ok this should be as easy as replacing 2 lines in you config (~/.gmvault/gmvault_defaults.conf)

[GoogleOauth2]
scope=https://mail.google.com/
# The URL root for accessing Google Accounts.
google_accounts_base_url=https://accounts.google.com
# Hardcoded dummy redirect URI for non-web apps.
redirect_uri=urn:ietf:wg:oauth:2.0:oob
#identifier and secret in app mode for gmvault
gmvault_client_id=1070918343777-0eecradokiu8i77qfo8e3stbi0mkrtog.apps.googleusercontent.com
gmvault_client_secret=IVkl_pglv5cXzugpmnRNqtT7

Those last 2 lines are what we are going to need to change. I generated 2 new ones but it will only allow for 100 "Sensitive logins" unless my "app" is approved. I don't really want to be the one responsible for this going forward and maybe it's best instead of publishing new keys we write a guide on how to generate your own? It literally takes less than 5 minutes.

[joshstrange]

It literally takes less than 5 minutes.

Note: I already had an account setup, I just had to create a new "Project" so I don't know how long it would take for someone who had never used the google API console or if they will have to setup billing info (My account does have billing on file).

[tofurky]

falling back to generating an app password and using that is an option. i think that should bypass any of the oauth stuff.
example:

$ gmvault sync --store-passwd user@gmail.com
<enter app password>
^C
$ gmvault sync -p user@gmail.com
...

see https://support.google.com/accounts/answer/185833?hl=en for details on app passwords.
not ideal, though.

[joshstrange]

@tofurky Yeah, I saw that mentioned in a different issue. That might be the easier long-term solution.

I wish github made it easy to see if any of the 200+ forks on this saw any real development. I'd switch to a fork of this if I knew there was at least some housekeeping going on.

[brechmos]

It sounds like there is some direction on fixes for this.

When there is an acceptable solution could someone post in the Wiki and then post back here of the "accepted" solution?

(Glad there are lots thinking and working on this!)

[Edit, Hah, for some reason I was able to post in the future...]

[gboudreau]

Procedure to get yourself a new client ID & secret:

• Start here: https://console.developers.google.com
• Accept conditions, if you never used the Google API Console (after reading all the terms and conditions, evidently)
• Create a new project (at the top); of choose one you already have, if that makes sense. If you create a new one, if takes a few seconds to created. You will need to manually select it after its been created.
• Go to https://console.developers.google.com/apis ; click + ENABLE APIS AND SERVICES at the top
• Search for Gmail API; select it; click Enable
• Go to https://console.developers.google.com/apis/credentials/consent
• User Type: choose External (`Internal is for Google Workspace accounts; it would limit usage to only accounts within your Workspace)
• Click Create
• App name: enter anything you like
• User support email: choose anything available
• Developer contact information is also required at the bottom; enter the same email adress
• Click SAVE AND CONTINUE at the bottom
• Click ADD OR REMOVE SCOPES
• Select the checkbox on the Gmail API : https://mail.google.com/ line; click UPDATE
• Click SAVE AND CONTINUE at the bottom
• Click ADD USERS
• Enter all the email addresses you'll want to backup using gmvault (if you want to add more later, come back here and add them)
• Click ADD
• Click SAVE AND CONTINUE at the bottom
• Click Credentials in the left menu
• Click Create Credentials; choose OAuth client ID
• Application Type: Desktop app; give it a name (anything; I suggest gmvault, since this credential will be used by gmvault)
• Click Create
• You will get a client ID and secret. Those two values needs to replace the existing ones in $HOME/.gmvault/gmvault_defaults.conf : gmvault_client_id=... and gmvault_client_secret=...
• If you are running gmvault 1.9.1, make sure conf_version=1.9 in that same conf file, and NOT conf_version=1.9.1, otherwise, gmvault will overwrite it every time it runs. (This is a bug in 1.9.1, fixed in 1.9.2; so you do not need to change conf_version if you are running 1.9.2-beta-1 or higher.)
• Finally, obtain a new OAuth token using the following command:
  gmvault check --renew-oauth2-tok your_email_address@gmail.com
  When navigating to the URL that gmvault provides, you will see Google hasn’t verified this app.
  Click Continue, then Allow twice.

Repeat this last step for all other Gmail accounts you are backing up using Gmvault on this particular computer, or the last 3 steps on other computers. (No need to create multiple clientIDs & secrets to backup multiple accounts or for multiple installs.)

Someone pointed out that:

It seems that there is a way to circumvent the 7 days expiration for the refresh token. You have to publish the application and then skip sending any verification details. Just clicking the publish button and creating a new refresh token is enough (sometimes you need to recreate again after 7 days in order for it to work).

[Pectojin]

FYI, it seems google is pretty sticky with handing out the API access, so it may take some time to get this working.

[brechmos]

There is a "Submit for verification" beside the "save" button. Not sure if that speeds up the process at all...

[DavoOZ]

Questions:

1. Does anyone have any idea of how often one's access credentials would need to be updated (using the manual method described above)?
   https://github.com/gaubert/gmvault/issues/335#issuecomment-475437988

2. My reading of the new Google API conditions suggests that the actual app may need to be accredited for access, as well as individual users - are I correct in this?

[chbug]

I’m hoping this can worked around by replacing an api token somewhere in the source and/or requiring users to register their own app with google. It’s not ideal but it’s easy enough and free for our level of usage I would guess.

I looked a little in the code but I’m on mobile so I’ll have to look later when I’m on my laptop.

I'm confused: why is there a need to work around it instead of getting the current key reviewed once and for all? gmvault doesn't move user data to a cloud storage or anything where the data of multiple users gets centralized, so it should be straightforward, no?

[daneroo]

I successfully created App credentials with @gboudreau 's great instructions above.
Thanks for taking the time to write that up, (and being so thorough!)

[dan86603]

Daneroo - did you get confirmation that you won't have access issues or will you have to wait until after March 31st?

Thanks

[daneroo]

Not sure @dan20047 ,

• It's working right now.
• I'm don't know what 100 "Sensitive logins" means
• I did not submit the OAuth consent screen for verification, since it's already working, we'll see how it goes...

[fortran77]

@joshstrange and others interested, here is a list of forks with a last commit date in 2017 and later. Now you can search for ones with the most current development. Anything useful should preferably be merged back here if possible.

2019:

https://github.com/Anmol-Singh-Jaggi/gmvault
https://github.com/mbarczak/gmvault

2018:

https://github.com/fossabot/gmvault
https://github.com/hengy2017/gmvault

2017:

https://github.com/andriusadamonis/gmvault
https://github.com/asez73/gmvault
https://github.com/csirmaz/gmvault
https://github.com/eliask/gmvault

[dygordon]

Answers re: the consent screen/verification here: https://support.google.com/cloud/answer/7454865

In brief:

1. you can only mark an app "internal" if you have G Suite and you are creating it within your organization, in which case it is internal by default.

2. 100 sensitive logins is defined as 100 new users, so this limit shouldn't matter anyway if it's just you. I think. Not sure if it will see each use of the OAuth credentials by gmvault as a new user or not. But some rclone forums setting up G Drive scopes suggest no.

EDIT:

Well, now that I actually try the login, I get the same "Sign in with Google temporarily disabled" screen some have reported despite not being anywhere close to the 100 new user cap.

Looking at the URL and then the file, looks like gmvault is reverting the changes to $HOME/.gmvault/gmvault_defaults.conf and putting its own app keys back in. I guess this is the known issue at #245 and #273 and some of you may be running the beta version as I am on 1.9.1.

@gboudreau @daneroo you may want to check if your changes to the conf file actually did stick or you perhaps just reauthed using the standard gmvault keys (client_id=1070918343777-0eecradokiu8i77qfo8e3stbi0mkrtog.apps.googleusercontent.com).

[seanlane]

In regards to the issue with $HOME/.gmvault/gmvault_defaults.conf being overwritten, one quick patch is to change the line conf_version=1.9.1 to conf_version=1.9, per issue #273. I wasn't able to find any documentation on the differences between config files of the two versions, but some initial testing hasn't found problems. Feel free to correct me if this would actually cause any issues though.

[gboudreau]

@dygordon Yes, my changes did stick. But while I am using version 1.9.1 of gmvault, my .conf shows 1.9 (because it was created when I was running 1.9 I would guess), so like @seanlane pointed out, that seems to prevent the overwriting problem from happening. I'll add that to the guide I posted above.

[yesrod]

I overlooked a critical step at first; I needed to move or delete the old oauth2 token and repeat the sign-in process.

I'm using the docker image from https://hub.docker.com/r/aubertg/gmvault-docker so for me the process was:

1. Follow the instructions in gboudreau's comment to generate and install new credentials
2. cd to the directory mounted as /data in the container
3. move email@gmail.com.oauth2 to email@gmail.com.oauth2.old
4. attach to the existing container with docker exec -it gmvault-docker /bin/bash
5. run su -c 'gmvault sync -d /data ${GMVAULT_EMAIL_ADDRESS}' gmvault and follow the instructions

[jaskerr]

Followed @gboudreau instructions for new client ID and secret.

Google's Oauth2 endpoint: https://accounts.google.com/o/oauth2/token returned Error: HTTP Error 401: Unauthorized.

Had to remove the old Oauth2 token and generate a new one (similar to @yesrod).

Might want to update instructions to include the generation of new tokens.

[benreic]

Just used @gboudreau instructions and worked perfectly. They've been updated with the feedback from the successive comments so the instructions are comprehensive.

[tidhub]

Thank you @gboudreau, could you please clarify what the name should be:

• Fill name, click Add scope <-- is name = Gmvault, or can I give it a name?
• Select the checkbox on the https://mail.google.com/ line; click Add
• Click Save
• Click Create Credentials; choose OAuth client ID
• Application Type: Other; give it a name <-- is this name the same as the one above?

[lukaszpiwko]

I'm writing this in case anyone is wondering if Gmvault is still working. I have just performed auth credentials update as per #335 (comment) and all emails are syncing without issue.

Thanks again for this great tool!

[mimijojo]

Procedure to set-up works fine :

Procedure to get yourself a new client ID & secret:

* Start here: https://console.developers.google.com

* Accept conditions, if you never used the Google API Console (after reading all the terms and conditions, evidently)

* Create a new project (at the top); of choose one you already have, if that makes sense

* Go to https://console.developers.google.com/apis ; click `+ ENABLED APIS AND SERVICES` at the top

* Search for Gmail; select it; click `Enable`

* Go to https://console.developers.google.com/apis/credentials; choose `OAuth Consent Screen` tab

* Fill name (enter anything), click `Add scope`

* Select the checkbox on the `https://mail.google.com/` line; click `Add`

* Click `Save`

* Click `Create Credentials`; choose `OAuth client ID`

* Application Type: `Other`; give it a name (anything; I suggest `gmvault`, since this credential will be used by gmvault)

* Click `Create`

* You will get a `client ID` and `secret`. Those two values needs to replace the existing ones in `$HOME/.gmvault/gmvault_defaults.conf` : `gmvault_client_id=...` and `gmvault_client_secret=...`

* If you are running gmvault 1.9.1, make sure `conf_version=1.9` in that same conf file, and NOT `conf_version=1.9.1`, otherwise, gmvault will overwrite it every time it runs. (This is a bug in 1.9.1, fixed in 1.9.2; so you do not need to change `conf_version` if you are running 1.9.2-beta-1 or higher.)

* Finally, obtain a new OAuth token using the following command:
  `gmvault check --renew-oauth2-tok your_email_address@gmail.com`
  Repeat this last step for all other Gmail accounts you are backing up using Gmvault on this particular computer, or the last 3 steps on other computers. (No need to create multiple clientIDs & secrets to backup multiple accounts or for multiple installs.)

But it stops working after a while (10 days or so, or maybe a number use, I've no clue.)

Can someone help on how to fix or troubleshoot this. I'm really stuck.

[mimijojo]

This works indeed, but it stops working after a while (10 days or so, or maybe a number use, I've no clue.) I get "http error 400 or 401". Anyone some expertise on this ? Or a way to troubleshoot ?

PS : In the meantime, current workaround is recreating the token each time again.

[gboudreau]

@mimijojo I updated my instructions above with more details, since the flow changed since I wrote that.
Maybe re-read them now, to make sure you did the right thing when setting it up.
I myself created a new project to test those updated instructions, and updated my OAuth tokens with that project.
I know my last project allowed me to use the same tokens for a long time (they never expired). I will continue using the new tokens I just created, to confirm they don't expire, as they should.

[mimijojo]

Thanks, I'll redo the procedure and hope it won't expire anymore...

[gboudreau]

I myself created a new project to test those updated instructions, and updated my OAuth tokens with that project.
I know my last project allowed me to use the same tokens for a long time (they never expired). I will continue using the new tokens I just created, to confirm they don't expire, as they should.

I can confirm the authorization seems to have expired after 7 days. The app I gave access to using the gmvault OAuth process was simply gone from https://myaccount.google.com/permissions
It was as-if I never authorized gmvault to access my account...
I would guess this is a new limitation added by Google, for non-production (in testing) apps.

[mimijojo]

My token was also revoked after +/- 7 days. I was about to try recreating the token one more time following your instructions even more accurately. According to what you just wrote, this won't help unfortunately.
Anyone another hint or workaround to make it persistent ?

[yesrod]

In the Google developer console, under the OAuth consent screen section (https://console.developers.google.com/apis/credentials/consent) my GMVault project had somehow been switch from "Testing" status to "Production" status. Once I switched it back to "Testing" status and added my email to the "Test users" list GMVault started working for me again.

[gboudreau]

A Google Cloud Platform project with an OAuth consent screen configured for an external user type and a publishing status of "Testing" is issued a refresh token expiring in 7 days.

Ref: https://developers.google.com/identity/protocols/oauth2#expiration

So sadly, until someone can have Google verify their OAuth Consent screen, refresh tokens obtained using gmvault will always expire after 7 days, unless you were lucky enough to already have a "In production" app in your Google developer console before they added this limit, or you are using Google Workspace (and want to backup only domain users).

When the refresh token expires, you just need to run gmvault check --renew-oauth2-tok your_email_address@gmail.com to get a new one. I guess one could do that weekly.

[mimijojo]

Refreshing with "gmvault check --renew-oauth2-tok your_email_address@gmail.com" works indeed ! Only solution is to schedule this manual process once a week... Rather tricky, as my goal was to run this gmvault backup
transparently in Docker on my NAS. :-(

[maovieira]

Thank you for the guide. I follow it and successfully backup my emails.

[DavidBerdik]

Procedure to get yourself a new client ID & secret:

• Start here: https://console.developers.google.com
• Accept conditions, if you never used the Google API Console (after reading all the terms and conditions, evidently)
• Create a new project (at the top); of choose one you already have, if that makes sense. If you create a new one, if takes a few seconds to created. You will need to manually select it after its been created.
• Go to https://console.developers.google.com/apis ; click + ENABLE APIS AND SERVICES at the top
• Search for Gmail API; select it; click Enable
• Go to https://console.developers.google.com/apis/credentials/consent
• User Type: choose External (`Internal is for Google Workspace accounts; it would limit usage to only accounts within your Workspace)
• Click Create
• App name: enter anything you like
• User support email: choose anything available
• Developer contact information is also required at the bottom; enter the same email adress
• Click SAVE AND CONTINUE at the bottom
• Click ADD OR REMOVE SCOPES
• Select the checkbox on the Gmail API : https://mail.google.com/ line; click UPDATE
• Click SAVE AND CONTINUE at the bottom
• Click ADD USERS
• Enter all the email addresses you'll want to backup using gmvault (if you want to add more later, come back here and add them)
• Click ADD
• Click SAVE AND CONTINUE at the bottom
• Click Credentials in the left menu
• Click Create Credentials; choose OAuth client ID
• Application Type: Desktop app; give it a name (anything; I suggest gmvault, since this credential will be used by gmvault)
• Click Create
• You will get a client ID and secret. Those two values needs to replace the existing ones in $HOME/.gmvault/gmvault_defaults.conf : gmvault_client_id=... and gmvault_client_secret=...
• If you are running gmvault 1.9.1, make sure conf_version=1.9 in that same conf file, and NOT conf_version=1.9.1, otherwise, gmvault will overwrite it every time it runs. (This is a bug in 1.9.1, fixed in 1.9.2; so you do not need to change conf_version if you are running 1.9.2-beta-1 or higher.)
• Finally, obtain a new OAuth token using the following command:
  gmvault check --renew-oauth2-tok your_email_address@gmail.com
  When navigating to the URL that gmvault provides, you will see Google hasn’t verified this app.
  Click Continue, then Allow twice.

Repeat this last step for all other Gmail accounts you are backing up using Gmvault on this particular computer, or the last 3 steps on other computers. (No need to create multiple clientIDs & secrets to backup multiple accounts or for multiple installs.)

This should be pinned so it's easier for newcomers to find. This information is extremely important, but requires some digging right now in order to be found.

[DavidBerdik]

@matthewhelmke It looks like Got-Your-Back also requires you to create a test project. Does it have this issue as well now?

[matthewhelmke]

@DavidBerdik Got-Your-Back is still working for me. I haven't touched my script that calls it in ages, but it runs without error regularly and spot checks of the backup are successful.

[DavidBerdik]

@matthewhelmke Awesome! Thanks! I experimented with moving to Got-Your-Back over the weekend, but I prefer GMvault over it. Fortunately, I've found a way to work around the renewal revocation issue: if you submit the app for verification but don't fill out the necessary forms to actually be formally verified, your project gets put in a state in which it is only available to users that are specified as test users, but since you are also not in test mode anymore, the 7-day expiration no longer happens.

[dlmv123]

@matthewhelmke Awesome! Thanks! I experimented with moving to Got-Your-Back over the weekend, but I prefer GMvault over it. Fortunately, I've found a way to work around the renewal revocation issue: if you submit the app for verification but don't fill out the necessary forms to actually be formally verified, your project gets put in a state in which it is only available to users that are specified as test users, but since you are also not in test mode anymore, the 7-day expiration no longer happens.

@DavidBerdik How to send for verification ? In the OAuth Consent screen does not have submit for verification button or similar ? Is it to publish app ? Sorry, novice here.

[dlmv123]

Procedure to get yourself a new client ID & secret:

• Start here: https://console.developers.google.com
• Accept conditions, if you never used the Google API Console (after reading all the terms and conditions, evidently)
• Create a new project (at the top); of choose one you already have, if that makes sense. If you create a new one, if takes a few seconds to created. You will need to manually select it after its been created.
• Go to https://console.developers.google.com/apis ; click + ENABLE APIS AND SERVICES at the top
• Search for Gmail API; select it; click Enable
• Go to https://console.developers.google.com/apis/credentials/consent
• User Type: choose External (`Internal is for Google Workspace accounts; it would limit usage to only accounts within your Workspace)
• Click Create
• App name: enter anything you like
• User support email: choose anything available
• Developer contact information is also required at the bottom; enter the same email adress
• Click SAVE AND CONTINUE at the bottom
• Click ADD OR REMOVE SCOPES
• Select the checkbox on the Gmail API : https://mail.google.com/ line; click UPDATE
• Click SAVE AND CONTINUE at the bottom
• Click ADD USERS
• Enter all the email addresses you'll want to backup using gmvault (if you want to add more later, come back here and add them)
• Click ADD
• Click SAVE AND CONTINUE at the bottom
• Click Credentials in the left menu
• Click Create Credentials; choose OAuth client ID
• Application Type: Desktop app; give it a name (anything; I suggest gmvault, since this credential will be used by gmvault)
• Click Create
• You will get a client ID and secret. Those two values needs to replace the existing ones in $HOME/.gmvault/gmvault_defaults.conf : gmvault_client_id=... and gmvault_client_secret=...
• If you are running gmvault 1.9.1, make sure conf_version=1.9 in that same conf file, and NOT conf_version=1.9.1, otherwise, gmvault will overwrite it every time it runs. (This is a bug in 1.9.1, fixed in 1.9.2; so you do not need to change conf_version if you are running 1.9.2-beta-1 or higher.)
• Finally, obtain a new OAuth token using the following command:
  gmvault check --renew-oauth2-tok your_email_address@gmail.com
  When navigating to the URL that gmvault provides, you will see Google hasn’t verified this app.
  Click Continue, then Allow twice.

Repeat this last step for all other Gmail accounts you are backing up using Gmvault on this particular computer, or the last 3 steps on other computers. (No need to create multiple clientIDs & secrets to backup multiple accounts or for multiple installs.)

I have followed the steps but after I paste the verification code, I have the error return as below :
Can someone please help ?

Error: Problems when trying to connect to Google oauth2 endpoint: https://accounts.google.com/o/oauth2/token.
Error: HTTP Error 401: Unauthorized.

=== Exception traceback ===
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/gmv/gmv_cmd.py", line 743, in run
credential = CredentialHelper.get_credential(args)
File "/usr/lib/python2.7/site-packages/gmv/credential_utils.py", line 235, in get_credential
credential = cls.get_oauth2_credential(args['email'], renew)
File "/usr/lib/python2.7/site-packages/gmv/credential_utils.py", line 387, in get_oauth2_credential
access_token, refresh_token, validity, type = cls._get_oauth2_tokens(email, use_webbrowser = True)
File "/usr/lib/python2.7/site-packages/gmv/credential_utils.py", line 342, in _get_oauth2_tokens
response = cls._get_authorization_tokens(verification_code)
File "/usr/lib/python2.7/site-packages/gmv/credential_utils.py", line 306, in _get_authorization_tokens
raise err
HTTPError: HTTP Error 401: Unauthorized

=== End of Exception traceback ===

[DavidBerdik]

@dlmv123 I'm not a computer at the moment so I can't check exactly what it is called, but it's somewhere in the Google Console that you use to set up the application. I believe it's under the OAuth credentials page.

[Pseudomax]

Procedure to get yourself a new client ID & secret:

• Start here: https://console.developers.google.com
• Accept conditions, if you never used the Google API Console (after reading all the terms and conditions, evidently)
• Create a new project (at the top); of choose one you already have, if that makes sense. If you create a new one, if takes a few seconds to created. You will need to manually select it after its been created.
• Go to https://console.developers.google.com/apis ; click + ENABLE APIS AND SERVICES at the top
• Search for Gmail API; select it; click Enable
• Go to https://console.developers.google.com/apis/credentials/consent
• User Type: choose External (`Internal is for Google Workspace accounts; it would limit usage to only accounts within your Workspace)
• Click Create
• App name: enter anything you like
• User support email: choose anything available
• Developer contact information is also required at the bottom; enter the same email adress
• Click SAVE AND CONTINUE at the bottom
• Click ADD OR REMOVE SCOPES
• Select the checkbox on the Gmail API : https://mail.google.com/ line; click UPDATE
• Click SAVE AND CONTINUE at the bottom
• Click ADD USERS
• Enter all the email addresses you'll want to backup using gmvault (if you want to add more later, come back here and add them)
• Click ADD
• Click SAVE AND CONTINUE at the bottom
• Click Credentials in the left menu
• Click Create Credentials; choose OAuth client ID
• Application Type: Desktop app; give it a name (anything; I suggest gmvault, since this credential will be used by gmvault)
• Click Create
• You will get a client ID and secret. Those two values needs to replace the existing ones in $HOME/.gmvault/gmvault_defaults.conf : gmvault_client_id=... and gmvault_client_secret=...
• If you are running gmvault 1.9.1, make sure conf_version=1.9 in that same conf file, and NOT conf_version=1.9.1, otherwise, gmvault will overwrite it every time it runs. (This is a bug in 1.9.1, fixed in 1.9.2; so you do not need to change conf_version if you are running 1.9.2-beta-1 or higher.)
• Finally, obtain a new OAuth token using the following command:
  gmvault check --renew-oauth2-tok your_email_address@gmail.com
  When navigating to the URL that gmvault provides, you will see Google hasn’t verified this app.
  Click Continue, then Allow twice.

Repeat this last step for all other Gmail accounts you are backing up using Gmvault on this particular computer, or the last 3 steps on other computers. (No need to create multiple clientIDs & secrets to backup multiple accounts or for multiple installs.)

I have followed the steps but after I paste the verification code, I have the error return as below : Can someone please help ?

Error: Problems when trying to connect to Google oauth2 endpoint: https://accounts.google.com/o/oauth2/token. Error: HTTP Error 401: Unauthorized.

=== Exception traceback === Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/gmv/gmv_cmd.py", line 743, in run credential = CredentialHelper.get_credential(args) File "/usr/lib/python2.7/site-packages/gmv/credential_utils.py", line 235, in get_credential credential = cls.get_oauth2_credential(args['email'], renew) File "/usr/lib/python2.7/site-packages/gmv/credential_utils.py", line 387, in get_oauth2_credential access_token, refresh_token, validity, type = cls._get_oauth2_tokens(email, use_webbrowser = True) File "/usr/lib/python2.7/site-packages/gmv/credential_utils.py", line 342, in _get_oauth2_tokens response = cls._get_authorization_tokens(verification_code) File "/usr/lib/python2.7/site-packages/gmv/credential_utils.py", line 306, in _get_authorization_tokens raise err HTTPError: HTTP Error 401: Unauthorized

=== End of Exception traceback ===

(this is a little late... but for anyone else)

I have just successfully completed the process of connecting GMvault...
Note the Google verification code is added into the GMvault terminal
You must also check that this is created in the correct Google account (I have 2 and failed the first time as I was logged into the wrong account)

[MysticCobra]

I was excited to find this information, and was following the instructions using a Windows install of GMvault...until I tried my first sync, and got a Google error that reads:

Access blocked: Gmvault’s request is invalid

You can’t sign in because Gmvault sent an invalid request. You can try again later, or contact the developer about this issue. Learn more about this error
If you are a developer of Gmvault, see error details.
Error 400: invalid_request

Following the error details link gets you here: https://developers.google.com/identity/protocols/oauth2/resources/oob-migration

I'm not a dev and may be misunderstanding, but it looks to me like GMVault is now broken unless there's a code update to use a new API...but also now requires a web server to be running on the machine running GMVault for the authorization to complete. Is that correct??

[DavidBerdik]

@MysticCobra Google deprecated the out-of-band OAuth flow on October 3.

Here are the instructions you need to follow to work around it: #361 (comment)

[nicolaskern]

I have been in a loop of failure, still getting an error after entering this command:

gmvault check --renew-oauth2-tok ****@gmail.com

I get this error after having entered the token:

Error: Problems when trying to connect to Google oauth2 endpoint: https://accounts.google.com/o/oauth2/token.
Error: HTTP Error 400: Bad Request.

Anybody got this error? Thanks a lot!