remote: oauth2: migrate from deprecated oauth2client library #242

granquet · 2023-05-31T10:24:31Z

oauth2client lib is deprecated and hasn't been updated in a while.

replace it with google-auth.

granquet · 2023-05-31T10:29:56Z

I'm not a python developer... but I've tried :)

the "key storage" looked a bit smart with the "oauth2client.file" library, now it's just a simple wrapper that writes the credentials in json format to the disk. not sure if there's anything better to do? opinions?

Also, the "argparser" from the "oauth2client.tools" doesn't seem to have an equivalent... opinions? :)

Hope this helps,
Guillaume.

PS: on Gentoo, the oauth2client python package has been removed... which triggered the need to provide the patch.
I expect other distros will remove it also.

gauteh · 2023-06-01T08:13:43Z

Nice, have you tested this throughly? I don't have a good setup for that at the moment.

granquet · 2023-06-01T09:21:00Z

Nice, have you tested this throughly? I don't have a good setup for that at the moment.

I should have given that precision in the previous message.
I have tested these use cases:

no credential file, creation using the provided client/api key by lieer
credential file exists, re-use of these credentials

I'm struggling to test out the refresh case, can't seem to generate short lived credentials?
I haven't looked yet at testing out the case with a provided CLIENT_SECRET_FILE.

I also haven't tried to re-use the previously created credential file by oauth2client, json looks different, so unless the new lib implements backward compatibility, user will probably have to delete their old credential file and re-request and authorization.

gauteh · 2023-06-01T09:40:59Z

That would be nice to test, or if possible, manually migrate to a new file (use a different file name so that users can roll-back).

yeled · 2023-06-16T08:30:21Z

I've massaged this a bit locally.. The first issue with this PR is that I get timeouts on my large sync. Is there a way to increase it do you think?

receiving metadata:  31%|███████████████████████████████████████▉                                                                                       | 222482/707725 [1:39:00<1:28:59, 90.88it/s]Traceback (most recent call last):
  File "/home/yeled/src/gmailieer/env3/bin/gmi", line 4, in <module>
    __import__('pkg_resources').run_script('lieer==1.5', 'gmi')
  File "/home/yeled/src/gmailieer/env3/lib/python3.9/site-packages/pkg_resources/__init__.py", line 666, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/home/yeled/src/gmailieer/env3/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1469, in run_script
    exec(script_code, namespace, namespace)
  File "/home/yeled/src/gmailieer/env3/lib/python3.9/site-packages/lieer-1.5-py3.9.egg/EGG-INFO/scripts/gmi", line 24, in <module>
  File "/home/yeled/src/gmailieer/env3/lib/python3.9/site-packages/lieer-1.5-py3.9.egg/lieer/gmailieer.py", line 229, in main
  File "/home/yeled/src/gmailieer/env3/lib/python3.9/site-packages/lieer-1.5-py3.9.egg/lieer/gmailieer.py", line 309, in sync
  File "/home/yeled/src/gmailieer/env3/lib/python3.9/site-packages/lieer-1.5-py3.9.egg/lieer/gmailieer.py", line 351, in push
  File "/home/yeled/src/gmailieer/env3/lib/python3.9/site-packages/lieer-1.5-py3.9.egg/lieer/remote.py", line 133, in func_wrap
  File "/home/yeled/src/gmailieer/env3/lib/python3.9/site-packages/lieer-1.5-py3.9.egg/lieer/remote.py", line 334, in get_messages
  File "/home/yeled/src/gmailieer/env3/lib/python3.9/site-packages/google_api_python_client-2.89.0-py3.9.egg/googleapiclient/_helpers.py", line 130, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/home/yeled/src/gmailieer/env3/lib/python3.9/site-packages/google_api_python_client-2.89.0-py3.9.egg/googleapiclient/http.py", line 1566, in execute
    self._execute(http, self._order, self._requests)
  File "/home/yeled/src/gmailieer/env3/lib/python3.9/site-packages/google_api_python_client-2.89.0-py3.9.egg/googleapiclient/http.py", line 1496, in _execute
    resp, content = http.request(
  File "/home/yeled/src/gmailieer/env3/lib/python3.9/site-packages/google_auth_httplib2-0.1.0-py3.9.egg/google_auth_httplib2.py", line 218, in request
    response, content = self.http.request(
  File "/home/yeled/src/gmailieer/env3/lib/python3.9/site-packages/httplib2-0.22.0-py3.9.egg/httplib2/__init__.py", line 1724, in request
    (response, content) = self._request(
  File "/home/yeled/src/gmailieer/env3/lib/python3.9/site-packages/httplib2-0.22.0-py3.9.egg/httplib2/__init__.py", line 1444, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/home/yeled/src/gmailieer/env3/lib/python3.9/site-packages/httplib2-0.22.0-py3.9.egg/httplib2/__init__.py", line 1396, in _conn_request
    response = conn.getresponse()
  File "/usr/lib/python3.9/http/client.py", line 1347, in getresponse
    response.begin()
  File "/usr/lib/python3.9/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.9/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.9/socket.py", line 704, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.9/ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.9/ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

granquet · 2023-07-10T16:50:52Z

Sorry I'm not very reactive...
I have spent some time trying to understand how to manage tokens on the google API... I've yet to understand how to get a token to expire to test the refresh token usecase.
Anyone able to direct me towards some documentation how I can request a new token and make it expire so that I can test the refresh method?

I've massaged this a bit locally.. The first issue with this PR is that I get timeouts on my large sync. Is there a way to increase it do you think?

I haven't been able to reproduce this, my inbox is roughly 10GB big and my internet connection is far from what you would call "fast".
Not sure here what exactly times out from the log?

(lieer-venv) testlieer@seychelles ~/mail/ranquet.guillaume@gmail.com $ gmi sync
push: everything is up-to-date.
pull: partial synchronization.. (hid: 3993042)
pull: historyId is too old, full sync required.
fetching messages: 33070it [01:08, 485.95it/s]                                                                                                                                                
removing deleted: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:01<00:00,  6.37it/s]
receiving content: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 543/543 [00:39<00:00, 13.65it/s]
receiving metadata:  11%|██████████████▎                                                                                                                 | 3651/32527 [00:44<05:41, 84.46it/s]remote: user rate error, increasing delay to 1
remote: waiting 1.0 seconds..
[...]
waiting 1.0 seconds..
remote: decreasing delay to 0
receiving metadata: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32527/32527 [14:35<00:00, 37.16it/s]
pull: complete, removing resume file
current historyId: 4040625, current revision: 118781
(lieer-venv) testlieer@seychelles ~/mail/ranquet.guillaume@gmail.com $

granquet · 2023-07-13T13:59:31Z

I have been able to test further, it has been a pain to navigate the google cloud web interface... anyway...

but what's tested (and works):

no credential file, creation using the provided client/api key by lieer
credential file exists, re-use of these credentials
refresh of old credentials
no credential file, creation using "user generated" client/api key

I haven't tested the migration from the previous json file, as I've deleted them from my system and the oauth2client python library is no more part of my system (I could reinstall using pip/venv though)

jaeyeom · 2023-07-16T23:50:56Z

At least, this solved my problem and I could successfully gmi init with granquet's repo. Thank you so much!

skangas · 2023-09-02T10:33:17Z

I've used this to do a full pull from scratch, including the whole authentication flow.

I've also tried syncing and sending emails. So far, so good.

gauteh · 2023-09-02T15:53:06Z

Sounds good, looks like this can probably be merged soon since the current solution is not really working anymore. Might require reauthorization, but that is not the end of the world.

skangas · 2023-09-04T08:11:01Z

The list of requirements in README.md should probably be updated too.

oauth2client lib is deprecated and hasn't been updated in a while. replace it with google-auth. fix gauteh#230 Signed-off-by: Guillaume Ranquet <granquet@baylibre.com>

gauteh · 2023-09-10T06:46:58Z

Thanks. Finally got around to merging this.

granquet mentioned this pull request May 31, 2023

Replace deprecated and unmaintained oauth2client library #230

Closed

granquet force-pushed the migrate_oauth2 branch from 4bf1d80 to 6f47213 Compare May 31, 2023 10:41

This was referenced Jun 2, 2023

github action for black formatter and running black on the repo #247

Merged

Access blocked: Lieer’s request is invalid #249

Closed

remote: oauth2: migrate from deprecated oauth2client library

2ccc4b4

oauth2client lib is deprecated and hasn't been updated in a while. replace it with google-auth. fix gauteh#230 Signed-off-by: Guillaume Ranquet <granquet@baylibre.com>

granquet force-pushed the migrate_oauth2 branch from 6f47213 to 2ccc4b4 Compare September 5, 2023 08:06

do not import http2lib

3fbe9c8

gauteh merged commit 3fbe9c8 into gauteh:master Sep 10, 2023
3 checks passed

gauteh temporarily deployed to github-pages September 10, 2023 06:47 — with GitHub Pages Inactive

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

remote: oauth2: migrate from deprecated oauth2client library #242

remote: oauth2: migrate from deprecated oauth2client library #242

granquet commented May 31, 2023

granquet commented May 31, 2023

gauteh commented Jun 1, 2023

granquet commented Jun 1, 2023

gauteh commented Jun 1, 2023

yeled commented Jun 16, 2023

granquet commented Jul 10, 2023

granquet commented Jul 13, 2023

jaeyeom commented Jul 16, 2023

skangas commented Sep 2, 2023

gauteh commented Sep 2, 2023

skangas commented Sep 4, 2023

gauteh commented Sep 10, 2023

remote: oauth2: migrate from deprecated oauth2client library #242

remote: oauth2: migrate from deprecated oauth2client library #242

Conversation

granquet commented May 31, 2023

granquet commented May 31, 2023

gauteh commented Jun 1, 2023

granquet commented Jun 1, 2023

gauteh commented Jun 1, 2023

yeled commented Jun 16, 2023

granquet commented Jul 10, 2023

granquet commented Jul 13, 2023

jaeyeom commented Jul 16, 2023

skangas commented Sep 2, 2023

gauteh commented Sep 2, 2023

skangas commented Sep 4, 2023

gauteh commented Sep 10, 2023