gdrive: a pair of cleanup refactors (upstream branch) #3382
Conversation
Employ @Retry(filter_errors) and @wrap_prop() to pack things up a bit.
- cache root transparently - make cache dirs and ids symmetric, i.e. both using full paths - implement .list_cache_paths() instead of .all() like in the rest of the remotes
Codecov Report
@@ Coverage Diff @@
## master #3382 +/- ##
==========================================
+ Coverage 92.96% 93.03% +0.06%
==========================================
Files 140 140
Lines 8515 8480 -35
==========================================
- Hits 7916 7889 -27
+ Misses 599 591 -8
Continue to review full report at Codecov.
|
| @@ -41,30 +38,18 @@ def __init__(self, path): | |||
| ) | |||
|
|
|||
|
|
|||
| @decorator | |||
| def _wrap_pydrive_retriable(call): | |||
| def gdrive_retry(func): | |||
| from pydrive2.files import ApiRequestError | |||
There was a problem hiding this comment.
Hm, I'm now getting:
Traceback (most recent call last):
File "/home/efiop/.pyenv/versions/3.7.0-dvc/bin/dvc", line 8, in <module>
from dvc.main import main
File "/home/efiop/git/dvc/dvc/main.py", line 6, in <module>
from dvc import analytics
File "/home/efiop/git/dvc/dvc/analytics.py", line 17, in <module>
from dvc.repo import Repo
File "/home/efiop/git/dvc/dvc/repo/__init__.py", line 18, in <module>
from dvc.remote.base import RemoteActionNotImplemented
File "/home/efiop/git/dvc/dvc/remote/__init__.py", line 5, in <module>
from dvc.remote.gdrive import RemoteGDrive
File "/home/efiop/git/dvc/dvc/remote/gdrive.py", line 70, in <module>
class RemoteGDrive(RemoteBASE):
File "/home/efiop/git/dvc/dvc/remote/gdrive.py", line 207, in RemoteGDrive
progress_name="",
File "/home/efiop/git/dvc/dvc/remote/gdrive.py", line 42, in gdrive_retry
from pydrive2.files import ApiRequestError
ModuleNotFoundError: No module named 'pydrive2'
it wasn't triggered before, not sure what went wrong.
There was a problem hiding this comment.
@efiop how do you reproduce this? I tried with some simple tests and was not able to get this error 🤔
I'm getting it if GDrive support is not enabled. Which means the we don't catch and transform the error properly (or what other mechanism do we use to give users instructions on how to install remote support?)
File "/Users/ivan/Projects/dvc/.env/bin/dvc", line 8, in <module>
from dvc.main import main
File "/Users/ivan/Projects/dvc/dvc/main.py", line 6, in <module>
from dvc import analytics
File "/Users/ivan/Projects/dvc/dvc/analytics.py", line 17, in <module>
from dvc.repo import Repo
File "/Users/ivan/Projects/dvc/dvc/repo/__init__.py", line 18, in <module>
from dvc.remote.base import RemoteActionNotImplemented
File "/Users/ivan/Projects/dvc/dvc/remote/__init__.py", line 5, in <module>
from dvc.remote.gdrive import RemoteGDrive
File "/Users/ivan/Projects/dvc/dvc/remote/gdrive.py", line 70, in <module>
class RemoteGDrive(RemoteBASE):
File "/Users/ivan/Projects/dvc/dvc/remote/gdrive.py", line 207, in RemoteGDrive
progress_name="",
File "/Users/ivan/Projects/dvc/dvc/remote/gdrive.py", line 42, in gdrive_retry
from pydrive2.files import ApiRequestError
ModuleNotFoundError: No module named 'pydrive2'
No error if DVC is installed with [all,test].
There was a problem hiding this comment.
@shcheklein Getting that error from any dvc command when pydrive is not installed. Not even trying to push to pydrive or anything. Looks like python tries to run that code for some strange reason.
No description provided.