-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How can I make cancellable uploading file to GCP storage? #3421
Comments
@sstepashka You are correct that |
@sstepashka I recommend checking out https://github.com/GoogleCloudPlatform/google-resumable-media-python Let me know if you have any questions about using it after check out the docs: https://googlecloudplatform.github.io/google-resumable-media-python/0.1.1/google.resumable_media.requests.html |
@dhermes How i can provide timeout for resumable upload? |
You would do it on your import google.auth.transport.requests as tr_requests
class TimeoutSession(tr_requests.AuthorizedSession):
def __init__(self, *args, **kwargs):
self.timeout = kwargs.pop('timeout') # Makes timeout a required kwarg
super(TimeoutSession, self).__init__(*args, **kwargs)
def request(self, *args, **kwargs):
kwargs['timeout'] = self.timeout
return super(TimeoutSession, self).request(*args, **kwargs) |
So, How I can using custom downloader for Blob (inside GCP environment)? |
Just import google.auth
# You may want other scopes, but read-only is enough for downloading.
ro_scope = u'https://www.googleapis.com/auth/devstorage.read_only'
credentials, _ = google.auth.default(scopes=(ro_scope,))
transport = TimeoutSession(credentials, timeout=25)
media_url = (
u'https://www.googleapis.com/download/storage/v1/b/'
u'yourbucket/o/yourobject?alt=media')
download = ChunkedDownload(
media_url, chunk_size, stream)
while not download.finished:
response = download.consume_next_chunk(transport) (Most of this is just copied from the docs I mentioned above.) |
Hello! |
Sometimes uploading is very slow. Uploading small files sometimes requires couple of seconds, sometimes about 15 mins. I think, The good solution is make request with timeout or cancel request then retry to upload. I see, In current implementation I cant provide timeout or cancellation to upload/download methods. Could you help me with it?
Thank you!
The text was updated successfully, but these errors were encountered: