Allow tracking upload progress.

[bachirelkhoury]

This is related to #1830 reopening here as this seems to have been closed many years ago.

We are please looking for this feature as we need to monitor large files being uploaded to Google Storage buckets. I am surprised not many people are after this essential feature, which makes me feel we haven't done our research properly or that the solution is very obvious or trivial.

Can someone please share an example of how we could track progress during upload?

Update: Should we be looking at google-resumable-media? will try that out and report back.

Thanks

[tseaver]

@frankyn Please help prioritize this feature. See the discussion in #1830 / #1077 for the tradeoffs involved.

[pdex]

Here's a workaround using tqdm.wrapattr:

import os
from google.cloud import storage

def upload_blob(client, bucket_name, source, dest, content_type=None):
  bucket = client.bucket(bucket_name)
  blob = bucket.blob(dest)
  with open(source, "rb") as in_file:
    total_bytes = os.fstat(in_file.fileno()).st_size
    with tqdm.wrapattr(in_file, "read", total=total_bytes, miniters=1, desc="upload to %s" % bucket_name) as file_obj:
      blob.upload_from_file(
        file_obj,
        content_type=content_type,
        size=total_bytes,
      )
      return blob

if __name__ == "__main__":
  upload_blob(storage.Client(), "bucket", "/etc/motd", "/path/to/blob.txt", "text/plain")