Storage: download_to_filename hangs if network disconnects in the middle #5909
Labels
api: storage
Issues related to the Cloud Storage API.
help wanted
We'd love to have community involvement on this issue.
type: question
Request for information or clarification. Not an issue.
Projects
I have an issue where I am downloading large files (a few GB) from Google Cloud Storage using the download_to_filename method. My application needs to be able to transition into and out of offline mode nicely so during testing I was disconnecting from the network and found that if I disconnect in the middle of download_to_filename, the application just hangs and the function never returns.
Example code:
If I disconnect from the network just before calling
blob.download_to_filename
, then I get a ConnectionError from requests which is what I would expect. But if I disconnect from the network afterblob.download_to_filename
starts, the function just hangs. There is no timeout or exception raised.I am using Python 3.6.1 on Windows 10 with these package versions:
I have not been able to find a good work-around to this, but in my opinion it would make sense for download_to_filename to take a timeout argument and raise an exception if it fails.
I realize that it does resume nicely after network connection is re-established, but in my application the state of those files can change while offline so that does not help me. So maybe having a timeout that by default is None and would maintain the existing behavior, but provides the option to force the function to return within a reasonable amount of time.
I also posted a question to StackOverflow about a possible solution (https://stackoverflow.com/questions/52239860/download-to-filename-hangs-if-network-disconnects-in-the-middle), but this is more of a request for an improvement to the API.
Also, I am not sure if this issue also applies to the upload_from_filename or not, but that might also require a similar modification.
The text was updated successfully, but these errors were encountered: