-
Notifications
You must be signed in to change notification settings - Fork 658
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
retry failed elevation tile download #4461
Conversation
…or the downloaded tile was corrupt
except URLError as e: | ||
LOGGER.critical(f"Download failed of elevation tile {tile.dir}/{tile.name}: {e.reason}") | ||
|
||
download_status = DownloadStatus.FAILED |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
most of the diff is just wrapping the previous logic into a for loop, where we request 15 times maximum with increased sleeps in between (exponential back off), up to 4.5 hours until we give up. could give much earlier though :D
try: | ||
uncompressed = gz.read() | ||
except Exception as e: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
protect for corrupt downloads
if download_status == DownloadStatus.CORRUPTED: | ||
LOGGER.error(f"Tile {tile.dir}/{tile.name} was corrupted, removing it...") | ||
filepath.unlink() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if it was corrupted in the end, let's remove that tile from the file system
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe use 4 or something instead of 15 tries
changed it to 5 retries, totalling 7.5 mins in worst case. |
not gonna merge master again with the admin changes |
fixes #4425