-
-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FloodWait Exception handling #1306
Comments
I found a solution after wasting many days. lets say you have 1000 files to download. sort all the files by |
In my function i walk through get_chat_history and this is sorted by date. Than i call download_media if the message media is MessageMediaType.PHOTO. |
I do not see Pyrogram Is this raising an error supported for this call? |
Better to use search_messages with photo filter. Also you can try this PR: #1313 |
I am trying to catch the exception using VS Code: |
The reason this error handling doesn't work is because the exception is trapped and logged within https://github.com/pyrogram/pyrogram/blob/master/pyrogram/client.py#L1020-L1021
The code should either re-throw the exception or return some details so that we can act on it. |
This was fixed in #1313, thanks @KurimuzonAkuma Works in this fork: |
Thanks for the update. Doesnt the link above need a |
I think it is a bug from pyrogram, we just can't catch the flood error from app.download_media, so I find another way to solve this, when show a FloodWait error, the file downloaded size usually is 0KB, so we can test whether the file size is 0KB. def is_file_empty(file_path):
if os.path.exists(file_path):
return os.path.getsize(file_path) == 0
else:
return True
flag = True
while flag:
file_path = app.download_media(file_id)
if file_path and not is_file_empty(file_path):
flag = False
# solve success situation
else:
time.sleep(10)
# sleep for 10 seconds and retry |
I solve this in an incredibly dumb way but it also respects the throttling delay value sent by Telegram. Basically I set up a Python logging filter on the pyrogram logger and trigger an import asyncio
from datetime import datetime, timedelta
import logging
from pyrogram.errors.exceptions.flood_420 import FloodWait
from .download_manager import DownloadManager
class RateLimitLogFilter(logging.Filter):
def __init__(self, dm: DownloadManager):
super().__init__()
self.download_manager = dm
def filter(self, record: logging.LogRecord) -> bool:
if record.exc_info is None:
return True
_, exc, _ = record.exc_info
if isinstance(exc, FloodWait):
exc: FloodWait
duration = timedelta(seconds=exc.value)
wake_time = datetime.now() + duration
print(f'Encountered rate limit - need to sleep for {duration.total_seconds()} seconds, until {wake_time}')
asyncio.create_task(self.download_manager.pause_downloads_for(duration))
return False
return True async def pause_downloads_for(self, duration: timedelta):
self.rate_limit_event.clear()
self.console.print("Starting sleep")
try:
await asyncio.sleep(duration.total_seconds())
except asyncio.CancelledError:
self.console.print("Sleep cancelled")
finally:
self.rate_limit_event.set()
self.console.print("Finished sleep") pyrogram_logger = logging.getLogger('pyrogram.client')
pyrogram_logger.addFilter(RateLimitLogFilter(dm)) Like you mentioned, if file size is 0 you can assume the download failed and just retry after the delay. |
Checklist
pip3 install -U https://github.com/pyrogram/pyrogram/archive/master.zip
and reproduced the issue using the latest development versionDescription
For a download script i try to respect the telegram rate limits as documented here. This does not work cause my FloodWait exception is never reached.
Steps to reproduce
Call download_media in a loop for more than 200 files to download. Check if the exception get catched with a log and sleep the amount of time from FloodWait value.
Code example
Logs
The text was updated successfully, but these errors were encountered: