Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LimitInvalidError while iter_download #3199

Closed
2 of 3 tasks
NotStatilko opened this issue Oct 31, 2021 · 2 comments
Closed
2 of 3 tasks

LimitInvalidError while iter_download #3199

NotStatilko opened this issue Oct 31, 2021 · 2 comments

Comments

@NotStatilko
Copy link
Contributor

Checklist

  • The error is in the library's code, and not in my own.
  • I have searched for this issue before posting it and there isn't a duplicate.
  • I ran pip install -U https://github.com/LonamiWebs/Telethon/archive/master.zip and triggered the bug in the latest version.

Code that causes the issue

...
async for chunk in TelegramClient.iter_download(
    message.document, offset=71, request_size=50000):
        ...

Traceback

File "/home/non/.local/lib/python3.7/site-packages/telethon/requestiter.py", line 74, in __anext__
    if await self._load_next_chunk():
  File "/home/non/.local/lib/python3.7/site-packages/telethon/client/downloads.py", line 66, in _load_next_chunk
    cur = await self._request()
  File "/home/non/.local/lib/python3.7/site-packages/telethon/client/downloads.py", line 76, in _request
    result = await self.client._call(self._sender, self.request)
  File "/home/non/.local/lib/python3.7/site-packages/telethon/client/users.py", line 79, in _call
    result = await future
telethon.errors.rpcerrorlist.LimitInvalidError: An invalid limit was provided. See https://core.telegram.org/api/files#downloading-files (caused by GetFileRequest)

More

In the first case i want to download file from 71 byte to 50000. As later i read in docs, request_size must be a multiply of 4096 bytes. So i changed my code to

...
async for chunk in TelegramClient.iter_download(
    message.document, request_size=53248):
        ...

With no changes so far. I still get this error.

But, when i do this:

...
async for chunk in TelegramClient.iter_download(
   message.document, offset=71, request_size=304):
       ...

Works just perfect. So am i doing something wrong, or there is problem in library code?
I asked in Telethon chat and didn't receive answer. Thanks.

@NotStatilko
Copy link
Contributor Author

NotStatilko commented Nov 27, 2021

The request_size must be divisible by 4096, and 1 MiB must be divisible by request_size as said in Telegram docs. In this case there is no any LimitInvalidError.

o The parameter offset must be divisible by 4 KB.
o The parameter limit must be divisible by 4 KB.
o 1048576 (1 MB) must be divisible by limit.

Yeah, said totally another, but

In my code, I wrote a function like this, and... it fixes my problem.
IMHO Telethon need to make such calculations under the hood.

def pad_request_size(request_size: int) -> int:
    '''
    This function pads `request_size` to divisible
    by 4096 bytes.  Note that 1MiB must be divisible 
    by `request_size`, so it will be not the closest number.
    
    If `request_size` < 4096, then it's not padded. 

    request_size (`int`):
        Amount of requested bytes.
    '''
    # Check amount of blocks
    block_count = request_size/4096
    
    if block_count <= 1:
        return request_size

    # If it's already divisible by 4096
    elif int(block_count) == block_count:
        request_size = int(block_count*4096)
    else:
        # Add 1 block.
        request_size = int(block_count+1)*4096
    
    # request_size must be divisible by 1MiB
    while 1048576 % request_size:
        request_size += 1
    return request_size

I haven't check it, but it seems that if request_size < 4096, Telethon auto pads it, so this case ignored in func.

Also, in Telethon docs there is no any information about 1MiB, so it's can be unclear.

o request_size (int, optional):
How many bytes will be requested to Telegram when more data is required. By default, as many bytes as possible are requested. If you would like to request data in smaller sizes, adjust this parameter.

Note that values outside the valid range will be clamped, and the final value will also be a multiple of the minimum allowed size.

I won't close issue for now, as i don't know if this should work like this, or i missing something.

@Lonami
Copy link
Member

Lonami commented Jan 24, 2022

Closing in favor of #1720, which is older.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants