-
Notifications
You must be signed in to change notification settings - Fork 90
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Suggestion] Use webhooks to bypass rate limits #12
Comments
that would only double the rate in one channel as all webhooks in a channel are under the same rate limit. |
@NoNameLmao, @asdfisdffdg I already tried with using multiple channels but I couldn't see any major performance improvements. Still in upcoming release I will try to add multiple channels as option. |
would migration be possible? |
You need to use webhooks in order to get more speed. Edit: Also if you use mutliple servers you can via the bot create 20 channels and webhooks per server without getting blocked I used this to fill up 5 server for a total of 100 channels. Now I can put up 2 Gbit/s ( which is my network limit :) ) |
@UnlimitedBytes , Can you create PR with your code? |
Sadly I wrote a completely different implementation for myself which uses lots of libraries and multipart/form-data instead of direct binary data. So it's not possible to PR these changes as they're running on a completely different architecture and git repository. I could however port it to DDrive will take some time tho I will prob. release the PR on the weekend as that's when I have time :) |
How exactly does this work? |
You send multiple parts at the same time in parallel. So for example you create 2 channels and add 20 webhooks per channel so you get 40 webhook urls in total. You then take the let's say 1 GB big file and split it into 40 chunks that are 8 MB big. You send all this 40 chunks at the same time, each to a different webhook. You then when a webhook finished it's 8 MB chunk stuff a new 8 MB chunk into this webhook. You constantly keep up the 40 connections in parallel until the file is finished and no more data is there to be uploaded. After that you have a ton of chunks uploaded with 40 chunks per webhook timing instead of 1 chunk per webhook timing. This process can be upscaled to as much as you want and as much as your processor / ram and network can handle. |
but all webhooks in a channel share the same rate limit ??? or does sending at the same time circumvent that somehow? |
A sorry my brain is miss functional you add 1 webhook per channel but 20 channels :D |
yes that's what i was saying. but the bot also has separate rate limit per channel, so you could just use the bot itself. or you could use both for double per channel. but it might be simpler to just make more channels. |
This improvement has been added in version v4.0. I was able to upload 5GB of file under 85 seconds. |
Instead of one bot sending messages, it could create webhooks which then would upload the files at speeds faster than one bot.
The text was updated successfully, but these errors were encountered: