Replies: 1 comment
-
There is a point where the rate limit eventually becomes the main bottleneck, after which increasing parallelism has no benefit |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Has anyone had problems with using parallel downloads with a bot token?
I know it's recommended to not use parallelism to avoid problems with accounts, but as far as I know, bots shouldn't be under any suspicion if they throw a lot of requests. But are there technical issues I should be careful of?
I'm still keeping it down to 3, just in case, but if anyone has bumped the numbers up and had a problem (or none!) I'd be thankful to know!
And another small question (that I will probably see for myself when I get around to test it): I have a huge list of channel IDs. Is it better to loop through them and launch a CLI command for each channel, or to feed the entire list to a single command?
Or, if we want to add parallelism, is it better to launch several CLI commands at once through python parallelism, or use DCE's built-in parallelism?
Beta Was this translation helpful? Give feedback.
All reactions