-
-
Notifications
You must be signed in to change notification settings - Fork 274
Description
Checklist before requesting
- I have searched for similar feature requests and didn't find a duplicate.
- I have checked the documentation to confirm this feature doesn't already exist.
Problem Statement
Most times web sites add anti-bot protection systems to prevent DoS attacks and enormous hosting bills. Some aggressive bots (for no apparent reason) request the same static web pages (with random query strings to bypass caching) more than 10-times per second from different residential IP addresses (to bypass IP-based rate-limiting). This results in 250k+ requests and 20+ GB bandwidth per day for only the HTML content (without CSS, JS, images, videos, fonts)!
Because there is no way to rate-limit bot traffic and a global rate-limiting would degrade the UX of legitimate human traffic, the only way to fight against aggressive bots is with anti-bot protection systems.
Proposed Solution
Therefore I am proposing that Pydoll and similar automating tools implement an additional delay/throttling for every request after a Captcha has been detected. By default it should limit the request rate to 1 request per 10 seconds (or at most 1 request per second).
Alternatives Considered
No response
Additional Context
No response
Importance
Important
Contribution
No, I don't have the capacity to implement this