You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi all, does anyone have up-to-date data for how many searches you can perform per IP per time period before getting blocked?
We have a limited scraping need, and spare server resources, so figured this would be a good solution, but would like to know how best to split the work.
I found old posts suggesting around 300 regular (not Google dorks) searches per 24 hours, but have no idea if this is still correct.
Any input greatly appreciated.
The text was updated successfully, but these errors were encountered:
I have been able to scrape for +12 hours with a 70-200 second sleep in between each query on a single IP, without detection. When I reduced the sleep range to 60-110 seconds, it was detected after ~15 minutes. Patience is key :)
Thanks @YvesBos, so thats inline with the older blog posts i found. I guess we can set up a couple of VPS's and run some tests.
Have you tried any tests with continuous sessions (same userdatadir) vs fresh sessions?
Ah yes, I forgot to mention that. I only scrape for a couple of keywords and then relaunch the scraper with a different country in the config, thus creating a new session. I have no experience with using continuous sessions.
Hi all, does anyone have up-to-date data for how many searches you can perform per IP per time period before getting blocked?
We have a limited scraping need, and spare server resources, so figured this would be a good solution, but would like to know how best to split the work.
I found old posts suggesting around 300 regular (not Google dorks) searches per 24 hours, but have no idea if this is still correct.
Any input greatly appreciated.
The text was updated successfully, but these errors were encountered: