Avoid getting banned?

[yopbeast]

Hey this tool is awesome, i was going to scrape huge data, but i got some doubt... what about... i usually make some jquery or js script to run in the browser because curl requests are often banned specially by cloudflare websites...

Do you have any advice about using this tool for to collect huge data (3000/5000 pages data)?

[huv1k]

Use different proxy every X-requests?

[IonicaBizau]

@yopbeast If you really want to simulate a human behavior, maybe nightmare would be better.

Also, you could try spoofing the user agent and like @HuVik said, you can use a proxy.

Hope that helps.

[yopbeast]

thanks yes, thanks for the advice!