-
Notifications
You must be signed in to change notification settings - Fork 799
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
timeframe doesn't work - Google returned a response with code 429 #596
Comments
Same here from one month |
At first it gave an error of 500, then it became 429 |
I have changed to use selenium to scrape instead for over a month :))) |
I'm trying the same with puppeteer, but not stable, time to time, I still have the 429 in headless mode. Is it fully working for u? Any tips ? |
I will try to change to the headless mode to see whether it works. First I get to https://trends.google.com/trends/, and then get to the url of https://trends.google.com.vn/trends/explore to get the cookies, then I perform a fake searching (shirt) for example. Then can get to another permalink with geo and time frame. I scrape using clicking on the download csv and parse that csv file to get the thing I want. I found that GG can track whether an IP address is scraping or not, so you can random interval time between performing automation step, and I highly recommend using proxies. An IP can scrape data again without getting 429 if it "relax" for more than one or two hourse after performing downloading a bunch of keywords |
Does it work if you use pytrends with a list of proxies? |
I haven't tried yet but I think It will not solve the problem, since you still can scrape the data with pytrends if your time frame is more than a year, so I think the problem it that your requests is regconized as a bot, not a client |
I see that pytrends works good again |
are you sure? |
You can try, my automatic program can scrawl with time frame 1 - H without 429 errors for more than 3 hours. If it's true so the problem is Google API backend :v |
So, has this issue been resolved? I tried the aforementioned method of adding the custom header into "dailydata.py", but I still encounter the 429 Error... It even triggered the error without executing data for six months. Here is my code and exception.
|
Same here, the example from the documentation fails with:
|
The timeframe parameter (any hourly and daily flags tested) keeps breaking the code, giving Google 429 even in the first call (not over calling) in code that has been running for long without any issues. (most updated version of pytrends)
This works:
kw_list=['']
pytrend.build_payload(kw_list)
related_queries = pytrend.related_queries()
This doesn't:
pytrend.build_payload(kw_list, timefreme = "now 1-H")
related_queries = pytrend.related_queries()
The text was updated successfully, but these errors were encountered: