429 After first request?

[alexsullivan114]

Howdy,

I'm attempting to use the library and I'm getting hit with a 429 error after copying the example code.
Here's my script:

from pytrends.request import TrendReq

pytrends = TrendReq(hl='en-US', tz=360)
kw_list = ["Blockchain"]
pytrends.build_payload(kw_list, cat=0, timeframe='today 5-y', geo='', gprop='')
pytrends.interest_over_time()

I can visit the trends website fine, and I can copy and paste the URL produced by the API and get a json file just fine. It's hard for me to imagine being rate limited on my first request and still being able to visit the site normally.

Any ideas?

[jersobh]

same problem here

[alexsullivan114]

For what its worth, this is also happening to me for an analogous javascript google trends library.

[hkpoint]

Same problem here. Seems it only started to happen today...

[alexsullivan114]

After doing a bit more sleuthing, this is also happening on the last working version I had used, which was version 4.1.1 of the library. Sounds like Google made some changes today.

[tdrobbin]

also getting the same problem today

[ratcliffej]

Same problem here. :(

[lopezdp]

I have been fighting with this all day. Even tried hitting it as such:

import urllib.request
with urllib.request.urlopen('https://trends.google.com/trends/explore?date=2009-01-03%202018-05-
28&q=bitcoin') as response:
    trnData = response.read()

I get the same 429...

This hurts, Totally killed progress on my research today.

[peeyush-ad2games]

Did anyone manage to find any solution? As I checked a request without any params on https://trends.google.com/trends/explore from code is failing.

[RKushnir]

The /trends/explore page also fails if you try to open it in a real browser in the incognito mode.
That is because google trends expect some cookie to be present. And if you refresh the page, the cookie is now present and the page works.

So, in the code you would need to simulate this behavior — make the initial request, remember the cookie, then make the same request again, but this time providing the stored cookie.

[dada-engineer]

@RKushnir is this possible as of now? I do not see in the code how.

I tried by changing the code and call trends.google.com in the init of TrendReq call and store it in a cookies attribute. In the get_data I did then provide cookies to requests.get call.

This works, but I am not sure if this is needed. Neither do I know if this should even be handled by TrendReq itself (additional call to google on all TrendReq inits), or shall be provided by the user somehow (for the user it seems kinda inconvinient to think about that every time he requests something)

[RKushnir]

@dabdada I mean it as a hint how to fix/work around this issue in general. Probably, the library code needs a fix.

[dada-engineer]

I created a PR with my suggestion on how to handle that. Thank you very much @RKushnir for the hint ;-)

#244

[zachokeeffe]

I got it to work in R: PMassicotte/gtrendsR#273.

Not sure how to implement it in Python, but it's apparently the "OTZ" cookie from apis.google.com that you need set.

[RKushnir]

@zachokeeffe In my experiments the name of the cookie was NID, but I was getting it not from apis.google.com, but from trends.google.com.

[petersonnek]

@dreyco676 just merged a PR from @dabdada that includes all cookies from trends.google.com. In a pinch handling all of the cookies solves the problem for now. If you confirm that it's the NID cookie, please update request.py and associated tests and submit a PR.

Thanks for helping with this issue!

[kakarla7]

I face the same issue now. If it's worth, I tried to clone the zip file and use it from there it seems to work.
Not entirely sure what's the reason, though!

[jersobh]

@kakarla7 so do a # sudo pip install pytrends --upgrade
(to install it globally or just do it on your venv. it may be pip3 or pip3.6 also). The latest version seems to work fine.

[dripti]

Anyone with any luck on it? I am getting the same issue

[ratcliffej]

Make sure you download the latest version of Pytrends from this repo. I had this problem, and the last check-in seems to have fixed it.

[kakarla7]

@dripti yes. directly installing from pytrends githu repo helped me (picture in my previous comment)

[dripti]

i have already uninstalled and installed again but still doesn't work

[dada-engineer]

Could you maybe try deleting pyc files?

[polzkey]

I am experiencing the same problem. I tried removing and installing it again but did not fix the problem. Tried updating and installing directly from the repo did not fix the problem.

[lopezdp]

Here is how I solved it:

https://stackoverflow.com/questions/50571317/pytrends-the-request-failed-google-returned-a-response-with-code-429/50616233#50616233

[MYRepoMan]

Anyone know how long you go to rate limit jail? I can't even do a google trends web search without error 429.

[RKushnir]

@MYRepoMan do you have cookies enabled? Maybe some browser add-on is blocking them?

[polzkey]

I tried your solution @lopezdp but the problem still there. Like updating it directly from the repo.

[rsmahabir]

I installed the package from pycharm and it works for me

[rsmahabir]

May be easier to clone the repository and install using the setup.py file

[rsmahabir]

Its looks like request.py was updated. I switched out that file and everything seems to work without issue :)

[kalagee-anjaria]

Hello all,
I am still getting the same error even though my pytrends library is of the latest version. Any suggestions on what should I do now to get rid of this error?

[dada-engineer]

can you provide an error message for what you are seeing and some code @kalagee-anjaria

[kalagee-anjaria]

The error message is The request failed: Google returned a response with code 429. The pytrends = TrendReq works fine in my localhost, but on AWS server it keeps giving this error.

[dada-engineer]

what aws setup do you use? lambda, ec2, anything else?

[kalagee-anjaria]

I am using EC2

[JessHPRH]

I've also found the problem again in my localhost. I'm using jupyter notebook.

I get this:
ResponseError: The request failed: Google returned a response with code 429.

I've tried making sure it's the most up-to-date version of pytrends as per this comment.

[kalagee-anjaria]

@JessHPRH Try using VS Code. It's working on localhost there

[JessHPRH]

I'm using Anaconda and when I tried to run my script in VS Code it told me it didn't recognise my installation of numpy. I don't fancy reconfiguring my whole system to make one script work :/

[eracle]

TLDR; I solved the problem with a custom patch
You can find the patch on this link.
Upvotes are strongly appreciated.

[siddhart1o1]

what aws setup do you use? lambda, ec2, anything else?

I want to use pytrend with lamda but getting error. Any one know how to fix it

https://stackoverflow.com/questions/70633679/getting-unable-to-import-module-lambda-function-in-aws-lambda-while-using-pytr

[Lockley139]

TLDR; I solved the problem with a custom patch You can find the patch on this link. Upvotes are strongly appreciated.

This didnt work for me unfortunately, after implementing into my script :(