Status code was not 200. Returned status code:500 #451

jdeverdun · 2023-07-04T07:57:30Z

Hello everyone,

Since a few day I've got the error (on any request):
Error in interest_over_time(widget, comparison_item, tz) :
Status code was not 200. Returned status code:500

I first though it was a daily limit, yet I still havethis issue on other computer. Am I the only one ?

EDIT : the issue seems to be related to the time range, it fails when I give for example
a=gtrends( 'bike' , geo = 'FR' ,time= 'now 1-d' , low_search_volume = TRUE)

but it "does not fail" when date range is specified.

I guess it is related to google trends itself, but wanted to make sure !

Thanks for your help.

Best regards

The text was updated successfully, but these errors were encountered:

lukschue · 2023-07-04T09:24:35Z

Hello jdeverdun,

You are not the only one. I gather Google Trends data automatically on a daily basis for a project and since today I got the same Error code for all my request. Though until yesterday it still seemed to work out for me.

And also when the range is specified it does still work.

I hope this problem is solvable.

Kind regards.

PMassicotte · 2023-07-04T10:40:24Z

I am also getting the error, but only for the "past day" search. I will investigate this.

jdeverdun · 2023-07-04T11:25:13Z

Just to let you know I tried various tools, like pytrends and even serpAPI but same, I guess google is doing some work

shrikant2002 · 2023-07-04T13:01:50Z

same happening with me since 3 days

ccharisis · 2023-07-04T14:41:31Z

Same issue for me. Started yesterday.

ccharisis · 2023-07-05T07:07:18Z

I am also getting the error, but only for the "past day" search. I will investigate this.

Hi! Do you have any update on the issue? I am still getting the same error when I use "now" time formats like "now 1-H" or "now 1-d" etc. The other time formats like 'today 1-m' seem to work fine. So it looks like an issue that affects only the "Last" n hours /days and not the "Past" queries. Please let me know if you have any updates on the problem.

xritzim · 2023-07-05T09:01:59Z

Hello, I'm experiencing the same issue. I get the status code 500.

PMassicotte · 2023-07-05T12:12:20Z

This is what we are sending:

{
  "userConfig": {
    "userType": "USER_TYPE_SCRAPER"
  },
  "locale": "en-US",
  "comparisonItem": [
    {
      "geo": {
        "country": "FR"
      },
      "complexKeywordsRestriction": {
        "keyword": [
          {
            "type": "BROAD",
            "value": "bike"
          }
        ]
      }
    }
  ],
  "resolution": "EIGHT_MINUTE",
  "requestOptions": {
    "category": 0,
    "backend": "CM",
    "property": ""
  },
  "time": "2023-07-04T12\\:03\\:43 2023-07-05T12\\:03\\:43"
}

This is what Google Trends (website) is sending:

{
  "time": "2023-07-04T11\\:36\\:47 2023-07-05T11\\:36\\:47",
  "resolution": "EIGHT_MINUTE",
  "locale": "en-US",
  "comparisonItem": [
    {
      "geo": {
        "country": "FR"
      },
      "complexKeywordsRestriction": {
        "keyword": [
          {
            "type": "BROAD",
            "value": "bike"
          }
        ]
      }
    }
  ],
  "requestOptions": {
    "property": "",
    "backend": "CM",
    "category": 0
  },
  "userConfig": {
    "userType": "USER_TYPE_LEGIT_USER"
  }
}

Besides userType, this is the same. So I do not know what is going on at the moment. Note that userType is also set to USER_TYPE_SCRAPER when we use other time ranges such as date=today 5-y so this is not obvious that this is the cause of the problem.

jdeverdun · 2023-07-05T12:16:49Z

Thanks for this information, I also checked on the google trends website and in fact we can see that if you try to access the embed code for the interest by city map, we also have a 505 error for this kind of request (still working fine for specific date request).

So again, google should be working on it.

Best

PMassicotte · 2023-07-05T12:58:17Z

I will keep an eye open for this.

ccharisis · 2023-07-05T13:17:33Z

I will keep an eye open for this.

Thank you.

Helldez · 2023-07-13T07:55:19Z

Hi, does anyone have any updates on this? It seems to be a Google Trends issue, but does anyone know if it's transient or we need to find another way to fetch the weekly and daily data?

JBleher · 2023-07-13T09:35:48Z

I tried to enhance the request by adding the part:
"userConfig": { "userType": "USER_TYPE_LEGIT_USER" }

However, then I get a HTML-401 error. I guess one needs to send a cookie or something with the request for the widget to the get the status USER_TYPE_LEGIT_USER.

PMassicotte · 2023-07-13T11:19:48Z

I do not see an easy fix at the moment. Any ideas are welcome :)

Helldez · 2023-07-14T07:06:47Z

I'm not a technician but could it be due to these errors on the google trends website?

ccharisis · 2023-07-17T12:50:22Z

I just noticed that the response changed from 500 (Internal Server Error) to 429 (Too many requests) .

Draptol · 2023-07-17T15:34:29Z

yeah in node.js lib I have the same problem. 500 http is replaced by 429 even if I use residential proxies.

MattCowgill · 2023-07-20T04:18:39Z

Long time {gtrendsR} user here, now encountering this same problem for the first time :(

Helldez · 2023-07-21T19:57:55Z

Same here, it has been going on for three weeks

barbara1998 · 2023-07-24T13:29:38Z

EDIT: We have a scheduler running to gather google trends data automatically on a daily basis, using now 1-d. This has not worked for the last 3 weeks, but since July 22, 5 am in the morning, it produced datasets again. But now when I run the script manually, it returns the "Status code was not 200. Returned status code:429" error code.

PMassicotte · 2023-07-24T13:42:19Z

@barbara1998 You are able to use now 1-d ?

barbara1998 · 2023-07-24T15:01:25Z

@PMassicotte sorry, maybe a false alarm, I made an edit to my comment.

PMassicotte · 2023-07-24T16:58:17Z

Tx for reporting.

janinepdasilva · 2023-07-26T10:49:33Z

I'm having the same issue.

xritzim · 2023-08-02T09:01:22Z

Hello all!! It seems it works now!!

ccharisis · 2023-08-02T09:09:10Z

I confirm, the API is functional!

PMassicotte · 2023-08-02T13:03:24Z

Oh good to know! Anyone has information on what happened?

lukschue · 2023-08-22T11:09:39Z

Hey everyone,

as I also mentioned in #452 I run several gtrendsR request automatically on a daily basis. Since the 16.08.2023 the problem from the 04.07. seems to repeat itself. Since then there is no new data.

danielasilver · 2023-10-13T15:22:49Z

Encountering this issue today for the first time:

gtrends("Biden", gprop = 'web', geo = 'US',time = "2023-02-15 2023-04-14" )
Error in interest_over_time(widget, comparison_item, tz) :
Status code was not 200. Returned status code:429

Tried various time ranges, got same result, except for on the second try, which worked:

gtrends("Biden", gprop = 'web',geo = 'US', time = "now 1-d" )

The first attempt with the above code resulted in:

Error in interest_over_time(widget, comparison_item, tz) :
Status code was not 200. Returned status code:500

PMassicotte · 2023-10-13T19:48:37Z

Not sure what is going on, working on my side:

library(gtrendsR)
plot(gtrends("Biden", gprop = 'web', geo = 'US',time = "2023-02-15 2023-04-14" ))

^{Created on 2023-10-13 with reprex v2.0.2}

danielasilver · 2023-10-13T20:20:37Z

thank you. not sure what is going on.
this works:
plot(gtrends("Biden", gprop = 'web', geo = 'US',time = "2023-02-15 2023-04-14" ))

but this does not:

plot(gtrends("Biden", gprop = 'web', geo = 'US',time = "all" ))
Error in FUN(X[[i]], ...) :
Status code was not 200. Returned status code:429

Nor does this:

plot(gtrends("Biden", gprop = 'web', geo = 'US',time = "today 3-m" ))
Error in FUN(X[[i]], ...) :
Status code was not 200. Returned status code:429

Nor does this, my real interest:

place2 <- gtrends(keyword = c("/m/0c3kw5"), geo = "", time="all", low_search_volume = F)
Error in FUN(X[[i]], ...) :
Status code was not 200. Returned status code:429

waynelapierre · 2023-10-16T16:56:52Z

I also encountered this issue:

gtrends("nuclear war", geo = "US", gprop = "web")
Error in FUN(X[[i]], ...) : 
  Status code was not 200. Returned status code:429

DataSocialist · 2023-10-17T18:27:55Z

I encounter the same problem since today.

gtrends(keyword = "svp",time = "now 1-d", geo = "CH",cookie_url = "http://trends.google.com/Cookies/NID", tz=Sys.timezone())
Error in interest_over_time(widget, comparison_item, tz) :
Status code was not 200. Returned status code:429Error in interest_over_time(widget, comparison_item, tz) :
Status code was not 200. Returned status code:429

jdeverdun · 2023-10-19T15:28:55Z

yep same here, on few request are accepted.

Helldez · 2023-10-20T08:26:20Z

Same here

eddelbuettel · 2023-10-20T08:38:32Z

It is failing for me too, on two machines including one with a rebuilt-from-git installation, and as an additional check also in a fresh Docker run (i.e. no possible stale cookie).

jdeverdun · 2023-11-27T11:32:35Z

Anyone has any updates ?

JBleher · 2023-11-27T11:52:49Z

the widget now contains the line:
"userConfig":{"userType":"USER_TYPE_SCRAPER"}
the google request sends:
"userConfig":{"userType":"USER_TYPE_LEGIT_USER"}

This might be what is causing the problem...

PMassicotte · 2023-11-27T12:14:10Z

#440 (comment)

jdeverdun · 2023-11-27T12:26:19Z

Just saw that embed trends were working again. I was hoping something was unlocked 😭

PMassicotte · 2023-11-27T14:37:45Z

Just tried to force using

  payload2$userConfig$userType <- "USER_TYPE_LEGIT_USER"

No success, this will require more investigation.

pensivedog · 2024-01-09T08:47:46Z

Still not working in 2024:

Error in interest_over_time(widget, comparison_item, tz) : 
  Status code was not 200. Returned status code:429

Anyone find a solution?

jdeverdun · 2024-01-09T08:54:39Z

In fact it was working back a few days ago, but the issue came back yesterday for everyone

BigTimeStats · 2024-03-06T17:04:27Z

Same issue for me this week -- does not work no matter the keyword or delay.

Is there any way to refresh the cookies? It seemed like a package reinstall fixed the issue, at least temporarily for a few test runs

EDIT: not working again even after reinstall.

ytmben · 2024-03-13T08:26:05Z

Same issue for me, tried to run K0 <- gtrends(keyword = term, geo = country, time = period, onlyInterest = TRUE)
and received the same error:
Error in interest_over_time(widget, comparison_item, tz) : Status code was not 200. Returned status code:429

PMassicotte · 2024-03-13T11:10:39Z

Please make a reproducible example. This gtrends("nhl") is working on my side.

ytmben · 2024-03-15T11:21:14Z

term = c("Mediterranean diet", "Veganism", "Vegetarianism", "Gluten-free diet", "Low-carbohydrate diet")
country = ""
period = "2010-01-01 2023-12-31"
K0 <- gtrends(keyword = term, geo = country, time = period, onlyInterest = TRUE)

PMassicotte · 2024-03-15T12:40:11Z

Works fine on my side.

library(gtrendsR)

term <- c("Mediterranean diet", "Veganism", "Vegetarianism", "Gluten-free diet", "Low-carbohydrate diet")
country <- ""
period <- "2010-01-01 2023-12-31"
K0 <- gtrends(keyword = term, geo = country, time = period, onlyInterest = TRUE)

plot(K0)

^{Created on 2024-03-15 with reprex v2.1.0}

Mettez14 · 2024-03-15T14:52:36Z

Works fine on my side.

library(gtrendsR)

term <- c("Mediterranean diet", "Veganism", "Vegetarianism", "Gluten-free diet", "Low-carbohydrate diet")
country <- ""
period <- "2010-01-01 2023-12-31"
K0 <- gtrends(keyword = term, geo = country, time = period, onlyInterest = TRUE)

plot(K0)

Created on 2024-03-15 with reprex v2.1.0

Your specific example also produces the error for me:

term <- c("Mediterranean diet", "Veganism", "Vegetarianism", "Gluten-free diet", "Low-carbohydrate diet")
country <- ""
period <- "2010-01-01 2023-12-31"
K0 <- gtrends(keyword = term, geo = country, time = period, onlyInterest = TRUE)
Error in interest_over_time(widget, comparison_item, tz) :
Status code was not 200. Returned status code:429

ytmben · 2024-03-15T17:21:13Z

Now my previous example works oddly enough, but I am having trouble with the exact next line of code:
K1 <- gtrends(keyword = term[1], geo = country, time = period, onlyInterest = TRUE)
It again returns the same error
Edit: I just re-tried it, now it works

eddelbuettel · 2024-03-15T17:23:50Z

@ytmben Keep in mind that 'experienced behaviour'' will likely depend on both your account, its cookies, past use, ... and what happens at the backend. In short, "we do not know" and this is not an official or sanctioned API anyway.

We can only assert sometimes that "it works for us" which may suggest the code is not broken, but does not generally imply that it must work for you once or repeatedly.

ytmben · 2024-03-15T18:05:13Z

@eddelbuettel Ok, understood, thanks for the clarification

cpaulsanders mentioned this issue Jul 5, 2023

Requests for hourly data (< 7 days) causes 500 server error pat310/google-trends-api#171

Open

Terseus mentioned this issue Jul 5, 2023

500 internal server errors GeneralMills/pytrends#586

Closed

Repository owner deleted a comment from shrikant2002 Jul 6, 2023

lukschue mentioned this issue Aug 22, 2023

inconsistencies with pulling data automatically #452

Closed

Status code was not 200. Returned status code:500 #451

Status code was not 200. Returned status code:500 #451

Comments

jdeverdun commented Jul 4, 2023 • edited Loading

lukschue commented Jul 4, 2023

PMassicotte commented Jul 4, 2023

jdeverdun commented Jul 4, 2023

shrikant2002 commented Jul 4, 2023

ccharisis commented Jul 4, 2023

ccharisis commented Jul 5, 2023

xritzim commented Jul 5, 2023

PMassicotte commented Jul 5, 2023

jdeverdun commented Jul 5, 2023

PMassicotte commented Jul 5, 2023

ccharisis commented Jul 5, 2023

Helldez commented Jul 13, 2023

JBleher commented Jul 13, 2023

PMassicotte commented Jul 13, 2023

Helldez commented Jul 14, 2023 • edited Loading

ccharisis commented Jul 17, 2023 • edited Loading

Draptol commented Jul 17, 2023

MattCowgill commented Jul 20, 2023

Helldez commented Jul 21, 2023

barbara1998 commented Jul 24, 2023 • edited Loading

PMassicotte commented Jul 24, 2023

barbara1998 commented Jul 24, 2023

PMassicotte commented Jul 24, 2023

janinepdasilva commented Jul 26, 2023

xritzim commented Aug 2, 2023

ccharisis commented Aug 2, 2023

PMassicotte commented Aug 2, 2023

lukschue commented Aug 22, 2023

danielasilver commented Oct 13, 2023 • edited Loading

PMassicotte commented Oct 13, 2023

danielasilver commented Oct 13, 2023

waynelapierre commented Oct 16, 2023

DataSocialist commented Oct 17, 2023

jdeverdun commented Oct 19, 2023

Helldez commented Oct 20, 2023

eddelbuettel commented Oct 20, 2023

jdeverdun commented Nov 27, 2023

JBleher commented Nov 27, 2023

PMassicotte commented Nov 27, 2023

jdeverdun commented Nov 27, 2023

PMassicotte commented Nov 27, 2023

pensivedog commented Jan 9, 2024

jdeverdun commented Jan 9, 2024

BigTimeStats commented Mar 6, 2024 • edited Loading

ytmben commented Mar 13, 2024

PMassicotte commented Mar 13, 2024

ytmben commented Mar 15, 2024 • edited Loading

PMassicotte commented Mar 15, 2024

Mettez14 commented Mar 15, 2024

ytmben commented Mar 15, 2024 • edited Loading

eddelbuettel commented Mar 15, 2024 • edited Loading

ytmben commented Mar 15, 2024

jdeverdun commented Jul 4, 2023 •

edited

Loading

Helldez commented Jul 14, 2023 •

edited

Loading

ccharisis commented Jul 17, 2023 •

edited

Loading

barbara1998 commented Jul 24, 2023 •

edited

Loading

danielasilver commented Oct 13, 2023 •

edited

Loading

BigTimeStats commented Mar 6, 2024 •

edited

Loading

ytmben commented Mar 15, 2024 •

edited

Loading

ytmben commented Mar 15, 2024 •

edited

Loading

eddelbuettel commented Mar 15, 2024 •

edited

Loading