[Google Shopping API] Increased response times

[alexbarron]

We've been seeing increased response times for the Google Shopping API since roughly 15:00 UTC August 12. These peaked for several hours between 10-17s per request. These have now settled down around 6-8s per request, but this is still 2-3x than normal.

https://serpapi.com/status/google_shopping

Intercom
Intercom 2

[marm123]

Thanks @alexbarron. Adding the customer who reported this initially:

Intercom

[NateSkiles]

Another customer reached out with this issue:

Intercom

[sonika-serpapi]

Another customer reached out for this issue

Intercom

[alexbarron]

Received another report:

Front

[hilmanski]

A customer reported this.

Intercom

[InfiniteCodeMonkeys]

Hey guys, any movement on this? We still have response times above 20secs with ludicrous speed.

[tanys123]

Hi @InfiniteCodeMonkeys, we are working to adapt to the new layout change. As there are new sections introduced in the new layout, we are taking some time to design the API response. We'll let you know once it is deploy, thank you for your patient!

[tanys123]

We have deployed new changes to address the response times (mainly for searches in US region), you should notice faster response time on your end 🎉 . It is a new layout from Google, but we've tried our best to keep the JSON structure consistent. Please let us know if you encounter any issue 🙏

Example with Ludicrous Speed:

[fawazransari]

Hi, It looks like the new changes doesn't include the product link on results page. There's a product_link which is pointing to google shopping product page but not to the original source link

[marm123]

Hi @fawazransari, thanks for mentioning that! Unfortunately, it looks like the link is no longer provided by Google in the new layout. Please check this thread: #1845

I'll open a separate feature request to scrape a direct link with our Google Product API, as this is possible, but we'll no longer be able to provide the direct link with Google Shopping API.

I'm very sorry for the inconvenience.

[marm123]

@fawazransari, here is the feature request to scrape a direct link on Google Product API: #1846

[InfiniteCodeMonkeys]

Hey is there a reason for changing link in shopping_results to product_link? Would it be possible to revert that change?

[alexbarron]

Hey is there a reason for changing link in shopping_results to product_link? Would it be possible to revert that change?

Sorry for this trouble. Unfortunately, Google now only returns a Google link in the shopping results. They don't provide the direct link anymore in the new layout.

We may be able to support returning direct links in the Google Product API as we're tracking here: #1846

[marm123]

On top of what @alexbarron mentioned, I want to also note that the link endpoint has not been replaced with the product_link. The product_link was always returned along with the link, but unfortunately, Google no longer returns the latter.

We prioritized the feature to introduce the direct link in the Google Product API. While this will require you to adjust the way you're extracting the link value now, it seems like the only viable way to get the information you were getting with the old Google Shopping layout. I'm very sorry for the inconvenience.

[InfiniteCodeMonkeys]

Thanks for the clarification

[fawazransari]

If I understand it correctly, we'll have to make another serp call on Google Product API to get the link for all the products on Google Shopping results? Is there a possibility we can avoid that and get the link directly

[alexbarron]

If I understand it correctly, we'll have to make another serp call on Google Product API to get the link for all the products on Google Shopping results? Is there a possibility we can avoid that and get the link directly

That is correct. Unfortunately, this is out of our control due to the layout changes Google released to their Google Shopping results. More about this is available in this tread: #1845

I understand it's not an ideal solution. Since it's not possible to scrape this link anymore from Google Shopping results, we have to scrape it from Google Product pages which requires additional requests.

Sorry that there isn't a more elegant solution available.

[sonika-serpapi]

Customers have reached out asking for a better solution than making so many more requests to the Google Product API to get the direct link.

Intercom
Intercom

[tanys123]

We have made a new release that adds link to Google Shopping API. You have to enable it by passing direct_link=true. Feel free to try it out and feedback to us.

[tanys123]

After the release (new layout release), the response time has consistently improved compared to before. We also made another release to restore product's link that was no longer available in the new layout. I am closing this thread as completed. Feel free to send feedback to us.

status page

[hartator]

Really like that screenshot. Good job @tanys123. 👍