"Too many requests to statamic.com" error in high traffic environments

[jojotl]

Bug Description

We are seeing the too many requests to statamic.com banner popping up frequently in environments that serve a large number of concurrent control panel requests.

The underlying issue seems to be a race condition that triggers, when multiple control panel requests initiate a request to outpost.statamic.com at the same time. Depending on the number of concurrent requests for some of them a 429 error will be returned from the Outpost API. With a large number of concurrent requests there is a high chance that a 200 response doesn't make in into the cache, because it will be overwritten by one of the 429 responses.

How to Reproduce

1. Login to statamic CP
2. Purge cache entry for statamic.outpost.response
3. Trigger concurrent requests by running something like this in the console:
   for (let i=0;i<25;i++) {fetch('https://[statamic]/cp/dashboard')}

Environment

Statamic version: 3.1.11

PHP version: 7.3

Install method (choose one):
custom

[ceesvanegmond]

Having the same issues. We're working with multiple developers in the same building. So the outgoing IP for all devs are the same. Getting an error 429. Which is quite annoying

.

[geertjanknapen1]

I'm going to necro this, I know it's old but I am facing the same issues.