Refine wait period in bomr upgrades to avoid secondary rate limit

[snicoll]

This is a follow-up of #29879. It looks like that 1sec is not enough unfortunately.

> Task :spring-boot-project:spring-boot-dependencies:bomrUpgrade
Upgrade to Awaitility 4.2.0
Upgrade to Cassandra Driver 4.14.0
Upgrade to Couchbase Client 3.2.6
Upgrade to Dropwizard Metrics 4.2.9
Upgrade to Ehcache3 3.10.0
Upgrade to Elasticsearch 7.17.1
Upgrade to Flyway 8.5.4
Upgrade to Groovy 3.0.10
Upgrade to Hazelcast 5.1.1 (supersedes #30154 Upgrade to Hazelcast 5.1)
Upgrade to Hibernate 5.6.7.Final
Upgrade to Hibernate Validator 6.2.3.Final
Upgrade to HtmlUnit 2.59.0
Upgrade to Infinispan 13.0.8.Final
Upgrade to Jackson Bom 2.13.2
Upgrade to Liquibase 4.9.0
Upgrade to Log4j2 2.17.2
Upgrade to Maven Compiler Plugin 3.10.1
Upgrade to Maven Dependency Plugin 3.3.0
Upgrade to Mockito 4.4.0
Upgrade to Neo4j Java Driver 4.4.4
Upgrade to Netty 4.1.75.Final

> Task :spring-boot-project:spring-boot-dependencies:bomrUpgrade FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':spring-boot-project:spring-boot-dependencies:bomrUpgrade'.
> 403 Forbidden: "{"message":"You have exceeded a secondary rate limit and have been temporarily blocked from content creation. Please retry your request again later.","documentation_url":"https://docs.github.com/rest/overview/resources-in-the-rest-api#secondary-rate-limits"}"

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 7m 0s

[wilkinsona]

Through the improved diagnostics, we can see the following headers in the response:

Server: GitHub.com
Date: Thu, 07 Jul 2022 16:37:26 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 258
X-OAuth-Scopes: repo, workflow
X-Accepted-OAuth-Scopes: 
X-GitHub-Media-Type: github.v3
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4977
X-RateLimit-Reset: 1657215394
X-RateLimit-Used: 23
X-RateLimit-Resource: core
Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset
Access-Control-Allow-Origin: *
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
X-Frame-Options: deny
X-Content-Type-Options: nosniff
X-XSS-Protection: 0
Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
Content-Security-Policy: default-src 'none'
Vary: Accept-Encoding, Accept, X-Requested-With
X-GitHub-Request-Id: E21C:5F26:36BF77:38FA6A:62C70BC6

Unfortunately, there's nothing there to tell us how long to wait before making another content creation request.

[mhalbritter]

There are the X-RateLimit headers: X-RateLimit-Reset specifies

The time at which the current rate limit window resets in UTC epoch seconds.

See here: https://docs.github.com/en/rest/overview/resources-in-the-rest-api?apiVersion=2022-11-28#rate-limit-headers

So if we hit the rate limit, it should be enough to wait until X-RateLimit-Reset is in the past and then try again.

[wilkinsona]

Unfortunately, the limit that we're hitting is a "secondary" rate limit to which the X-RateLimit and X-RateLimit-Reset headers do not appear to apply. The headers above are taken from a 403 Forbidden response and the rate limit described in the headers shows 4977 requests remaining.

[mhalbritter]

Ah, thanks for clarification.

[mhalbritter]

I played a bit around with it, and it seems that waiting 3 seconds between each write request is enough to not trigger the rate limit.