Improve ratelimit bucket handling

[MinnDevelopment]

Pull Request Etiquette

• I have checked the PRs for upcoming features/bug fixes.
• I have read the contributing guidelines.

Changes

• Internal code
• Library interface (affecting end-user code)
• Documentation
• Other: _____

Closes Issue: NaN

Description

Rewrite for the rate limiter to correctly handle buckets.

[MinnDevelopment]

The new rate limit system includes both the method and bucket hash for queues:

A bucket is determined via the Path+Method+Major in the following way:

1. Get Hash from Path+Method (we call this route)
2. Get bucket from Hash+Major (we call this bucketid)

If no hash is known we default to the unlimited+method/path hash. The hash is loaded from HTTP responses using the X-RateLimit-Bucket response header. This hash is per Method+Path and can be stored indefinitely once received. Some endpoints don't return a hash, this means that the endpoint is unlimited and will be in queue with only the major parameter.

To explain this further, lets look at the example of message history. The endpoint to fetch message history is GET/channels/{channel.id}/messages. This endpoint does not have any rate limit (unlimited) and will thus use the hash unlimited. The bucket id for this will be unlimited+GET/channels/{channel_id}/messages:guild_id:{channel.id}:webhook_id where {channel.id} would be replaced with the respective id. This means you can fetch history concurrently for multiple channels but it will be in sequence for the same channel.

If the endpoint is not unlimited we will receive a hash on the first response. Once this happens every unlimited bucket will start moving its queue to the correct bucket. This is done during the queue work iteration so many requests to one endpoint would be moved correctly. For example, the first message sending:

public void onReady(ReadyEvent event) {
  TextChannel channel = event.getJDA().getTextChannelById("123");
  for (int i = 1; i <= 100; i++) {
    channel.sendMessage("Message: " + i).queue();
  }
}

This will send 100 messages on startup. At this point we don't yet know the hash for this route so we put them all in "unlimited+POST/channels/{channel_id}/messages:guild_id:123:webhook_id". The bucket iterates the requests in sync and gets the first response. This response provides the hash for this route and we create a bucket for it. Once the response is handled we continue with the next request in the unlimited bucket and notice the new bucket. We then move all related requests to this bucket.

I have done some exhaustive testing on this and it seems to work properly now. We won't know how well it works until putting it into a real bot.

Edit: The unlimited hash now includes the route to restore old per-route concurrency.

[MinnDevelopment]

We should wait on a response to discord/discord-api-docs#1295