Protocol

Gatekeepers and masters communicate using a pair of ZeroMQ PUB/SUB sockets. Both are physically outgoing from the gatekeeper (bind() on the master, connect() on the gatekeeper), but one (the accounting socket) sends messages from gatekeeper to master, while the other (the control socket) sends messages from master to gatekeepers. Both sockets use the domain as the pubsub topic. A ZeroMQ router may be employed to shard accounting messages to multiple masters, provided that all messages for the same domain go to the same master.

When the master receives an accounting message, it applies the "Throttling logic" above to determine when the next request for the given identifier is allowed on the given domain, using the rules for that domain. If the next request is allowable right now it does nothing (except logging and statistics collection); otherwise it publishes a control message informing gatekeepers of the next allowable request timestamp for the identifier.

A gatekeeper must keep a memory of control messages received from the master. If a control message exists for the current domain and identifier with a timestamp after the current timestamp, then the gatekeeper must delay or reject the request. The decision as to whether to delay or reject can be made based on the length of the delay (requests that would be delayed for an unreasonably long time should be rejected instead) or based on resource considerations (delayed requests consume resources, and a gatekeeper may choose to reject further requests if these resources are exhausted).

In the context of HTTP, a request is "delayed" by keeping the client connection open, but delaying the presentation of the request to the resource until a later time. A request is "rejected" by sending the client a 503 "Service Unavailable" response (optionally with a Retry-After header indicating a later time at which a request would be acceptable), and preventing the request from reaching the resource.

A gatekeeper must send an "accepted" accounting message for each request it accepts, with the identifier and request timestamp.

A gatekeeper must send a "delayed" accounting message for each request it delays, with the identifier, request timestamp, and timestamp to which the request was delayed.

A gatekeeper may send a "rejected" accounting message for each request it rejects, with the identifier and request timestamp. Rejected accounting may be disabled or sampled if it is desirable to reduce network traffic or master load.

All accounting messages may have an optional final field which contains information of use for debugging (for example the client's User-Agent or Referrer headers). The format is unspecified but should be suitable for logging.