Limit the maximum rate of requests per IP #614
Conversation
Configure the metrics endpoints to filter out requests that happen more than X times per second from the same IP to the same endpoint. The exact number is not decided yet but I'm tentatively setting it at 100. Right now, the filter is just logging if the limit is exceeded so we can get a feel for how often this happens. We may decide to make it a real filter as part of this PR or later. https://phabricator.wikimedia.org/T135240 Bug: T135240
|
Let us know if this is ok, but we should probably talk tomorrow to see if others agree with doing logOnly at first. |
| x-request-handler: | ||
| - get_from_backend: | ||
| request: | ||
| uri: '{{options.host}}/pageviews/aggregate/{project}/{access}/{agent}/{granularity}/{start}/{end}' | ||
| headers: | ||
| cache-control: '{{cache-control}}' | ||
| if-unmodified-since: '{{if-unmodified-since}}' |
There was a problem hiding this comment.
Out of curiosity, how would AQS use the cache-control and if-unmodified-since headers?
There was a problem hiding this comment.
I was just thinking they'd be convenient to have in general, but maybe you're right, maybe it doesn't make sense at our layer.
There was a problem hiding this comment.
In general we use cache-control to force a rerender, and if-unmodified-since to skip a forced rerender if requests come out of order. Since AQS data is practically immutable, you can't rerender, so unlikely you need them.
|
LGTM overall. Some of the forwarded headers (cache-control and if-unmodified-since) probably do nothing for you right now. |
|
I agree, I'll get those out, since there's no immediate use I can think of. |
Configure the metrics endpoints to filter out requests that happen more
than X times per second from the same IP to the same endpoint. The
exact number is not decided yet but I'm tentatively setting it at 100.
Right now, the filter is just logging if the limit is exceeded so we can
get a feel for how often this happens. We may decide to make it a real
filter as part of this PR or later.
https://phabricator.wikimedia.org/T135240
Bug: T135240