Implement rate-limit by IP address for critical endpoints

[nabokihms]

Is your feature request related to a problem?

On every request to auth/healthz endpoint, dex creates an authrequest entity in storage. An intruder with any HTTP load generator can cause an outage.

Describe the solution you'd like to see

It would be marvelous to add middleware to limit requests on certain endpoints using the token/bucket algorithm.

Describe alternatives you've considered

We can add a doc with suggestions to configure some external tool, e.g., proxy, load balancer. Users need to know which endpoints they need to secure and how.

However, with this variant, we have no option using just dex (and limit all requests in one place).

Additional context

It causes a lot of pain in environments without proxy in front of a dex. I have faced that problem myself.
There is a comment suggesting the same.

[nabokihms]

Slightly related to #1941

[sagikazarmark]

I think it'd be better to move the healthz endpoint to the telemetry server instead. It shouldn't be public in the first place.

As for other rate limits: I'm not a huge fan of tackling this issue from applications. Although Dex is not technically HA at the moment, it would perfectly make sense to run it in a HA setup. Then you need a distributed storage to store the bucket for rate limiting.

IMHO Dex should rarely be running on it's own. There is usually a proxy/load balancer more equipped to handle rate limiting.

[nabokihms]

@sagikazarmark thanks for the quick answer. I have two questions:

1. Can you open the issue about problems you have faced running dex in HA mode? We have a few HA installations on top of Kubernetes.
2. Frankly, I kinda agree with you about running dex on its own, but it's not good to keep the rate-limit problem concealed. What do you think about adding solutions to the documentation?

[sagikazarmark]

Can you open the issue about problems you have faced running dex in HA mode? We have a few HA installations on top of Kubernetes.

It's more of a theory than actual issues. The gc job inside Dex can technically make instances step on each others toes.
In a HA environment running the GC as a cron job outside of Dex might be better.

What do you think about adding solutions to the documentation?

That's exactly what I was aiming at. Not necessarily concrete solutions, but overall definition of the problem with some general suggestions. Either from an attack vector or an operational best practices point of view. Documenting the related use cases (public vs internal Dex instances) might also be a good approach.