Configurable server throttle

[rmichela]

Out of the box, gRPC will use unlimited resources. By default it uses an executor with an unbounded work queue and an unlimited number of worker threads.

A configurable server throttle would allow you to constrain these dimensions for a server.

• Set a server's max concurrency
• Set the max amount or waiting work before turning away requests

If a request gets turned away because of too much work, reply with a UNAVAILABLE status.

[rmichela]

See: https://groups.google.com/forum/#!topic/grpc-io/LrnAbWFozb0

[rmichela]

Netflix did it!

• https://medium.com/@NetflixTechBlog/performance-under-load-3e6fa9a60581
• https://github.com/Netflix/concurrency-limits