Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Configuration option to let a portion of hit traffic through Casper #7
This idea stemmed from a discussion with @mattiskan. If a high QPS service is proxied through Casper, the service naturally gets provisioned less and less over time. However, if Casper is down, we're in a tight spot: the traffic gets forwarded to the underlying service (because of the "fail-safe" philosophy baked in proxied_through), the underlying service is under-provisioned and may error/time out, causing user-facing problems until either (a) Casper is brought back up or (b) sufficient capacity is added to the underlying service.
To avoid these situations, let's add a new per-namespace configuration option to let a fixed portion of hit traffic through. Something like
In case of a cache hit, we'd still forward the request in Casper's post-request callback (with a 65% likelihood). This will not only let us ensure we keep some capacity in proxied services, but could be a useful tool to gauge whether a particular service would collapse if Casper were to be down (currently the only way to find out is to shut down Casper for real. Not ideal!)