New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configuration option to let a portion of hit traffic through Casper #7

ArnaudBrousseau opened this Issue Feb 27, 2018 · 0 comments


None yet
1 participant
Copy link

ArnaudBrousseau commented Feb 27, 2018

This idea stemmed from a discussion with @mattiskan. If a high QPS service is proxied through Casper, the service naturally gets provisioned less and less over time. However, if Casper is down, we're in a tight spot: the traffic gets forwarded to the underlying service (because of the "fail-safe" philosophy baked in proxied_through), the underlying service is under-provisioned and may error/time out, causing user-facing problems until either (a) Casper is brought back up or (b) sufficient capacity is added to the underlying service.

To avoid these situations, let's add a new per-namespace configuration option to let a fixed portion of hit traffic through. Something like hit_passthrough: 0.65 (name TBD)

In case of a cache hit, we'd still forward the request in Casper's post-request callback (with a 65% likelihood). This will not only let us ensure we keep some capacity in proxied services, but could be a useful tool to gauge whether a particular service would collapse if Casper were to be down (currently the only way to find out is to shut down Casper for real. Not ideal!)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment