Preloading features #198
Preloading features #198
Conversation
When N features are being checked in a single request this would result in N requests to the underlying store at best with memoization enabled. This commit adds `get_multi` to the adapter protocol, backfilled to all adapters through the Flipper::Adapter module implementing it as multiple `get` calls. This allows adapters to optimize `get_multi` when possible to improve the performance of retrieving multiple flags. To take advantage of this `preload` and `preload_all` were added to the DSL, which allow the specified features to eagerly memoized. A new eager version of the memoizing middleware was added to demonstrate this. `get_multi` is currently only implement for the Memoizable and Redis adapters.
We've starting running this in production and it seems to be having the desired effect. |
Super helpful to see this. I definitely got a few ideas by having some code in front of me. I like the overall approach. I am keen to hear your thoughts on the middleware specifically. Also, I would love to see tests for preload, preload_all, get_multi and the new middleware (or whatever we go with there). Do you have time to add those? If not, I can work on seeing this through, but it might be a bit before I have time. I am definitely down to merge this with tests and a few minor tweaks and I really appreciate the thoughtful PR. |
@@ -0,0 +1 @@ | |||
2.3.0-dev |
jnunemaker
Nov 19, 2016
Owner
What are your thoughts on removing this and dropping .ruby-version in .gitignore?
What are your thoughts on removing this and dropping .ruby-version in .gitignore?
gshutler
Nov 19, 2016
Author
Contributor
Sure, I didn't mean to commit it in the first place!
Sure, I didn't mean to commit it in the first place!
features | ||
end | ||
|
||
def preload_all |
jnunemaker
Nov 19, 2016
Owner
I'm on the fence about encouraging this as it could become equally as damaging as getting things one by one, but I can always add a caveat to the caveats.md file about being careful to not have too many features while using preload_all.
I'm on the fence about encouraging this as it could become equally as damaging as getting things one by one, but I can always add a caveat to the caveats.md file about being careful to not have too many features while using preload_all.
gshutler
Nov 19, 2016
Author
Contributor
I understand, I think we'll be able to get rid of it.
I understand, I think we'll be able to get rid of it.
|
||
module Flipper | ||
module Middleware | ||
class EagerMemoizer |
jnunemaker
Nov 19, 2016
Owner
What do you think about calling this Flipper::Middleware::PreloadAll?
Also, what do you think about having the preload all memoizer only call preload_all
. We could then recommend including the memoizer first in the stack and PreloadAll
second. It would keep logic in the same place.
In fact, I think we could even set an env var like flipper.memoizing in the memoizing middleware and then log a warning to stderr or something here if it is missing, as a way of letting people know they are preloading without memoization.
Another option, we could just add a preload_all option to the memoizing middleware so people could just do:
use Flipper::Middleware::Memoizer, preload_all => true
# or
use Flipper::Middleware::Memoizer, preload: [:my, :collection, :of, :features, :to, :preload]
What do you think about calling this Flipper::Middleware::PreloadAll?
Also, what do you think about having the preload all memoizer only call preload_all
. We could then recommend including the memoizer first in the stack and PreloadAll
second. It would keep logic in the same place.
In fact, I think we could even set an env var like flipper.memoizing in the memoizing middleware and then log a warning to stderr or something here if it is missing, as a way of letting people know they are preloading without memoization.
Another option, we could just add a preload_all option to the memoizing middleware so people could just do:
use Flipper::Middleware::Memoizer, preload_all => true
# or
use Flipper::Middleware::Memoizer, preload: [:my, :collection, :of, :features, :to, :preload]
gshutler
Nov 19, 2016
Author
Contributor
I like the idea of making it into an option of Memoizer
as it would avoid them being able to be registered in the wrong order and so on.
I like the idea of making it into an option of Memoizer
as it would avoid them being able to be registered in the wrong order and so on.
Also, if you don't mind, I would love to know what you are using flipper for as well and how it is working out for you (what went well, what was hard). If you aren't comfortable leaving that information here in a public comment, feel free to email me directly (it shows up on my profile page). I'm just always curious. :) |
Happy to write some tests, I also plan to have a look at how to implement We use Flipper at Cronofy for all sorts. Hiding UI features, easing in new features to our synchronisation layer, making new endpoints/data available to specific clients during development, experimenting with new implementations of things that should be transparent. The big thing that triggered this work for us is we're changing our data storage and so rather than generally 4-5 flags in play at any time we've got 20+. We were seeing roughly 1ms per feature flag fetch and with API endpoints that respond in around 90ms adding 20ms really shifted the needle. Using this approach has led to ~4ms on every request. Flipper was easy to get going with, probably the fiddliest thing was wrapping things to expose a Any other feedback from our use @stephenbinns? |
The behavior will be added to the default memoizer via configuration.
@jnunemaker I think this addresses your key points. I'm not sure if I've added tests in all the places you would want them or not so any pointers you can give would be good. I also plan to look into implementing |
Worked out where to add the spec that confirms |
response = @app.call(env) | ||
response[2] = Rack::BodyProxy.new(response[2]) { | ||
flipper.adapter.memoize = original | ||
} | ||
response | ||
rescue |
jnunemaker
Nov 21, 2016
Owner
One question: what was this added for?
One question: what was this added for?
gshutler
Nov 21, 2016
Author
Contributor
One of the spec groups I added failed one of the shared examples. This resolved that failure but I couldn't really see why it was needed if I'm honest. If you take it out you'll see the failure and perhaps be able to solve it some other way.
One of the spec groups I added failed one of the shared examples. This resolved that failure but I couldn't really see why it was needed if I'm honest. If you take it out you'll see the failure and perhaps be able to solve it some other way.
gshutler
Nov 22, 2016
Author
Contributor
Ah, I see from bdbee31 I was misreading the BodyProxy
, thinking it was wrapping the whole @app.call
not just the body. Makes sense now why it was needed.
Ah, I see from bdbee31 I was misreading the BodyProxy
, thinking it was wrapping the whole @app.call
not just the body. Makes sense now why it was needed.
Sorry just got round to replying to this I don't really have much to add to what @gshutler's already said about our usage. The defaults are sensible and we've had no real issues other than when we've introduced a large number of flags. One thing we've noticed from our usage is we rarely use the 75% enabled on the UI as we generally would tend towards going from 50% -> 100% (as usually by 50% you're pretty committed). For us the biggest feature gap is the audit trail (I see its already on the issues list) we use Slack for most of our integrations, so we might have a look into hooking into the UI to send notifications on flag changes. |
Some code to help the discussion in #190
When N features are being checked in a single request this would result in N requests to the underlying store at best with memoization enabled.
This PR adds
get_multi
to the adapter protocol, backfilled to all adapters through theFlipper::Adapter
module implementing it as multipleget
calls.This allows adapters to optimize
get_multi
when possible to improve the performance of retrieving multiple flags.To take advantage of this
preload
was added to the DSL, which allows the specified features to be eagerly memoized. New configuration options were added to the memoizing middleware to take advantage of this.get_multi
is currently only implement for theMemoizable
andRedis
adapters.