Ensure flags, switches, and samples are read from write DB #249
Conversation
|
Ah, yes that's definitely a concern, but I don't know if the right thing to do is throw all that read traffic at the write DB—at least without letting users opt out of that. What about write-through cache updates? |
|
I'm definitely open to taking that approach. It makes me a little nervous (but only a little) because there's technically still a race condition in that case where the DB and cache could get out of sync if there are two near-simultaneous writes. I will also point out that "all that read traffic" is kept to an absolute minimum by waffle's design. Because of the way waffle is implemented, it almost never hits the DB for switches or samples (depending on cache expiration of course). That said if your preference in light of these considerations is to take the write-through approach just say the word and I'll be happy to create a different PR that goes that route. |
|
Oh and I am also happy to keep this as is but add support for a Django setting (maybe something like READ_SWITCHES_FROM_WRITE_DB) to make this behavior configurable. |
|
True, you make good points. Let's put it behind a setting (defaulting to the current behavior) and add a note to the docs? |
|
OK, I've updated the PR to make the behavior configurable (disabled by default) via the Note that I also extended this setting to flags along with switches and samples, as after taking a closer look at how flags work I realized this makes just as much sense for flags. Let me know if you need me to squash this into a single commit before it can be merged. |
In an environment with sufficiently high traffic and DB replication set up, there is a race condition where when an update is immediately followed by a read, the read will get a stale value from the DB replica and cache it. This fixes the race condition by ensuring that flags, switches, and samples are always read from the write DB.
dtao
force-pushed
the
read-switches-and-samples-from-write-db
branch
from
9bdb3e1
to
f4d6243
Compare
dtao
changed the title
|
Update: I went ahead and squashed the commits in this PR, since that's what it says to do in CONTRIBUTING.rst. |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
This issue has been closed as stale because it has not had any recent activity. Please feel free to re-open if the issue is still relevant. Thank you for your contributions! |
|
Is it possible to re-open this? I definitely understand having an automated process for closing stale PRs; but unless I'm missing something this PR is basically good to merge and was just waiting on a maintainer to make sure it looked good and click the button. |
|
Yes, re-opening. Honestly I didn't expect the bot to apply to PRs, we'll probably need to tweak it a bit, and I'm going to go back through the PRs it closed and re-open still-relevant ones. |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
Oh, stale bot... |
|
I need to tweak those settings, it's using the defaults and that's too aggressive. |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
One of these days ;) |
|
Are there any actionable items before this can go in? |
|
Sweet! Out of curiosity, what's the process for releasing new versions? Is that entirely up to the discretion of @jsocol or should I be doing something (like creating a version bump PR)? No rush, just making sure I'm not dropping the ball if it's supposed to be up to me at this point. |
|
Me or any of the other maintainers. I’ll probably have time to take a look at it later during this holiday weekend. |
In an environment with sufficiently high traffic and DB replication set
up, there is a race condition with switches and samples where when an
update is immediately followed by a read, the read will get a stale
value from the DB replica and cache it.
This fixes the race condition by ensuring that switches and samples are
always read from the write DB.