Suggested way to handle read latency?

[KensoDev]

Hey,

Using the gem in Amazon RDS.
When Amazon RDS sets up a read slave, there are times when you have 1 minute of replication latency.

Even though 1 minute is a rare case, 10 seconds latencies happen a few times a day.

From looking and the source code, this is obviously not handles, so I am thinking about implementing a Dirty kind of flag, so when the flag is true, you also read from the master.

Opinions? Advice?

[jer0m]

Did you try this:

Octopus.using(:master) do
// Your queries
end

You force the queries to be executed on your master. Useful to avoid replication latency in certain cases.

[KensoDev]

I don't want to foce EVERYTHING to the master, that's missing the point of replication.
the problem is that let's say I write a review, when I refresh I expect to see it on the page right?

Now, if there's a 3 second read latency, I will not see my review which will result in me thinking it's a bug (which it is)

What I am trying to think about is a dirty flag, when you write something to the database, you will read from master for the next 30 seconds.

The problem is doing it across servers on a load balancer without a sticky session.

[nowthatsamatt]

Well KensoDev, you ever get something going on this? Running into the same spot.

[KensoDev]

@nowthatsamatt yeah, actually I did.
I did not get a chance to open source it yet, but here's the gist of things

first, in my app, only logged in users can write to the database, never logged out users.
So, in an around_filer, I inject the current_user into Octopus.

When there's a write to the database, I fire an even using ActiveSupport::Notifications with the user that wrote to the database.

I save it to redis and I force to master for that user for 5 minutes.

[nowthatsamatt]

We're using MySQL but this gave me something to think about - thank you!

[KensoDev]

@nowthatsamatt We are using mySql as well, I am only using Redis as a faster data layer, and it's really a key-value need.

"user:#{user_id}:last_write" => DateTime.now

[sobrinho]

I will do some research this week about how master/slaves databases (not applications) works in that situation.

[dreyks]

Is there any advancement in this feature?

[sobrinho]

I recently did a research about that and there is multiple ways to solve that.

If you need to handle the delay only for the current user, you could use redis to store the last write like "user:#{user.id}:last_write" => Time.current (as @KensoDev suggested).

And using a around filter you could use a threshold like this:

around_filter :handle_master_slave

def handle_master_slave
  last_write = Time.parse(Redis.get("user:#{user.id}:last_write"))
  threshold = 5.minutes # Tune to an appropriate threshold

  if Time.current + threshold < last_write
    Octopus.using(:master, &block)
  else
    Octopus.using(:slave, &block)
  end
end

There is a lot of other options like do that only for specific tables, objects, all users and etc. You must analyse your case to chose the appropriated solution.

I'm planning to write a wiki page about that ASAP.