Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Have SRANDMEMBER and RANDOMKEY accept a seed #870

Open
tobsn opened this Issue Jan 9, 2013 · 1 comment

Comments

Projects
None yet
2 participants

tobsn commented Jan 9, 2013

(Feature request)

SRANDMEMBER myset [count] [seed]

SRANDMEMBER myset
// return one random member
SRANDMEMBER myset 2
// return two random members
SRANDMEMBER myset 1 "723432"
// return one random member, randomness based on seed
// first attribute could be switched to seed if second attribute is string not int (count of members vs. "seed")

RANDOMKEY [seed]

RANDOMKEY
// return one random key
RANDOMKEY "723432"
// return one random key, randomness based on seed

The problem hit me today as I want to have a random set member but I want to have the same random set member based on a seed. In my example, the randomness should stay the same per IP.

+1 on this, as it would also solve the rather hairy problem of using SRANDMEMBER in Lua scripts.

I'm not seeing any good way of implementing what I imagine is not a completely unusual pattern of having two level queues.

Essentially, my set up is this. Say I have a set of tasks that customers can trigger to take place. I want to process these as quickly as possible, but I don't want a single customer to 'hog' all available resources, even if they started first.

One nice way to do this in Redis is as follows:

Have a set 'queues' that contains the names of all the active queues, each customer having one queue keyed off their customer id.

For each customer, have a sorted set of tasks.

To add a new job, first push into the sorted set for that customer, say 'jobs:1234', then add that queue name to our 'queues' set. (so sadd queues jobs:1234)

To work through the jobs 'fairly' you simply do:

  1. srandmember on the queues to get what queue to pop off of
  2. do a 'zpop' in lua
  3. if your zpop fails, then remove the queue from the 'queues' set

Ideally 1-3 are done in Lua to prevent any races, it should all be quick enough. Right now I can do 2 and 3 together in Lua and that works great:

local val = redis.call('zrange', ARGV[2], 0, 0)" 
if next(val) == nil then 
  redis.call('srem', ARGV[1], ARGV[2]) 
  return nil
else 
  redis.call('zremrangebyrank', ARGV[2], 0, 0) 
  return val[1] 
end

The problem is that right now 1 has to take place separately, introducing the possibility of a race. Sure you can recover and try again, but if these queues are typically empty and there is high concurrency then that might happen a lot.

Having a way to 'seed' the srandmember would fix this, as our Lua script would then be 'pure'. Sadly, I'm not seeing any clear way to simulate a srandmember in Lua either, so I'm kind of stuck with a suboptimal solution.

Hope that makes some kind of sense!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment