Ent Leader Election
Clone this wiki locally
Sidekiq Enterprise will automatically elect a leader from the set of running processes, aka the "Senate". The leader is extremely useful when you want to coordinate work across the entire cluster without duplication of effort.
The Election Process
The first time a Sidekiq Enterprise process starts, it will check Redis for the current leader. If the leader's info is current, the process will automatically configure itself as a follower. If there's no leader information, the process will attempt a coup and elect itself leader. Viva la revolución!
There's no need for Raft, Paxos or other fancy algorithms because there's no distributed consensus going on here: Redis determines who is leader.
Leaders renew and verify their leadership every 20 seconds. Followers verify there is a leader every 60 seconds. Followers are prepared to take over as leader during that verification check. If the leader loses connectivity to Redis, its leadership will expire and a follower with connectivity will take over as leader.
Leader processes automatically "step down" when they exit, ensuring that another leader is elected promptly when deploying.
Running Code on the Leader
You can start your own actors within Sidekiq and use the Leader API to determine if your actor is running within the Leader process:
# Define an actor which does something within the Leader process only # every 30 seconds. class MyActor def initialize @done = false end def start @thread = Thread.new(&method(:process)) end def stop @done = true end private def process until @done if Sidekiq::Senate.leader? # I am the leader, do something! end sleep 30 end end end Sidekiq.configure_server do |config| config.on(:startup) do $my = MyActor.new $my.start end config.on(:quiet) do $my.stop end end