Permalink
Browse files

Handle the node discovery error condition where the znode points to a…

… master that is now a slave.
  • Loading branch information...
1 parent 68b4d7d commit c98f07f95b428e9876695f9312c7741180c51a3a @ryanlecompte committed Sep 1, 2012
Showing with 20 additions and 1 deletion.
  1. +4 −0 Changes.md
  2. +16 −1 lib/redis_failover/node_manager.rb
View
4 Changes.md
@@ -1,3 +1,7 @@
+HEAD
+-----------
+- Handle the node discovery error condition where the znode points to a master that is now a slave.
+
0.9.5
-----------
- Introduce a safer master node discovery process for the Node Manager (#34)
View
17 lib/redis_failover/node_manager.rb
@@ -11,13 +11,22 @@ class NodeManager
# Number of seconds to wait before retrying bootstrap process.
TIMEOUT = 3
+
# ZK Errors that the Node Manager cares about.
ZK_ERRORS = [
ZK::Exceptions::LockAssertionFailedError,
ZK::Exceptions::InterruptedSession,
ZKDisconnectedError
].freeze
+ # Errors that can happen during the node discovery process.
+ NODE_DISCOVERY_ERRORS = [
+ InvalidNodeRoleError,
+ NodeUnavailableError,
+ NoMasterError,
+ MultipleMastersError
+ ].freeze
+
# Creates a new instance.
#
# @param [Hash] options the options used to initialize the manager
@@ -248,7 +257,7 @@ def discover_nodes
redirect_slaves_to(@master)
true
end
- rescue NodeUnavailableError, NoMasterError, MultipleMastersError => ex
+ rescue *NODE_DISCOVERY_ERRORS => ex
msg = <<-MSG.gsub(/\s+/, ' ')
Failed to discover master node: #{ex.inspect}
In order to ensure a safe startup, redis_failover requires that all redis
@@ -268,6 +277,12 @@ def find_existing_master
nodes = symbolize_keys(decode(data))
master = node_from(nodes[:master])
logger.info("Master from existing znode config: #{master || 'none'}")
+ # Check for case where a node previously thought to be the master was
+ # somehow manually reconfigured to be a slave outside of the node manager's
+ # control.
+ if master && master.slave?
+ raise InvalidNodeRoleError.new(node, :master, :slave)
+ end
master
end
rescue ZK::Exceptions::NoNode

0 comments on commit c98f07f

Please sign in to comment.