Permalink
Browse files

Adding the ability to trip the circuit breaker when a method takes to…

…o long to respond. This is very useful with web-based services where a timeout of less than the default http timeout is desired.

The default invocation_timeout is 30 seconds, which can be overridden along with the failure_threshold and failure_timeout properties.

Tests included, docs updated.
  • Loading branch information...
1 parent 1c44c2c commit c34d09714217aad6ef19c17fb329a9b113054f78 @briandoll committed Aug 6, 2009
View
@@ -24,6 +24,11 @@
immediately pop the circuit open again, and a success will close the
circuit and reset the failure count.
+ For services that can take an unmanagable amount of time to respond an
+ invocation timeout threshold is provided. If the service fails to return
+ before the invocation_timeout duration has passed, the circuit will "trip",
+ setting the circuit into an "open" state.
+
require 'circuit_breaker'
class TestService
@@ -38,6 +43,7 @@
handler.logger = Logger.new(STDOUT)
handler.failure_threshold = 5
handler.failure_timeout = 5
+ handler.invocation_timeout = 10
end
# Optional
@@ -17,6 +17,11 @@
# immediately pop the circuit open again, and a success will close the
# circuit and reset the failure count.
#
+# For services that can take an unmanagable amount of time to respond an
+# invocation timeout threshold is provided. If the service fails to return
+# before the invocation_timeout duration has passed, the circuit will "trip",
+# setting the circuit into an "open" state.
+#
# require 'circuit_breaker'
# class TestService
#
@@ -31,6 +36,7 @@
# handler.logger = Logger.new(STDOUT)
# handler.failure_threshold = 5
# handler.failure_timeout = 5
+# handler.invocation_timeout = 10
# end
#
# # Optional
@@ -1,6 +1,6 @@
class CircuitBreaker::CircuitBrokenException < StandardError
- def initialize(msg, circuit_state)
+ def initialize(msg, circuit_state = :closed)
@circuit_state = circuit_state
super(msg)
end
@@ -1,3 +1,5 @@
+require 'timeout'
+
#
#
# CircuitHandler is stateless,
@@ -17,17 +19,24 @@ class CircuitBreaker::CircuitHandler
attr_accessor :failure_timeout
#
+ # The period of time the circuit_method has to return before a timeout exception is thrown.
+ #
+ attr_accessor :invocation_timeout
+
+ #
# Optional logger.
#
attr_accessor :logger
- DEFAULT_FAILURE_THRESHOLD = 5
- DEFAULT_FAILURE_TIMEOUT = 5
+ DEFAULT_FAILURE_THRESHOLD = 5
+ DEFAULT_FAILURE_TIMEOUT = 5
+ DEFAULT_INVOCATION_TIMEOUT = 30
def initialize(logger = nil)
@logger = logger
@failure_threshold = DEFAULT_FAILURE_THRESHOLD
@failure_timeout = DEFAULT_FAILURE_TIMEOUT
+ @invocation_timeout = DEFAULT_INVOCATION_TIMEOUT
end
#
@@ -47,8 +56,11 @@ def handle(circuit_state, method, *args)
end
begin
- out = method[*args]
- on_success(circuit_state)
+ out = nil
+ Timeout.timeout(@invocation_timeout, CircuitBreaker::CircuitBrokenException) do
+ out = method[*args]
+ on_success(circuit_state)
+ end
rescue Exception
on_failure(circuit_state)
raise
@@ -28,13 +28,18 @@ def call_external_method()
"hello world!"
end
- def second_method()
- raise 'EPIC FAIL'
- end
+ def second_method()
+ raise 'EPIC FAIL'
+ end
+
+ def unresponsive_method
+ sleep 1.1
+ "unresponsive method returned"
+ end
# Register this method with the circuit breaker...
#
- circuit_method :call_external_method, :second_method
+ circuit_method :call_external_method, :second_method, :unresponsive_method
#
# Define what needs to be set for configuration...
@@ -43,6 +48,7 @@ def second_method()
handler.logger = Logger.new(STDOUT)
handler.failure_threshold = 5
handler.failure_timeout = 5
+ handler.invocation_timeout = 1
end
end
@@ -107,6 +113,11 @@ def second_method()
@test_object.circuit_state.open?.should == true
@test_object.circuit_state.failure_count.should == 1
end
+
+ it 'should trip the circuit when the method takes too long to return' do
+ lambda { @test_object.unresponsive_method }.should raise_error(CircuitBreaker::CircuitBrokenException)
+ @test_object.circuit_state.open?.should == true
+ end
end

0 comments on commit c34d097

Please sign in to comment.