Permalink
Browse files

Reworked Functor to recursively redefine its method

Fixed minor problems for compatibility with Ruby 1.9
Also added a caching system that provides a substantial speed increase
  • Loading branch information...
automatthew committed Feb 6, 2009
1 parent 6c24eda commit cb65702d7fc0ef089b14f89933f06547e83a01ab
Showing with 259 additions and 186 deletions.
  1. +81 −0 README
  2. +0 −71 doc/README
  3. +81 −49 lib/functor.rb
  4. +1 −1 metrics/many_args.rb
  5. +23 −15 metrics/one_arg.rb
  6. +0 −24 test.rb
  7. +2 −2 test/fib.rb
  8. +6 −4 test/functor.rb
  9. +2 −2 test/guards.rb
  10. +13 −8 test/inheritance.rb
  11. +4 −4 test/matchers.rb
  12. +3 −3 test/reopening.rb
  13. +20 −0 test/supplement.rb
  14. +15 −0 test/wildcard.rb
  15. +8 −3 test/with_self.rb
View
81 README
@@ -0,0 +1,81 @@
+Functor provides pattern-based function and method dispatch for Ruby, originally inspired by Topher Cyll's multi gem.
+
+= Method Functors
+
+To use it in a class:
+
+ class Repeater
+ attr_accessor :times
+ include Functor::Method
+ functor( :repeat, Integer ) { |x| x * @times }
+ functor( :repeat, String ) { |s| [].fill( s, 0, @times ).join(' ') }
+ end
+
+ r = Repeater.new
+ r.times = 5
+ r.repeat( 5 ) # => 25
+ r.repeat( "-" ) # => "- - - - -"
+ r.repeat( 7.3 ) # => Functor::NoMatch!
+
+= Stand-Alone Functors
+
+You can also define Functor objects directly:
+
+ fib ||= Functor.new do |f|
+ f.given( Integer ) { | n | f.call( n - 1 ) + f.call( n - 2 ) }
+ f.given( 0 ) { |x| 0 }
+ f.given( 1 ) { |x| 1 }
+ end
+
+You can use functors directly with functions taking a block like this:
+
+ [ *0..10 ].map( &fib ) # => [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55]
+
+You can call a functor as a method using #call:
+
+ fun.call( obj, 7 )
+
+= Pattern Matching
+
+Arguments are matched first using #===, so anything that supports these methods can be matched against. In addition, you may pass as a "guard" any object that responds to #call and which takes an object (the argument) and return true or false. This allows you to do things like this:
+
+ stripe ||= Functor.new do
+ given( lambda { |x| x % 2 == 0 } ) { |x| 'white' }
+ given( lambda { |x| x % 2 == 1 } ) { |x| 'silver' }
+ end
+
+ stripe.call( 3 ) # => 'silver'
+ stripe.call( 4 ) # => 'white'
+
+= Precedence
+
+Precedence works similarly to Ruby method definition, i.e. Last In, First Out. Thus, you need to be careful in how you define your functor. The Fibonacci example above would not work properly if the Integer pattern was given last.
+
+= Caching Options
+
+Methods defined with functors are substantially slower than methods defined natively with "def". To (partially) alleviate the performance hit, Functor keeps track of which functor block matches each particular *args set, allowing it to short circuit the matching process when an *args set is seen again. If the *args cache were unlimited, this would represent a serious memory leak. Naturally, we have not implemented an unlimited cache.
+
+Rather, the *args cache is subjected to flexible and scalable limits that also have the effect of promoting frequently encountered *args sets and allowing those less frequently encountered to languish, yea even unto death. The caching system uses two configuration parameters, which may be set for Functor or for any class which has included Functor::Method. When a class does not set these parameters, it will inherit the options set at the Functor level. Functor comes with (what we think are) sensible defaults.
+
+The :size parameter controls, albeit indirectly, the number of items the caching system may store. If your :size is greater than the number of *args sets the functor will encounter, you do not need to worry (or even know) about the other parameter. You can set the :size param thusly:
+
+ Functor.cache_config :size => 10_000
+
+ class A
+ include Functor::Method
+ functor_cache_config :size => 700
+ end
+
+The other parameter, :base, determines the thresholds for promotion, i.e. how many times an *args set must be seen before it "becomes more important". While the use of the :size parameter is somewhat straightforward (albeit indirect), you need some idea of how the caching system works to understand the :base param. Functor classes maintain cached *args sets in tiers. When attempting to short-circuit the matching process, Functor checks each of these caches, starting at the top and working downward. Each tier has a promotion threshold, which represents the number of hits an item must receive before it can jump to the next tier. The promotion thresholds are determined using exponents of the :base parameter. The threshold for exiting the lowest tier is (base ** 1); for the next tier, it is (base ** 2); for the next, (base ** 3).
+
+Assume the following configuration:
+
+ Functor.cache_config :base => 10
+
+Functor currently uses 4 tiers. Let us call them c0, c1, c2, and c3. To pass from c0 to c1, an item must receive 10 (i.e. 10**1) hits. To pass from c1 to c2, it must receive 100 (10**2) hits. For promotion to c3, it must receive 1,000 hits. A lower :base setting results in lower thresholds across all tiers. The thresholds for a base of 8 would be 8, 64, and 512. Again, if your cache :size is large enough, the promotion thresholds are irrelevant. If, however, the number of distinct *args sets you expect to encounter is larger than your desired cache size, the :base parameter allows you to tune the promotion thresholds for a better fit with the frequency distribution of the *args sets. A high :base setting means that only very high-frequency items will make it into the top cache tier.
+
+Tier size limits are enforced in a useful way. Each tier has a size limit. When that size is reached, all of its items are dropped into the next lower tier, which drops all of its items into the next lower tier, etc. Tier size limits increase from top to bottom, so that when c3 dumps its items into c2, ample space remains in c2. After a tier cascade, all of the items in a particular tier have a hit count higher than the promotion threshold, guaranteeing that the next hit will promote the item. The effect of this is to allow "languishing" items at a certain level to eventually drop out the bottom, while preserving the valuable, active, high-frequency items.
+
+= Credits And Support
+
+Functor was written by Dan Yoder, Matthew King, and Lawrence Pit. Send email to dan at zeraweb.com for support or questions.
View
@@ -1,71 +0,0 @@
-Functor provides pattern-based function and method dispatch for Ruby, originally inspired by Topher Cyll's multi gem.
-
-= Method Functors
-
-To use it in a class:
-
- class Repeater
- attr_accessor :times
- include Functor::Method
- functor( :repeat, Integer ) { |x| x * @times }
- functor( :repeat, String ) { |s| [].fill( s, 0..@times ).join(' ') }
- end
-
- r = Repeater.new
- r.times = 5
- r.repeat( 5 ) # => 25
- r.repeat( "-" ) # => "- - - - -"
- r.repeat( 7.3 ) # => ArgumentError!
-
-Warning: This defines a class instance variable <tt>@__functors</tt> behind the scenes as a side-effect. Also, although inheritance works within a functor method, super does not. To call the parent method, you need to call it explicitly using the <tt>#functors</tt> class method, like this:
-
- A.functors[ :foo ].apply( self, 'bar' )
-
-= Stand-Alone Functors
-
-You can also define Functor objects directly:
-
- fib = Functor.new do
- given( 0 ) { 0 }
- given( 1 ) { 1 }
- given( Integer ) { |n| self.call( n - 1 ) + self.call( n - 2 ) }
- end
-
-You can use functors directly with functions taking a block like this:
-
- [ *0..10 ].map( &fib ) # => [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55]
-
-You can call a functor as a method using #apply:
-
- fun.apply( obj, 7 )
-
-which is actually how the method functors are implemented.
-
-= Pattern Matching
-
-Arguments are matched first using === and then ==, so anything that supports these methods can be matched against. In addition, you may pass "guards," any object that responds to #call and which take and object (the argument) and return true or false. This allows you to do things like this:
-
- stripe ||= Functor.new do
- given( lambda { |x| x % 2 == 0 } ) { 'white' }
- given( lambda { |x| x % 2 == 1 } ) { 'silver' }
- end
-
-which will return "white" and "silver" alternately for a sequence of numbers.
-
-= Precedence
-
-Precedence is defined in order of declaration: first-come, first-serve, aka FIFO. Thus, you need to be careful in how you define your functor. The Fibonacci example above would not work properly if the Integer pattern was given first. That said, it is possible to redefine earlier cases, which, in effect, "demotes" it, as if it had not been declared before. So the following will work properly:
-
- fib = Functor.new do
- given( Integer ) { |n| raise "this would start an infinite loop ..." }
- given( 0 ) { 0 }
- given( 1 ) { 1 }
- # but this will "demote" the Integer pattern and now it will work ...
- given( Integer ) { |n| self.call( n - 1 ) + self.call( n - 2 ) }
- end
-
-This isn't perfect, but it is very easy to predict, simple to implement, and reasonably fast, which other approaches (such as implementing a precedence scheme) are not.
-
-= Credits And Support
-
-Functor was written by Dan Yoder, Matthew King, and Lawrence Pit. Send email to dan at zeraweb.com for support or questions.
View
@@ -2,71 +2,103 @@
class Functor
+ class NoMatch < ArgumentError; end
+
+ def self.cache_config(options={})
+ (@cache_config ||= { :size => 4_096, :base => 8 }).merge!(options)
+ end
+
module Method
- def self.copy_functors( functors )
- r = {} ; functors.each do | name, functor |
- r[ name ] = functor.clone
- end
- return r
- end
+
def self.included( k )
- def k.functors
- @__functors ||= superclass.respond_to?( :functors ) ?
- Functor::Method.copy_functors( superclass.functors ) : {}
+
+ def k.functor_cache
+ @functor_cache ||= Hash.new { |hash, key| hash[key] = [ {},{},{},{} ] }
+ end
+
+ def k.functor_cache_config(options={})
+ @functor_cache_config = ( @functor_cache_config || Functor.cache_config ).merge(options)
+ end
+
+ def k.functor( name, *pattern, &action )
+ _functor( name, false, *pattern, &action)
end
- def k.functor( name, *args, &block )
- name = name.to_sym
- ( f = ( functors[ name ] or
- ( functors[ name ] = Functor.new ) ) ).given( *args, &block )
- define_method( name ) { | *args | instance_exec( *args, &f.match( *args ) ) }
+
+ def k.functor_with_self( name, *pattern, &action )
+ _functor( name, true, *pattern, &action)
end
- def k.functor_with_self( name, *args, &block )
- name = name.to_sym
- ( f = ( functors[ name ] or
- ( functors[ name ] = Functor.new ) ) ).given( *args, &block )
- define_method( name ) { | *args | instance_exec( *args, &f.match( self, *args ) ) }
+
+ # undefined methods beginning with '_' can be used as wildcards in Functor patterns
+ def k.method_missing(name, *args)
+ args.empty? && name.to_s =~ /^_/ ? lambda { |args| true } : super
end
+
+ private
+
+ def k._functor( name, with_self=false, *pattern, &action)
+ name = name.to_s
+ mc = functor_cache[name] # grab the cache tiers for The Method
+ cache_size, cache_base = functor_cache_config[:size], functor_cache_config[:base]
+ c0_size, c1_size, c2_size, c3_size = cache_size * 4, cache_size * 3, cache_size * 2, cache_size
+ c1_thresh,c2_thresh,c3_thresh = cache_base.to_i, (cache_base ** 2).to_i, (cache_base ** 3).to_i
+ old_method = instance_method(name) if method_defined?( name ) # grab The Method's current incarnation
+ define_method( name, action ) # redefine The Method
+ newest = instance_method(name) # grab newly redefined The Method
+
+ # Recursively redefine The Method using the newest and previous incarnations
+ define_method( name ) do | *args |
+ match_args = with_self ? [self] + args : args
+ sig = match_args.hash
+ if meth = mc[3][sig] # check caches from top down
+ meth[0].bind(self).call(*args)
+ elsif meth = mc[2][sig]
+ meth[1] += 1 # increment hit count
+ mc[3][sig] = mc[2].delete(sig) if meth[1] > c3_thresh # promote sig if it has enough hits
+ (mc[0], mc[1], mc[2], mc[3] = mc[1], mc[2], mc[3], {}) if mc[3].size >= c3_size # cascade if c3 is full
+ meth[0].bind(self).call(*args)
+ elsif meth = mc[1][sig]
+ meth[1] += 1
+ mc[2][sig] = mc[1].delete(sig) if meth[1] > c2_thresh
+ mc[0], mc[1], mc[2] = mc[1], mc[2], {} if mc[2].size >= c2_size
+ meth[0].bind(self).call(*args)
+ elsif meth = mc[0][sig]
+ meth[1] += 1
+ mc[1][sig] = mc[0].delete(sig) if meth[1] > c1_thresh
+ mc[0], mc[1] = mc[1], {} if mc[1].size >= c1_size
+ meth[0].bind(self).call(*args)
+ elsif Functor.match?(match_args, pattern) # not cached? Try newest meth/pat.
+ (mc[0], mc[1], mc[2], mc[3] = mc[1], mc[2], mc[3], {}) if mc[3].size >= c3_size
+ mc[3][sig] = [newest, 0] # methods are cached as [ method, counter ]
+ newest.bind(self).call(*args)
+ elsif old_method # or call the previous incarnation of The Method
+ old_method.bind(self).call(*args)
+ else # and if there are no older incarnations, whine about it
+ raise NoMatch.new( "No functor matches the given arguments for method :#{name}." )
+ end
+ end
+ end
+
end
end
-
def initialize( &block )
- @rules = [] ; yield( self ) if block_given?
- end
-
- def initialize_copy( from )
- @rules = from.instance_eval { @rules.clone }
+ class << self; include Functor::Method; end
+ yield( self ) if block_given?
end
def given( *pattern, &action )
- @rules << [ pattern, action ]
+ (class << self; self; end)._functor( "call", false, *pattern, &action)
end
- def call( *args, &block )
- match( *args, &block ).call( *args )
- end
-
- def []( *args, &block )
- call( *args, &block )
- end
+ def []( *args, &block ); call( *args, &block ); end
- def to_proc ; lambda { |*args| self.call( *args ) } ; end
+ def to_proc ; lambda { |*args| call( *args ) } ; end
- def match( *args, &block )
- args << block if block_given?
- pattern, action = @rules.reverse.find { | p, a | match?( args, p ) }
- action or
- raise ArgumentError.new( "Argument error: no functor matches the given arguments." )
- end
-
- private
-
- def match?( args, pattern )
- args.zip( pattern ).all? { | arg, rule | pair?( arg, rule ) } if args.length == pattern.length
- end
-
- def pair?( arg, rule )
- ( rule.respond_to? :call and rule.call( arg ) ) or rule === arg
+ def self.match?( args, pattern )
+ args.all? do |arg|
+ pat = pattern[args.index(arg)]
+ pat === arg || ( pat.respond_to?(:call) && pat.call(arg))
+ end if args.length == pattern.length
end
end
View
@@ -68,4 +68,4 @@ class ManyArgs < Steve
end
end
-ManyArgs.compare_instances( 16, 32)
+ManyArgs.compare_instances( 4, 64)
View
@@ -2,6 +2,8 @@
class A
include Functor::Method
+ functor_cache_config :size => 700, :base => 6
+
functor( :foo, Integer ) { |x| :integer }
functor( :foo, String ) { |x| :string }
functor( :foo, Float ) { |x| :float }
@@ -24,30 +26,36 @@ def foo(x)
end
class OneArg < Steve
-
+ before do
+ nums = (1..200).to_a
+ alphas = ("a".."gr").to_a
+ @args_set = nums + nums.map { |i| i.to_f } + alphas + alphas.map { |i| i.to_sym } + Array.new(200, "one")
+ @args = []
+ srand(46)
+ 9000.times { @args << @args_set[rand(@args_set.size)] }
+ end
end
-OneArg.new "native method" do
- before do
- @n = Native.new
+OneArg.new "functor method" do
+ before_sample do
+ @a = A.new
end
measure do
- 200.times do
- [ 1, 2, 3, 1.0, 2.0, 3.0, "1", "2", "3", :uno, :dos, :tres, "one"].each { |item| @n.foo item }
- end
+ @args.each { |item| @a.foo item }
+ end
+
+ after_sample do
+ puts A.functor_cache["foo"].map{ |c| c.size }.inspect
end
end
-OneArg.new "functor method" do
- before do
- @a = A.new
+OneArg.new "native method" do
+ before_sample do
+ @n = Native.new
end
measure do
- 200.times do
- [ 1, 2, 1.0, 2.0, "1", "2", :uno, :dos, "one"].each { |item| @a.foo item }
- end
+ @args.each { |item| @n.foo item }
end
end
-
-OneArg.compare_instances( 16, 32)
+OneArg.compare_instances( 4, 96)
Oops, something went wrong.

0 comments on commit cb65702

Please sign in to comment.