Skip to content

Commit

Permalink
Optimize zero-argument methods
Browse files Browse the repository at this point in the history 
This commit adds a performance optimization to zero-argument
methods; these methods now have their memoized results stored
in an array rather than a hash. This is the same as the recent
optimization we made for one-argument methods, except because
we store zero-argument results directly (rather than a hash
containing results as with one-argument methods), we need a
sentinel system to differentiate between an unmemoized method
(in which case the lookup will return `nil`) and a memoized
result that also happens to be `nil`.

This optimization offers a slight speedup because array
accesses are faster than hash lookups. In addition, this
optimization is faster even for the "slow path" case in which
the memoized value is falsey, because previously that check
was also performing a hash lookup and our new sentinel system
uses an array for this as well.
  • Loading branch information
@JacobEvelyn
JacobEvelyn committed Sep 24, 2021
1 parent 422de9e commit d768e10
Show file tree
Hide file tree
Showing 4 changed files with 117 additions and 75 deletions.
2 changes: 2 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## Unreleased

### Updated
- Improve performance of zero-argument methods by using an outer Array instead
of a Hash
- Improve performance of single-argument methods by using an outer Array instead
of a Hash
### Fixed
Expand Down
36 changes: 18 additions & 18 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -108,29 +108,29 @@ Results using Ruby 3.0.2:

|Method arguments|`Dry::Core` (0.7.1)|`Memery` (1.4.0)|
|--|--|--|
|`()` (none)|1.22x|17.10x|
|`(a)`|2.05x|10.08x|
|`(a, b)`|1.00x|4.59x|
|`(a:)`|1.92x|20.29x|
|`(a:, b:)`|1.07x|10.56x|
|`(a, b:)`|1.02x|9.26x|
|`(a, *args)`|1.95x|4.53x|
|`(a:, **kwargs)`|1.54x|6.12x|
|`(a, *args, b:, **kwargs)`|1.04x|2.62x|
|`()` (none)|1.49x|21.54x|
|`(a)`|2.07x|9.56x|
|`(a, b)`|1.01x|4.59x|
|`(a:)`|1.87x|18.45x|
|`(a:, b:)`|1.02x|8.92x|
|`(a, b:)`|1.02x|8.97x|
|`(a, *args)`|1.98x|4.25x|
|`(a:, **kwargs)`|1.46x|5.50x|
|`(a, *args, b:, **kwargs)`|1.05x|2.67x|

Results using Ruby 2.7.4 (because these gems raise errors in Ruby 3.x):

|Method arguments|`DDMemoize` (1.0.0)|`Memoist` (0.16.2)|`Memoized` (1.0.2)|`Memoizer` (1.0.3)|
|--|--|--|--|--|
|`()` (none)|26.89x|2.81x|1.21x|3.29x|
|`(a)`|21.26x|14.65x|11.53x|12.74x|
|`(a, b)`|3.98x|2.94x|2.31x|2.61x|
|`(a:)`|29.65x|24.27x|20.54x|21.69x|
|`(a:, b:)`|6.43x|5.45x|4.72x|5.01x|
|`(a, b:)`|6.25x|5.08x|4.46x|4.68x|
|`(a, *args)`|5.20x|3.72x|3.31x|3.36x|
|`(a:, **kwargs)`|3.93x|3.30x|2.84x|3.00x|
|`(a, *args, b:, **kwargs)`|2.79x|2.42x|2.17x|2.22x|
|`()` (none)|38.84x|3.65x|1.73x|4.66x|
|`(a)`|21.59x|14.87x|11.23x|12.67x|
|`(a, b)`|4.10x|2.98x|2.41x|2.58x|
|`(a:)`|30.91x|25.18x|21.22x|22.85x|
|`(a:, b:)`|6.72x|5.55x|4.88x|5.14x|
|`(a, b:)`|6.29x|5.24x|4.50x|4.75x|
|`(a, *args)`|5.50x|4.05x|3.42x|3.46x|
|`(a:, **kwargs)`|3.90x|3.25x|2.84x|2.99x|
|`(a, *args, b:, **kwargs)`|2.92x|2.51x|2.31x|2.33x|

You can run benchmarks yourself with:

Expand Down
99 changes: 57 additions & 42 deletions lib/memo_wise.rb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -166,21 +166,10 @@ def klass.extended(base)
klass.send(:alias_method, original_memo_wised_name, method_name)
klass.send(:private, original_memo_wised_name)

case MemoWise::InternalAPI.method_arguments(method)
when MemoWise::InternalAPI::NONE
# Zero-arg methods can use simpler/more performant logic because the
# hash key is just the method name.
klass.module_eval <<-END_OF_METHOD, __FILE__, __LINE__ + 1
def #{method_name}
output = @_memo_wise[:#{method_name}]
if output || @_memo_wise.key?(:#{method_name})
output
else
@_memo_wise[:#{method_name}] = #{original_memo_wised_name}
end
end
END_OF_METHOD
when MemoWise::InternalAPI::ONE_REQUIRED_POSITIONAL, MemoWise::InternalAPI::ONE_REQUIRED_KEYWORD
method_arguments = MemoWise::InternalAPI.method_arguments(method)
if method_arguments == MemoWise::InternalAPI::NONE ||
method_arguments == MemoWise::InternalAPI::ONE_REQUIRED_POSITIONAL ||
method_arguments == MemoWise::InternalAPI::ONE_REQUIRED_KEYWORD
# `@_memo_wise_indices` stores the `@_memo_wise_single_argument`
# indices of different method names. We only use this data structure
# when resetting or presetting memoization. It looks like:
Expand All @@ -195,12 +184,29 @@ def #{method_name}
index = memo_wise_index_counter
memo_wise_indices[method_name] = memo_wise_index_counter
klass.instance_variable_set(:@_memo_wise_index_counter, memo_wise_index_counter + 1)
end

case method_arguments
when MemoWise::InternalAPI::NONE
# Zero-arg methods can use simpler/more performant logic because the
# hash key is just the method name.
klass.module_eval <<-END_OF_METHOD, __FILE__, __LINE__ + 1
def #{method_name}
output = @_memo_wise[#{index}]
if output || @_memo_wise_sentinels[#{index}]
output
else
@_memo_wise_sentinels[#{index}] = true
@_memo_wise[#{index}] = #{original_memo_wised_name}
end
end
END_OF_METHOD
when MemoWise::InternalAPI::ONE_REQUIRED_POSITIONAL, MemoWise::InternalAPI::ONE_REQUIRED_KEYWORD
key = method.parameters.first.last

klass.module_eval <<-END_OF_METHOD, __FILE__, __LINE__ + 1
def #{method_name}(#{MemoWise::InternalAPI.args_str(method)})
hash = (@_memo_wise_single_argument[#{index}] ||= {})
hash = (@_memo_wise[#{index}] ||= {})
output = hash[#{key}]
if output || hash.key?(#{key})
output
Expand All @@ -213,13 +219,13 @@ def #{method_name}(#{MemoWise::InternalAPI.args_str(method)})
klass.module_eval <<-END_OF_METHOD, __FILE__, __LINE__ + 1
def #{method_name}(#{MemoWise::InternalAPI.args_str(method)})
key = #{MemoWise::InternalAPI.key_str(method)}
output = @_memo_wise[key]
if output || @_memo_wise.key?(key)
output = @_memo_wise_multi_argument[key]
if output || @_memo_wise_multi_argument.key?(key)
output
else
hashes = (@_memo_wise_hashes[:#{method_name}] ||= Set.new)
hashes << key
@_memo_wise[key] = #{original_memo_wised_name}(#{MemoWise::InternalAPI.call_str(method)})
@_memo_wise_multi_argument[key] = #{original_memo_wised_name}(#{MemoWise::InternalAPI.call_str(method)})
end
end
END_OF_METHOD
Expand All @@ -228,7 +234,7 @@ def #{method_name}(#{MemoWise::InternalAPI.args_str(method)})

klass.module_eval <<-END_OF_METHOD, __FILE__, __LINE__ + 1
def #{method_name}(#{args_str})
hash = (@_memo_wise[:#{method_name}] ||= {})
hash = (@_memo_wise_multi_argument[:#{method_name}] ||= {})
key = #{MemoWise::InternalAPI.key_str(method)}
output = hash[key]
if output || hash.key?(key)
Expand Down Expand Up @@ -449,32 +455,36 @@ def preset_memo_wise(method_name, *args, **kwargs)
method_arguments = MemoWise::InternalAPI.method_arguments(method)

case method_arguments
when MemoWise::InternalAPI::NONE then @_memo_wise[method_name] = yield
when MemoWise::InternalAPI::NONE
index = api.index(method_name)

@_memo_wise_sentinels[index] = true
@_memo_wise[index] = yield
when MemoWise::InternalAPI::ONE_REQUIRED_POSITIONAL
hash = (@_memo_wise_single_argument[api.index(method_name)] ||= {})
hash = (@_memo_wise[api.index(method_name)] ||= {})
hash[args.first] = yield
when MemoWise::InternalAPI::ONE_REQUIRED_KEYWORD
hash = (@_memo_wise_single_argument[api.index(method_name)] ||= {})
hash = (@_memo_wise[api.index(method_name)] ||= {})
hash[kwargs.first.last] = yield
when MemoWise::InternalAPI::SPLAT
hash = (@_memo_wise[method_name] ||= {})
hash = (@_memo_wise_multi_argument[method_name] ||= {})
hash[args.hash] = yield
when MemoWise::InternalAPI::DOUBLE_SPLAT
hash = (@_memo_wise[method_name] ||= {})
hash = (@_memo_wise_multi_argument[method_name] ||= {})
hash[kwargs.hash] = yield
when MemoWise::InternalAPI::MULTIPLE_REQUIRED
key_args = method.parameters.map.with_index do |(type, name), index|
type == :req ? args[index] : kwargs[name]
key_args = method.parameters.map.with_index do |(type, name), idx|
type == :req ? args[idx] : kwargs[name]
end
key = [method_name, *key_args].hash
hashes = (@_memo_wise_hashes[method_name] ||= Set.new)
hashes << key
@_memo_wise[key] = yield
@_memo_wise_multi_argument[key] = yield
else # MemoWise::InternalAPI::SPLAT_AND_DOUBLE_SPLAT
key = [method_name, args, kwargs].hash
hashes = (@_memo_wise_hashes[method_name] ||= Set.new)
hashes << key
@_memo_wise[key] = yield
@_memo_wise_multi_argument[key] = yield
end
end

Expand Down Expand Up @@ -549,7 +559,8 @@ def reset_memo_wise(method_name = nil, *args, **kwargs)
raise ArgumentError, "Provided kwargs when method_name = nil" unless kwargs.empty?

@_memo_wise.clear
@_memo_wise_single_argument.clear
@_memo_wise_sentinels.clear
@_memo_wise_multi_argument.clear
@_memo_wise_hashes.clear
return
end
Expand All @@ -564,40 +575,44 @@ def reset_memo_wise(method_name = nil, *args, **kwargs)
method_arguments = MemoWise::InternalAPI.method_arguments(method)

case method_arguments
when MemoWise::InternalAPI::NONE then @_memo_wise.delete(method_name)
when MemoWise::InternalAPI::NONE
index = api.index(method_name)

@_memo_wise_sentinels[index] = nil
@_memo_wise[index] = nil
when MemoWise::InternalAPI::ONE_REQUIRED_POSITIONAL
index = api.index(method_name)

if args.empty?
@_memo_wise_single_argument[index]&.clear
@_memo_wise[index]&.clear
else
@_memo_wise_single_argument[index]&.delete(args.first)
@_memo_wise[index]&.delete(args.first)
end
when MemoWise::InternalAPI::ONE_REQUIRED_KEYWORD
index = api.index(method_name)

if kwargs.empty?
@_memo_wise_single_argument[index]&.clear
@_memo_wise[index]&.clear
else
@_memo_wise_single_argument[index]&.delete(kwargs.first.last)
@_memo_wise[index]&.delete(kwargs.first.last)
end
when MemoWise::InternalAPI::SPLAT
if args.empty?
@_memo_wise.delete(method_name)
@_memo_wise_multi_argument.delete(method_name)
else
@_memo_wise[method_name]&.delete(args.hash)
@_memo_wise_multi_argument[method_name]&.delete(args.hash)
end
when MemoWise::InternalAPI::DOUBLE_SPLAT
if kwargs.empty?
@_memo_wise.delete(method_name)
@_memo_wise_multi_argument.delete(method_name)
else
@_memo_wise[method_name]&.delete(kwargs.hash)
@_memo_wise_multi_argument[method_name]&.delete(kwargs.hash)
end
else # MemoWise::InternalAPI::MULTIPLE_REQUIRED, MemoWise::InternalAPI::SPLAT_AND_DOUBLE_SPLAT
if args.empty? && kwargs.empty?
@_memo_wise.delete(method_name)
@_memo_wise_multi_argument.delete(method_name)
@_memo_wise_hashes[method_name]&.each do |hash|
@_memo_wise.delete(hash)
@_memo_wise_multi_argument.delete(hash)
end
@_memo_wise_hashes.delete(method_name)
else
Expand All @@ -610,7 +625,7 @@ def reset_memo_wise(method_name = nil, *args, **kwargs)
key = [method_name, *key_args].hash
end
@_memo_wise_hashes[method_name]&.delete(key)
@_memo_wise.delete(key)
@_memo_wise_multi_argument.delete(key)
end
end
end
Expand Down
55 changes: 40 additions & 15 deletions lib/memo_wise/internal_api.rb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,37 +10,62 @@ class InternalAPI
#
# @return [Object] the passed-in obj
def self.create_memo_wise_state!(obj)
# `@_memo_wise` and `@_memo_wise_single_argument` store memoized results
# `@_memo_wise` and `@_memo_wise_multi_argument` store memoized results
# of method calls. For performance reasons, the structure differs for
# different types of methods.
#
# `@_memo_wise` looks like:
# `@_memo_wise_multi_argument` looks like:
# {
# no_args_method_name: :memoized_result,
# [:multi_arg_method_name, arg1, arg2].hash => :memoized_result
# }
#
# `@_memo_wise_single_argument` looks like:
# `@_memo_wise` looks like:
# [
# { arg1 => :memoized_result, ... }, # For method 0
# { arg1 => :memoized_result, ... }, # For method 1
# :memoized_result, # For method 0 (which takes no arguments)
# { arg1 => :memoized_result, ... }, # For method 1 (which takes an argument)
# { arg1 => :memoized_result, ... }, # For method 2 (which takes an argument)
# ]
# This is a faster alternative to:
# {
# zero_arg_method_name: :memoized_result,
# single_arg_method_name: { arg1 => :memoized_result, ... }
# }
# because we can give each single-argument method its own array index at
# load time and perform that array lookup more quickly than a hash lookup
# by method name.
obj.instance_variable_set(:@_memo_wise, {}) unless obj.instance_variables.include?(:@_memo_wise)
unless obj.instance_variables.include?(:@_memo_wise_single_argument)
obj.instance_variable_set(:@_memo_wise_single_argument, [])
# because we can give each method its own array index at load time and
# perform that array lookup more quickly than a hash lookup by method
# name.
obj.instance_variable_set(:@_memo_wise, []) unless obj.instance_variable_defined?(:@_memo_wise)
unless obj.instance_variable_defined?(:@_memo_wise_multi_argument)
obj.instance_variable_set(:@_memo_wise_multi_argument, {})
end

# For zero-arity methods, memoized values are stored in the `@_memo_wise`
# array. Arrays do not differentiate between "unset" and "set to nil" and
# so to handle this case we need another array to store sentinels and
# store `true` at indexes for which a zero-arity method has been memoized.
# `@_memo_wise_sentinels` looks like:
# [
# true, # A zero-arity method's result has been memoized
# nil, # A zero-arity method's result has not been memoized
# nil, # A one-arity method will always correspond to `nil` here
# ...
# ]
# NOTE: Because `@_memo_wise` stores memoized values for more than just
# zero-arity methods, the `@_memo_wise_sentinels` array can end up being
# sparse (see above), even when all methods' memoized values have been
# stored. If this becomes an issue we could store a separate index for
# zero-arity methods to make every element in `@_memo_wise_sentinels`
# correspond to a zero-arity method.
# NOTE: Surprisingly, lookups on an array of `true` and `nil` values
# appear to outperform even bitwise operators on integers (as of Ruby
# 3.0.2), allowing us to avoid more complex sentinel structures.
unless obj.instance_variable_defined?(:@_memo_wise_sentinels)
obj.instance_variable_set(:@_memo_wise_sentinels, [])
end

# `@_memo_wise_hashes` stores the `Array#hash` values for each key in
# `@_memo_wise` that represents a multi-argument method call. We only use
# this data structure when resetting memoization for an entire method. It
# looks like:
# `@_memo_wise_multi_argument`. We only use this data structure when
# resetting memoization for an entire method. It looks like:
# {
# multi_arg_method_name: Set[
# [:multi_arg_method_name, arg1, arg2].hash,
Expand All @@ -49,7 +74,7 @@ def self.create_memo_wise_state!(obj)
# ],
# ...
# }
obj.instance_variable_set(:@_memo_wise_hashes, {}) unless obj.instance_variables.include?(:@_memo_wise_hashes)
obj.instance_variable_set(:@_memo_wise_hashes, {}) unless obj.instance_variable_defined?(:@_memo_wise_hashes)

obj
end
Expand Down

0 comments on commit d768e10

Please sign in to comment.