Add many_to_one_pk_lookup plugin, for using a simple primary key look…

…up for many_to_one associations (great with caching)

This can be a major speed boost to models that have many_to_one
associations to associated models that use caching, because it
will generally use a cached lookup first.

CHANGELOG

@@ -1,5 +1,7 @@
 === HEAD
 
+* Add many_to_one_pk_lookup plugin, for using a simple primary key lookup for many_to_one associations (great with caching) (jeremyevans)
+
 * Use bigint type instead of integer for Bignum generic type on SQLite (jeremyevans)
 
 * Add Database#dump_foreign_key_migration for just dumping foreign key constraints to the schema dumper extension (jeremyevans)

lib/sequel/plugins/many_to_one_pk_lookup.rb

@@ -0,0 +1,71 @@
+module Sequel
+  module Plugins
+    # This is a fairly simple plugin that modifies the internal association loading logic
+    # for many_to_one associations to use a simple primary key lookup on the associated
+    # class, which is generally faster as it uses mostly static SQL.  Additional, if the
+    # associated class is caching primary key lookups, you get the benefit of a cached
+    # lookup.
+    #
+    # This plugin is generally not as fast as the prepared_statements_associations plugin
+    # in the case where the model is not caching primary key lookups, however, it is
+    # probably significantly faster if the model is caching primary key lookups.  If
+    # the prepared_statements_associations plugin has been loaded first, this
+    # plugin will only use the primary key lookup code if the associated model is
+    # caching primary key lookups.
+    #
+    # This plugin attempts to determine cases where the primary key lookup would have
+    # different results than the regular lookup, and use the regular lookup in that case,
+    # but it cannot handle all situations correctly, which is why it is not Sequel's
+    # default behavior.
+    #
+    # You can disable primary key lookups on a per association basis with this
+    # plugin using the :many_to_one_pk_lookup=>false association option.
+    # 
+    # Usage:
+    #
+    #   # Make all model subclass instances use primary key lookups for many_to_one
+    #   # association loading
+    #   Sequel::Model.plugin :many_to_one_pk_lookup
+    #
+    #   # Do so for just the album class.
+    #   Album.plugin :many_to_one_pk_lookup
+    module ManyToOnePkLookup
+      module InstanceMethods
+        private
+
+        # If the current association is a fairly simple many_to_one association, use
+        # a simple primary key lookup on the associated model, which can benefit from
+        # caching if the associated model is using caching.
+        def _load_associated_object(opts, dynamic_opts)
+          klass = opts.associated_class
+          cache_lookup = opts.fetch(:many_to_one_pk_lookup) do 
+            opts[:many_to_one_pk_lookup] = opts[:type] == :many_to_one &&
+              opts[:key] &&
+              opts.primary_key == klass.primary_key
+          end
+          if cache_lookup &&
+            !dynamic_opts[:callback] &&
+            (o = klass.send(:primary_key_lookup, ((fk = opts[:key]).is_a?(Array) ? fk.map{|c| send(c)} : send(fk))))
+            o
+          else
+            super
+          end
+        end
+
+        # Deal with the situation where the prepared_statements_associations plugin is
+        # loaded first, by using a primary key lookup for many_to_one associations if
+        # the associated class is using caching, and using the default code otherwise.
+        # This is done because the prepared_statements_associations code is probably faster
+        # than the primary key lookup this plugin uses if the model is not caching lookups,
+        # but probably slower if the model is caching lookups.
+        def _load_associated_objects(opts, dynamic_opts={})
+          if opts.can_have_associated_objects?(self) && opts[:type] == :many_to_one && opts.associated_class.respond_to?(:cache_get_pk)
+            _load_associated_object(opts, dynamic_opts)
+          else
+            super
+          end
+        end
+      end
+    end
+  end
+end

spec/extensions/many_to_one_pk_lookup_spec.rb

@@ -0,0 +1,102 @@
+require File.join(File.dirname(File.expand_path(__FILE__)), "spec_helper")
+
+describe "Sequel::Plugins::ManyToOnePkLookup" do
+  before do
+    @cache_class = Class.new(Hash) do
+      attr_accessor :ttl
+      def set(k, v, ttl); self[k] = v; @ttl = ttl; end
+      def get(k); self[k]; end
+    end
+    cache = @cache_class.new
+    @cache = cache
+
+    class ::CachingModel < Sequel::Model
+      columns :id, :id2
+    end
+    @cc = CachingModel
+    @cc.plugin :caching, @cache
+    @cc.dataset._fetch = {:id=>1}
+    @cm1 = @cc[1]
+    @cm2 = @cc[2]
+    @cm12 = @cc[1, 2]
+    @cm21 = @cc[2, 1]
+
+    class ::LookupModel < ::Sequel::Model
+      plugin :many_to_one_pk_lookup
+      columns :id, :caching_model_id, :caching_model_id2
+      many_to_one :caching_model
+      many_to_one :caching_model2, :key=>[:caching_model_id, :caching_model_id2], :class=>:CachingModel
+    end
+    @c = LookupModel
+
+    @db = MODEL_DB
+    @db.reset
+  end
+  after do
+    Object.send(:remove_const, :CachingModel)
+    Object.send(:remove_const, :LookupModel)
+  end
+
+  it "should use a simple primary key lookup when retrieving many_to_one associated records via a composite key" do
+    @cc.set_primary_key([:id, :id2])
+    @db.sqls.should == []
+    @c.load(:id=>3, :caching_model_id=>1).caching_model.should equal(@cm1)
+    @c.load(:id=>4, :caching_model_id=>2).caching_model.should equal(@cm2)
+    @db.sqls.should == []
+    @c.load(:id=>4, :caching_model_id=>3).caching_model
+    @db.sqls.should_not == []
+  end
+
+  it "should use a simple primary key lookup when retrieving many_to_one associated records" do
+    @db.sqls.should == []
+    @c.load(:id=>3, :caching_model_id=>1, :caching_model_id2=>2).caching_model2.should equal(@cm12)
+    @c.load(:id=>3, :caching_model_id=>2, :caching_model_id2=>1).caching_model2.should equal(@cm21)
+    @db.sqls.should == []
+    @c.load(:id=>4, :caching_model_id=>2, :caching_model_id2=>2).caching_model2
+    @db.sqls.should_not == []
+  end
+
+  it "should not use a simple primary key lookup if the assocation has a nil :key option" do
+    @c.many_to_one :caching_model, :key=>nil, :dataset=>proc{CachingModel.filter(:caching_model_id=>caching_model_id)}
+    @c.load(:id=>3, :caching_model_id=>1).caching_model
+    @db.sqls.should_not == []
+  end
+
+  it "should not use a simple primary key lookup if the assocation has a nil :key option" do
+    @c.many_to_one :caching_model, :many_to_one_pk_lookup=>false
+    @c.load(:id=>3, :caching_model_id=>1).caching_model
+    @db.sqls.should_not == []
+  end
+
+  it "should not use a simple primary key lookup if the assocation's :primary_key option doesn't match the primary key of the associated class" do
+    @c.many_to_one :caching_model, :primary_key=>:id2
+    @c.load(:id=>3, :caching_model_id=>1).caching_model
+    @db.sqls.should_not == []
+  end
+
+  it "should not use a simple primary key lookup if the prepared_statements_associations method is being used" do
+    c2 = Class.new(Sequel::Model(:not_caching_model))
+    c2.dataset._fetch = {:id=>1}
+    c = Class.new(Sequel::Model(:lookup_model))
+    c.class_eval do
+      plugin :prepared_statements_associations
+      plugin :many_to_one_pk_lookup
+      columns :id, :caching_model_id
+      many_to_one :caching_model, :class=>c2
+    end
+    c.load(:id=>3, :caching_model_id=>1).caching_model.should == c2.load(:id=>1)
+    @db.sqls.should_not == []
+  end
+
+  it "should use a simple primary key lookup if the prepared_statements_associations method is being used but associated model also uses caching" do
+    c = Class.new(Sequel::Model(:lookup_model))
+    c.class_eval do
+      plugin :prepared_statements_associations
+      plugin :many_to_one_pk_lookup
+      columns :id, :caching_model_id
+      many_to_one :caching_model
+    end
+    c.load(:id=>3, :caching_model_id=>1).caching_model.should equal(@cm1)
+    @db.sqls.should == []
+  end
+end

spec/integration/plugin_test.rb

@@ -1554,3 +1554,45 @@ class ::Lorem < Sequel::Model
     @c[o.id].should == @c.load(:id=>o.id, :name=>nil, :i=>40)
   end
 end
+
+describe "Caching plugins" do
+  before(:all) do
+    @db = INTEGRATION_DB
+    @db.drop_table?(:albums, :artists)
+    @db.create_table(:artists) do
+      primary_key :id
+    end
+    @db.create_table(:albums) do
+      primary_key :id
+      foreign_key :artist_id, :artists
+    end
+    @db[:artists].insert
+    @db[:albums].insert(:artist_id=>1)
+    @cache_class = Class.new(Hash) do
+      def set(k, v, ttl) self[k] = v end
+      alias get []
+    end
+    @cache = @cache_class.new
+
+    @Artist = Class.new(Sequel::Model(@db[:artists]))
+    @Album = Class.new(Sequel::Model(@db[:albums]))
+    @Artist.plugin :caching, @cache
+    @Album.plugin :many_to_one_pk_lookup
+    @Album.many_to_one :artist, :class=>@Artist
+  end
+  after(:all) do
+    @db.drop_table?(:albums, :artists)
+  end
+
+  it "should work with looking up using Model.[]" do 
+    @Artist[1].should equal(@Artist[1])
+    @Artist[:id=>1].should == @Artist[1]
+    @Artist[0].should == nil
+    @Artist[nil].should == nil
+  end
+
+  it "should work with lookup up many_to_one associated objects" do 
+    a = @Artist[1]
+    @Album.first.artist.should equal(a)
+  end
+end

www/pages/plugins

@@ -25,6 +25,7 @@
 <li><a href="rdoc-plugins/classes/Sequel/Plugins/LazyAttributes.html">lazy_attributes</a>: Allows you to set some attributes that should not be loaded by default, but only loaded when an object requests them.</li>
 <li><a href="rdoc-plugins/classes/Sequel/Plugins/List.html">list</a>: Allows you to treat model objects as being part of a list, so you can move them up/down and get next/previous entries.</li>
 <li><a href="rdoc-plugins/classes/Sequel/Plugins/ManyThroughMany.html">many_through_many</a>: Allows you to create an association to multiple objects through multiple join tables.</li>
+<li><a href="rdoc-plugins/classes/Sequel/Plugins/ManyToOnePkLookup.html">many_to_one_pk_lookup</a>: Uses optimized simple primary key lookups for most many_to_one associations (great if associated class uses caching).</li>
 <li><a href="rdoc-plugins/classes/Sequel/Plugins/NestedAttributes.html">nested_attributes</a>: Allows you to modified associated objects directly through a model object, similar to ActiveRecord's Nested Attributes.</li>
 <li><a href="rdoc-plugins/classes/Sequel/Plugins/OptimisticLocking.html">optimistic_locking</a>: Adds a database-independent locking mechanism to models to prevent concurrent updates overwriting changes.</li>
 <li><a href="rdoc-plugins/classes/Sequel/Plugins/PreparedStatements.html">prepared_statements</a>: Makes models use prepared statements for deletes, inserts, updates, and lookups by primary key.</li>