Allow the :eager_loader association proc to accept a single hash inst…

…ead of 3 arguments

The previously supported API of using 3 separate arguments is still
supported.  However, the new API is more flexible, and is required
to get access to the dataset that is doing the eager loading.  That
will be used in the future to make sure the sharding plugin works
for eager loading.

With the new API, the hash passed to the eager_loader proc is
populated with the keys :key_hash, :rows, :associations, and
:self, with the first three keys corresponding to the old
argument order, and the new :self key being the dataset doing
the eager loading.

The Model.eager_loading_dataset method now also accepts an
options hash as an optional fifth argument, which should be
backwards compatible.

CHANGELOG

@@ -1,5 +1,7 @@
 === HEAD
 
+* Allow the :eager_loader association proc to accept a single hash instead of 3 arguments (jeremyevans)
+
 * Add a Dataset#order_append alias for order_more, for consistency with order_prepend (jeremyevans)
 
 * Add a Dataset#order_prepend method that adds to the end of an existing order (jeremyevans)

doc/active_record.rdoc

@@ -337,7 +337,7 @@ With either way of eager loading, you must call +all+ to retrieve all records at
 
 Like ActiveRecord, Sequel supports cascading of eager loading for both methods of eager loading.
 
-Unlike ActiveRecord, Sequel allows you to eager load custom associations using the <tt>:eager_loader</tt> and <tt>:eager_grapher</tt> association options.
+Unlike ActiveRecord, Sequel allows you to eager load custom associations using the <tt>:eager_loader</tt> and <tt>:eager_grapher</tt> association options.  See the {Advanced Associations guide}[link:files/doc/advanced_associations_rdoc.html] for more details.
 
 Table aliasing when eager loading via +eager_graph+ is different in Sequel than ActiveRecord.  Sequel will always attempt to use the association name, not the table name, for any associations.  If the association name has already been used, Sequel will append _N to it, where N starts at 0 and increases by 1.  For example, for a self referential association:
 

doc/advanced_associations.rdoc

@@ -77,9 +77,21 @@ option.  Though it can often be verbose (compared to other things in Sequel),
 it allows you complete control over how to eagerly load associations for a
 group of objects.
 
-:eager_loader should be a proc that takes 3 arguments, a key_hash,
-an array of records, and a hash of dependent associations.  Since you
-are given all of the records, you can do things like filter on
+:eager_loader should be a proc that takes 1 or 3 arguments.  If the proc
+takes one argument, it will be given a hash with the following keys:
+
+:key_hash :: A key_hash, described below
+:rows :: An array of model objects
+:associations :: A hash of dependent associations to eagerly load
+:self :: The dataset that is doing the eager loading
+
+If the proc takes three arguments, it gets passed the :key_hash, :rows,
+and :associations values.  The only way to get the :self value is to
+accept one argument.  The 3 argument procs are allowed for backwards
+compatibility, and it is recommended to use the 1 argument proc format
+for new code.
+
+Since you are given all of the records, you can do things like filter on
 associations that are specified by multiple keys, or do multiple
 queries depending on the content of the records (which would be
 necessary for polymorphic associations).  Inside the :eager_loader
@@ -277,9 +289,9 @@ Sequel::Model:
         klass = attachable_type.constantize
         klass.filter(klass.primary_key=>attachable_id)
       end), \
-      :eager_loader=>(proc do |key_hash, assets, associations|
+      :eager_loader=>(proc do |eo|
         id_map = {}
-        assets.each do |asset|
+        eo[:rows].each do |asset|
           asset.associations[:attachable] = nil 
           ((id_map[asset.attachable_type] ||= {})[asset.attachable_id] ||= []) << asset
         end
@@ -370,9 +382,9 @@ design, but sometimes you have to play with the cards you are dealt).
   class Client < Sequel::Model
     one_to_many :invoices, :reciprocal=>:client, \
       :dataset=>proc{Invoice.filter(:client_name=>name)}, \
-      :eager_loader=>(proc do |key_hash, clients, associations|
+      :eager_loader=>(proc do |eo|
         id_map = {}
-        clients.each do |client|
+        eo[:rows].each do |client|
           id_map[client.name] = client
           client.associations[:invoices] = []
         end
@@ -400,9 +412,9 @@ design, but sometimes you have to play with the cards you are dealt).
   class Invoice < Sequel::Model
     many_to_one :client, :key=>:client_name, \
       :dataset=>proc{Client.filter(:name=>client_name)}, \
-      :eager_loader=>(proc do |key_hash, invoices, associations|
-        id_map = key_hash[:client_name]
-        invoices.each{|inv| inv.associations[:client] = nil}
+      :eager_loader=>(proc do |eo|
+        id_map = eo[:key_hash][:client_name]
+        eo[:rows].each{|inv| inv.associations[:client] = nil}
         Client.filter(:name=>id_map.keys).all do |client|
           id_map[client.name].each{|inv| inv.associations[:client] = client}
         end
@@ -439,9 +451,9 @@ Here's the old way to do it via custom associations:
       :dataset=>(proc do
         FavoriteTrack.filter(:disc_number=>disc_number, :number=>number, :album_id=>album_id)
       end), \
-      :eager_loader=>(proc do |key_hash, tracks, associations|
+      :eager_loader=>(proc do |eo|
         id_map = {}
-        tracks.each do |t|
+        eo[:rows].each do |t|
           t.associations[:favorite_track] = nil
           id_map[[t[:album_id], t[:disc_number], t[:number]]] = t
         end
@@ -458,9 +470,9 @@ Here's the old way to do it via custom associations:
       :dataset=>(proc do
         Track.filter(:disc_number=>disc_number, :number=>number, :album_id=>album_id)
       end), \
-      :eager_loader=>(proc do |key_hash, ftracks, associations|
+      :eager_loader=>(proc do |eo|
         id_map = {}
-        ftracks.each{|ft| id_map[[ft[:album_id], ft[:disc_number], ft[:number]]] = ft}
+        eo[:rows].each{|ft| id_map[[ft[:album_id], ft[:disc_number], ft[:number]]] = ft}
         Track.filter([:album_id, :disc_number, :number]=>id_map.keys).all do |t|
           id_map[[t[:album_id], t[:disc_number], t[:number]]].associations[:track] = t
         end
@@ -490,10 +502,10 @@ without knowing the depth of the tree?
 
   class Node < Sequel::Model
     many_to_one :ancestors, :class=>self,
-     :eager_loader=>(proc do |key_hash, nodes, associations|
+     :eager_loader=>(proc do |eo|
       # Handle cases where the root node has the same parent_id as primary_key
       # and also when it is NULL
-      non_root_nodes = nodes.reject do |n| 
+      non_root_nodes = eo[:rows].reject do |n| 
         if [nil, n.pk].include?(n.parent_id)
           # Make sure root nodes have their parent association set to nil
           n.associations[:parent] = nil 
@@ -514,9 +526,9 @@ without knowing the depth of the tree?
         end 
       end 
     end)
-    many_to_one :descendants, :eager_loader=>(proc do |key_hash, nodes, associations|
+    many_to_one :descendants, :eager_loader=>(proc do |eo|
       id_map = {}
-      nodes.each do |n| 
+      eo[:rows].each do |n| 
         # Initialize an empty array of child associations for each parent node
         n.associations[:children] = []
         # Populate identity map of nodes
@@ -549,9 +561,9 @@ supports recursive common table expressions):
           Node.join(:t, :id=>:parent_id).
           select(:nodes.*))
       end),
-      :eager_loader=>(proc do |key_hash, nodes, associations|
-        id_map = key_hash[:id]
-        nodes.each{|n| n.associations[:descendants] = []}
+      :eager_loader=>(proc do |eo|
+        id_map = eo[:key_hash][:id]
+        eo[:rows].each{|n| n.associations[:descendants] = []}
         Node.from(:t).
         with_recursive(:t, Node.filter(:parent_id=>id_map.keys).
            select(:parent_id___root, :id, :parent_id),
@@ -565,8 +577,11 @@ supports recursive common table expressions):
       end)
   end
 
-You could modify the code to also store direct children relationships at the same time,
-for functionality similar to the non-common table expression case.
+Sequel ships with an +rcte_tree+ plugin that allows simple creation
+of ancestors and descendants relationships that use recursive common
+table expressions:
+
+  Node.plugin :rcte_tree
 
 === Joining multiple keys to a single key, through a third table
 
@@ -584,10 +599,10 @@ name, with no duplicates?
   class Artist < Sequel::Model
     one_to_many :songs, :order=>:songs__name, \
       :dataset=>proc{Song.select(:songs.*).join(Lyric, :id=>:lyric_id, id=>[:composer_id, :arranger_id, :vocalist_id, :lyricist_id])}, \
-      :eager_loader=>(proc do |key_hash, records, associations|
-        h = key_hash[:id]
+      :eager_loader=>(proc do |eo|
+        h = eo[:key_hash][:id]
         ids = h.keys
-        records.each{|r| r.associations[:songs] = []}
+        eo[:rows].each{|r| r.associations[:songs] = []}
         Song.select(:songs.*, :lyrics__composer_id, :lyrics__arranger_id, :lyrics__vocalist_id, :lyrics__lyricist_id)\
          .join(Lyric, :id=>:lyric_id){{:composer_id=>ids, :arranger_id=>ids, :vocalist_id=>ids, :lyricist_id=>ids}.sql_or}\
          .order(:songs__name).all do |song|
@@ -596,7 +611,7 @@ name, with no duplicates?
             recs.each{|r| r.associations[:songs] << song} if recs
           end
         end
-        records.each{|r| r.associations[:songs].uniq!}
+        eo[:rows].each{|r| r.associations[:songs].uniq!}
       end)
   end
 
@@ -614,13 +629,13 @@ associated tickets.
     one_to_many :tickets
     many_to_one :ticket_hours, :read_only=>true, :key=>:id,
      :dataset=>proc{Ticket.filter(:project_id=>id).select{sum(hours).as(hours)}},
-     :eager_loader=>(proc do |kh, projects, a|
-      projects.each{|p| p.associations[:ticket_hours] = nil}
-      Ticket.filter(:project_id=>kh[:id].keys).
+     :eager_loader=>(proc do |eo|
+      eo[:rows].each{|p| p.associations[:ticket_hours] = nil}
+      Ticket.filter(:project_id=>eo[:key_hash][:id].keys).
        group(:project_id).
        select{[project_id, sum(hours).as(hours)]}.
        all do |t|
-        p = kh[:id][t.values.delete(:project_id)].first
+        p = eo[:key_hash][:id][t.values.delete(:project_id)].first
         p.associations[:ticket_hours] = t
        end
      end)

doc/association_basics.rdoc

@@ -1093,16 +1093,9 @@ to eagerly load:
 
 ==== :eager_loader
 
-A custom loader to use when eagerly load associated objects via eager. If
-specified, should be a proc that takes three arguments: a key hash (used
-solely to enhance performance), an array of current model instances, and a
-hash of dependent associations to eagerly load.  The proc is responsible for
-querying the database to retrieve all associated records for any of the model
-instances (the second argument), and modifying the associations cache for each
-record to correctly set the associated records for that record.
-
+A custom loader to use when eagerly load associated objects via eager.
 For many details and examples of custom eager loaders, please see the
-{Advanced Associations page}[link:files/doc/advanced_associations_rdoc.html].
+{Advanced Associations guide}[link:files/doc/advanced_associations_rdoc.html].
 
 ==== :eager_loader_key
 

lib/sequel/model/associations.rb

@@ -445,8 +445,8 @@ module AssociationDatasetMethods
       #   => {:type => :many_to_one, :name => :portfolio, :class_name => "Portfolio"}
       #
       # For a more in depth general overview, as well as a reference guide,
-      # see the {Association Basics page}[link:files/doc/association_basics_rdoc.html].
-      # For examples of advanced usage, see the {Advanced Associations page}[link:files/doc/advanced_associations_rdoc.html].
+      # see the {Association Basics guide}[link:files/doc/association_basics_rdoc.html].
+      # For examples of advanced usage, see the {Advanced Associations guide}[link:files/doc/advanced_associations_rdoc.html].
       module ClassMethods
         # All association reflections defined for this model (default: none).
         attr_reader :association_reflections
@@ -525,9 +525,11 @@ def all_association_reflections
         #     Takes three arguments, a dataset, an alias to use for the table to graph for this association,
         #     and the alias that was used for the current table (since you can cascade associations),
         #     Should return a copy of the dataset with the association graphed into it.
-        #   - :eager_loader - A proc to use to implement eager loading, overriding the default.  Takes three arguments,
-        #     a key hash (used solely to enhance performance), an array of records,
-        #     and a hash of dependent associations.  The associated records should
+        #   - :eager_loader - A proc to use to implement eager loading, overriding the default.  Takes one or three arguments.
+        #     If three arguments, the first should be a key hash (used solely to enhance performance), the second an array of records,
+        #     and the third a hash of dependent associations. If one argument, is passed a hash with keys :key_hash,
+        #     :rows, and :associations, corresponding to the three arguments, and an additional key :self, which is
+        #     the dataset doing the eager loading. In the proc, the associated records should
         #     be queried from the database and the associations cache for each
         #     record should be populated for this to work correctly.
         #   - :eager_loader_key - A symbol for the key column to use to populate the key hash
@@ -614,6 +616,7 @@ def associate(type, name, opts = {}, &block)
           raise(Error, 'one_to_many association type with :one_to_one option removed, used one_to_one association type') if opts[:one_to_one] && type == :one_to_many
           raise(Error, 'invalid association type') unless assoc_class = ASSOCIATION_TYPES[type]
           raise(Error, 'Model.associate name argument must be a symbol') unless Symbol === name
+          raise(Error, ':eager_loader option must have an arity of 1 or 3') if opts[:eager_loader] && ![1, 3].include?(opts[:eager_loader].arity)
 
           # merge early so we don't modify opts
           orig_opts = opts.dup
@@ -653,7 +656,7 @@ def associations
         end
 
         # Modify and return eager loading dataset based on association options. Options:
-        def eager_loading_dataset(opts, ds, select, associations)
+        def eager_loading_dataset(opts, ds, select, associations, eager_options={})
           ds = ds.select(*select) if select
           if c = opts[:conditions]
             ds = (c.is_a?(Array) && !Sequel.condition_specifier?(c)) ? ds.filter(*c) : ds.filter(c)
@@ -761,12 +764,12 @@ def def_many_to_many(opts)
           opts[:dataset] ||= proc{opts.associated_class.inner_join(join_table, rcks.zip(opts.right_primary_keys) + lcks.zip(lcpks.map{|k| send(k)}))}
           database = db
 
-          opts[:eager_loader] ||= proc do |key_hash, records, associations|
-            h = key_hash[left_pk]
-            records.each{|object| object.associations[name] = []}
+          opts[:eager_loader] ||= proc do |eo|
+            h = eo[:key_hash][left_pk]
+            eo[:rows].each{|object| object.associations[name] = []}
             r = uses_rcks ? rcks.zip(opts.right_primary_keys) : [[right, opts.right_primary_key]]
             l = uses_lcks ? [[lcks.map{|k| SQL::QualifiedIdentifier.new(join_table, k)}, SQL::SQLArray.new(h.keys)]] : [[left, h.keys]]
-            model.eager_loading_dataset(opts, opts.associated_class.inner_join(join_table, r + l), Array(opts.select), associations).all do |assoc_record|
+            model.eager_loading_dataset(opts, opts.associated_class.inner_join(join_table, r + l), Array(opts.select), eo[:associations], eo).all do |assoc_record|
               hash_key = if uses_lcks
                 left_key_alias.map{|k| assoc_record.values.delete(k)}
               else
@@ -827,16 +830,16 @@ def def_many_to_one(opts)
             klass = opts.associated_class
             klass.filter(opts.primary_keys.map{|k| SQL::QualifiedIdentifier.new(klass.table_name, k)}.zip(cks.map{|k| send(k)}))
           end
-          opts[:eager_loader] ||= proc do |key_hash, records, associations|
-            h = key_hash[key]
+          opts[:eager_loader] ||= proc do |eo|
+            h = eo[:key_hash][key]
             keys = h.keys
             # Default the cached association to nil, so any object that doesn't have it
             # populated will have cached the negative lookup.
-            records.each{|object| object.associations[name] = nil}
+            eo[:rows].each{|object| object.associations[name] = nil}
             # Skip eager loading if no objects have a foreign key for this association
             unless keys.empty?
               klass = opts.associated_class
-              model.eager_loading_dataset(opts, klass.filter(uses_cks ? {opts.primary_keys.map{|k| SQL::QualifiedIdentifier.new(klass.table_name, k)}=>SQL::SQLArray.new(keys)} : {SQL::QualifiedIdentifier.new(klass.table_name, opts.primary_key)=>keys}), opts.select, associations).all do |assoc_record|
+              model.eager_loading_dataset(opts, klass.filter(uses_cks ? {opts.primary_keys.map{|k| SQL::QualifiedIdentifier.new(klass.table_name, k)}=>SQL::SQLArray.new(keys)} : {SQL::QualifiedIdentifier.new(klass.table_name, opts.primary_key)=>keys}), opts.select, eo[:associations], eo).all do |assoc_record|
                 hash_key = uses_cks ? opts.primary_keys.map{|k| assoc_record.send(k)} : assoc_record.send(opts.primary_key)
                 next unless objects = h[hash_key]
                 objects.each{|object| object.associations[name] = assoc_record}
@@ -877,16 +880,16 @@ def def_one_to_many(opts)
             klass = opts.associated_class
             klass.filter(cks.map{|k| SQL::QualifiedIdentifier.new(klass.table_name, k)}.zip(cpks.map{|k| send(k)}))
           end
-          opts[:eager_loader] ||= proc do |key_hash, records, associations|
-            h = key_hash[primary_key]
+          opts[:eager_loader] ||= proc do |eo|
+            h = eo[:key_hash][primary_key]
             if one_to_one
-              records.each{|object| object.associations[name] = nil}
+              eo[:rows].each{|object| object.associations[name] = nil}
             else
-              records.each{|object| object.associations[name] = []}
+              eo[:rows].each{|object| object.associations[name] = []}
             end
             reciprocal = opts.reciprocal
             klass = opts.associated_class
-            model.eager_loading_dataset(opts, klass.filter(uses_cks ? {cks.map{|k| SQL::QualifiedIdentifier.new(klass.table_name, k)}=>SQL::SQLArray.new(h.keys)} : {SQL::QualifiedIdentifier.new(klass.table_name, key)=>h.keys}), opts.select, associations).all do |assoc_record|
+            model.eager_loading_dataset(opts, klass.filter(uses_cks ? {cks.map{|k| SQL::QualifiedIdentifier.new(klass.table_name, k)}=>SQL::SQLArray.new(h.keys)} : {SQL::QualifiedIdentifier.new(klass.table_name, key)=>h.keys}), opts.select, eo[:associations], eo).all do |assoc_record|
               hash_key = uses_cks ? cks.map{|k| assoc_record.send(k)} : assoc_record.send(key)
               next unless objects = h[hash_key]
               if one_to_one
@@ -1526,7 +1529,12 @@ def eager_load(a, eager_assoc=@opts[:eager])
           end
 
           reflections.each do |r|
-            r[:eager_loader].call(key_hash, a, eager_assoc[r[:name]])
+            loader = r[:eager_loader]
+            if loader.arity == 1
+              loader.call(:key_hash=>key_hash, :rows=>a, :associations=>eager_assoc[r[:name]], :self=>self)
+            else
+              loader.call(key_hash, a, eager_assoc[r[:name]])
+            end
             a.each{|object| object.send(:run_association_callbacks, r, :after_load, object.associations[r[:name]])} unless r[:after_load].empty?
           end 
         end