Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

partitioning updates working #7584

Open
wants to merge 1 commit into from

5 participants

@keithgabryelski

basic work for supporting partitioned tables in postgresql

these changes are associated with this pull request: #7573
which were changes associated with rails 3.2.8.

If I were to sum up the work, it would be:

  • provide an instance method arel_table used for any operation that has access to the instance should use the instance method to acquire an arel_table associated with the current models attributes.
  • alter class method arel_table to handle parameters, with no parameters do original work -- with parameters associate the table with the specific partitioned table determined by key attributes
  • provide methods to manage key attributes and values (these are the fields the db table is partitioned on)
  • provide an instance method table_name which calls the altered class method table_name which now takes attribute values that should determine the specific partitioned table to name
  • bunch of helper methods in postgresql connection area associated with schema management. this is for reasons 1) create_schema seems like a useful method, 2) adding foreign key is needed because in postgres child partitions need to manage the foreign key references, 3) some sequence method changes to support tables in non-public (well non search path) schema: this is probably generally useful work as rails seems broken about non-public schemas
  • change any self.class.arel_table to self.arel_table
  • create is a little weird because it needs to acquire the primary key if it isn't supplied (for instance ID where your need to fetch from the sequence -- for this work to be complete we need to supply the model instance method "prefetch_primary_key?" instead of it being on connection since prefetching isn't needed for any tables that aren't partitioned by a primary key)
  • some helper methods for finds (from_partition(*x)) which we've found useful in our day to day coding. this method just sets the table name (this is useful because find from the parent table even when partition keys are provided can take an inordinate time if the number of child tables is large -- so specifying the specific child table is useful).

the rest of the code to support partitioning is here: https://github.com/fiksu/partitioned/tree/rails-3-2-8-patching -- you'll need to pull from that branch (which doesn't try to patch rails -- so use it with this pull request). the master branch patches rails 3.2.8 correctly -- you can use it on your own. The current rubygem of partitioned patches rails in a different (and more conservative way) -- I don't think you should look at that code.

You could probably remove a bunch of stuff to make this code faster for the common non-partitioned case.

  • instance arel_table could just call class method arel_table
  • self.class.arel_table could just do the old work
  • instance table_name could just call class method table_name which did just the old work

then one might provide fixups for those methods for models where partitioning is desired.

I think the ugliest part of this code is update -- although I haven't walked down this path, it would seem the best way to manage this would be to add a hook to attribute modifications and fix up the all arel_tables that the attributes point to if the partitioned key values changes

I'm willing to help in any way that makes sense to support partitioning in a future rails version.

@tenderlove
Owner

Thanks! I'll review this soon (leaving town today).

@keithgabryelski

It's important to understand my pull isn't designed for you to accept directly into any release (I think @tenderlove didn't request anything from this pull other than something useful for understanding how I would provide partitioning in activerecord conceptually).

I believe this code is incomplete -- in fact, in my haste to present this pull (refactoring my work into something more digestible) I've found three issues associated with non-partitioned tables [related to non partitioned tables in a non public schema] that don't exist in my original work (the original work is in the monkey patch files in partitioned gem version 1.1.0 or earlier).

I believe the basic work is sound but I think deeper thought is worthy.

I can also conceive of solutions that color outside of the lines -- that aren't necessarily interesting to me directly (as this work solves my business needs) but may be more compelling to rails people.
Specifically: dynamically generating a model's class from partitioned keys as a row is fetched, that is Foo.first results in a single instance of a dynamically generated class named Foo::Partitioned42 associated with a table named foos_partitioned.p42 (i have completely thought this through -- but I think there is something worthy here)

I have other thoughts -- but I think @tenderlove should soak this work in.

@simonoff

Any update?

@keithgabryelski

How is this going? Can I help push this along?

@frodsan frodsan referenced this pull request
Closed

Partitioned #7573

@gaurish

Bumping this so @tenderlove or others might have chance to give feedback.

@tenderlove tenderlove commented on the diff
...rd/connection_adapters/abstract/schema_definitions.rb
@@ -73,6 +73,14 @@ def initialize(base)
@base = base
end
+ #
+ # Builds a SQL check constraint
+ #
+ # @param [String] constraint a SQL constraint
+ def check_constraint(constraint)
+ @columns << Struct.new(:to_sql).new("CHECK (#{constraint})")
@tenderlove Owner

We should move this to ARel (I think). I see why we need this, but I don't like the implementation. ;-)

But it's a DB-aware setting. MySQL has different syntax for partitioning. I think it can be a stub in base adapter with different implementations for each DB.

@tenderlove Owner

I'm specifically complaining about the Struct thing. We should add a node to ARel that handles CHECK

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@tenderlove tenderlove commented on the diff
activerecord/lib/active_record/core.rb
@@ -133,8 +133,25 @@ def ===(object)
# class Post < ActiveRecord::Base
# scope :published_and_commented, published.and(self.arel_table[:comments_count].gt(0))
# end
- def arel_table
- @arel_table ||= Arel::Table.new(table_name, arel_engine)
+ def arel_table(arel_attribute_values = {})
+ @arel_tables ||= {}
+
+ if arel_attribute_values.blank?
@tenderlove Owner

Change this to empty?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@tenderlove
Owner

I've merged master in to this branch and pushed a copy to my fork of Rails.

(I'm going to write some stuff here, and I may be off base, so please correct me or confirm my comments!)

From what I can gather, one of the main goals of this pull request is to make the value that arel_table returns dynamic. This is because we need to somehow calculate the table name for the partitioned table?

I am in favor of making the return value of arel_table dynamic, but I don't think passing in the attribute values, then calling partition_keys is the right way to go. It seems to me that we have two different strategies for calculating the correct table:

  1. Just the table name derived from the class (the current behavior)
  2. Calculate the table name based on partition keys

Rather than passing stuff to arel_table, maybe we should split out these two objects and configure the class to use the particular strategy. e.g.:

class User < AR::Base
  partitioned [:name] # Configures the class to be partitioned and teaches the class about it's keys
end

Does this seem reasonable?

@keithgabryelski

This sounds reasonable given information I probably don't have.

arel_table is fussed with for cases where the target table in the sql statement is acquired from attributes.
(i believe the update case does this by reaching into the first attribute that is to be modified and acquiring its arel_table and finding the table from THAT).

which I believe is here:

module Arel
  ###
  # FIXME hopefully we can remove this
  module Crud
    def compile_update values
      um = UpdateManager.new @engine

      if Nodes::SqlLiteral === values
        relation = @ctx.from
      else
        relation = values.first.first.relation
      end
      um.table relation
      um.set values
      um.take @ast.limit.expr if @ast.limit
      um.order(*@ast.orders)
      um.wheres = @ctx.wheres
      um
    end

Since there is no back reference to the model instance in the Arel::Table (only a reference to the model's class and the table's name at Arel::Table instance creation time) something must be done to get the information needed to calculate the table's name at the code "relation = values.first.first.relation" invocation time.

What is the best way to do that?

@keithgabryelski

There are probably many ways to slice this bacon that I haven't considered. Marking attributes as part of the partition keys (and then forcing those to be passed to compile_update since they wouldn't normally be passed because they would generally not be attributes that are changed). This would have the benefit of supplying the partition keys in the where clause OR supplying the specific table name for the relation (for partition solutions that wish to use one or the other solution).

It also seems like @arel_table could simply be re-calculated whenever a partition key is modified. I'm investigating this solution (although haven't tried to implement it yet). I noticed the correct hook locations seem to be:

active_record/attribute_methods/write.rb#write_attribute
active_record/attribute_methods/serialization.rb#initialize_attributes

[edited note: I investigated re-calculating @arel_table in a #write_attribute hook. It works, but I see no win. It doesn't make the code cleaner and is just less obvious what is happening if anyone is trying to figure out how the code works.]

@keithgabryelski

From what I can gather, one of the main goals of this pull request is to make the value that
arel_table returns dynamic. This is because we need to somehow calculate the table name
for the partitioned table?

Stepping back, we actually want a dynamic "model::table_name" which could be simply model#table_name and model::table_name(attributes = {}) and we want said method to be lazily evaluated such that the table name is only computed after all attributes (actually just the partition key attributes) have been set before sql statements for insert/update/delete are constructed.

It just turns out we need to do this through the arel_table because THAT is the context statement construction is given to work with to compute the table name.

For select statement construction we need something else to determine the partition because the key values may not be specified (although we could ignore this case since "select * from foos" should visit all relevant partitions). I found a very simple way of fast tracking the specific table -- I use a scope like thing: from_partition(x) where 'x' are the partition key values, so:

Employee.from_partition("yoyodyne").find(1)

This is described in a different way, in the README for the partitioned gem

[brain fart edited out]

@keithgabryelski

(i've been reviewing my patches to ensure I respond to your questions in the most accurate way possible)

with respect to the issue with passing attributes to dynamic_arel_table. This is done for exactly one case, constructing an INSERT statement, here:

   class Relation
     #
     # Patches {ActiveRecord}'s building of an insert statement to request
     # of the model a table name with respect to attribute values being
     # inserted.
     #
     # The differences between this and the original code are small and marked
     # with PARTITIONED comment.
     def insert(values)
       primary_key_value = nil

       if primary_key && Hash === values
         primary_key_value = values[values.keys.find { |k|
           k.name == primary_key
         }]

         if !primary_key_value && connection.prefetch_primary_key?(klass.table_name)
           primary_key_value = connection.next_sequence_value(klass.sequence_name)
           values[klass.arel_table[klass.primary_key]] = primary_key_value
         end
       end

       im = arel.create_insert
       #
       # PARTITIONED ADDITION. get arel_table from class with respect to the
       # current values to placed in the table (which hopefully hold the values
       # that are used to determine the child table this insert should be
       # redirected to)
       #
       actual_arel_table = @klass.dynamic_arel_table(Hash[*values.map{|k,v| [k.name,v]}.flatten]) if @klass.respond_to?(:dynamic_arel_table)
       actual_arel_table = @table unless actual_arel_table
       im.into actual_arel_table

       conn = @klass.connection

       substitutes = values.sort_by { |arel_attr,_| arel_attr.name }
       binds       = substitutes.map do |arel_attr, value|
         [@klass.columns_hash[arel_attr.name], value]
       end

       substitutes.each_with_index do |tuple, i|
         tuple[1] = conn.substitute_at(binds[i][0], i)
       end

       if values.empty? # empty insert
         im.values = Arel.sql(connection.empty_insert_statement_value)
       else
         im.insert substitutes
       end

       conn.insert(
         im,
         'SQL',
         primary_key,
         primary_key_value,
         nil,
         binds)
     end
  end

the code (around: im.into actual_arel_table) has access to the model class and the Arel::Table instance as calculated by the model class. We need to pass an appropriate Arel::Table for the specific partition table to im.into, my choice is to pass attributes to arel_table and let it manage the caching all Arel::Tables that are generated for a specific model.

I originally considered (but this seemed intrusive for me to put in a patch) to refactor the call path to Relation#insert such that it had access to the model instance (so Relation can call back to the model instance to generate the Arel::Table for the specific partition table needed for this insert). I admit it never got past the consideration phase once I realized ::dynamic_arel_table could simply be passed all attribute values and calculate it when needed.

@keithgabryelski

Another consideration is a "dynamic default scope" for lack of a better word.

Operations on a model instance, like #delete, #reload, and #update should not use class.unscoped, but rather allow the model to assist in scoping associated with partitioning.

the partitioned gem adds ::from_partition, a class method that resolves to model_class.from("#{partition_table_name} AS #{parent_table_name}")

This is required because the only attribute passed to delete is id. For partitioned tables the best results result from including all keys needed to specify the specific partition AND the primary key for the target row.

Here is delete, as patched by partitioned:

    def delete
      if persisted?
        self.class.from_partition(*self.class.partition_key_values(attributes)).delete(id)
      end
      @destroyed = true
      freeze
    end

It's my opinion ActiveRecord should provide some instance method that, by default, returns self.class.unscoped (i'm guessing that is the correct default) which methods such as #delete, #reload, and #update use and partitioned models can override to better target the specific partition.

For those reading and wondering why the following sql isn't sufficient:

delete from foos where id = 1

if a table is partitioned by created_at::date, the above sql will search all partitioned tables for the target row. if the sql is changed to:

delete from foos where id = 1 and created_at::date = '2010-08-05'

or better yet:

delete from foos_20100805 where id = 1

there is far less work for the database to find the target row.

@keithgabryelski

another consideration, which I've only recently started looking at, is how to handle associations to partitioned tables.

Here is a convoluted example of a message model with attachments:

The messages table is partitioned by created_at.

class Message < ::Partitioned::ByCreatedAt
  attr_accessible :from_user_id, :to_user_id, :subject
  belongs_to :from_user, :class_name => 'User'
  belongs_to :to_user, :class_name => 'User'
  has_many :attachments, :class_name => 'Attachment', :conditions => lambda {|a| "attachments.created_at::date = '#{created_at.to_date}'" }
end

The attachments table is partitioned along with messages table so that foreign key references can be applied and keep referential integrity (this is an issue with how Postgres handles foreign keys TO a partitioned table -- there is no other option other than to partition both tables by the same values)

class Attachment < ByMessageCreatedAt
  attr_accessible :message_created_at, :message_id, :data
  belongs_to :message, :class_name => 'Message', :conditions => lambda {|a| "messages.created_at::date = '#{message_created_at.to_date}'" }
end

I've been handling these cases by using conditions, which is not perfect. It would be ideal if an association could call back to the model to request the scope to fetch the target with all the knowledge a model instance can provide for targeting.

@keithgabryelski

I'm wondering if this has fallen by the wayside? Is there anything I can do to help it along?
If you'd like me to take another crack at the work, please speak up. To sum up what I believe needs to be done:

  • alter and write methods to fetch table name:
    • ::table_name(attributes = {}) and #table_name
  • alter and write methods to fetch arel_table:
    • ::arel_table(attributes = {}) and #arel_table
  • write Arel::Node for CHECK constraint
  • add TableDefinition::check_constraint(constraint_expression)
  • add method to fetch scope from model instance:
    • #scope (default returns model_class.unscoped)
  • update methods #delete, #reload, #update to use #scope
  • add method to assist association with scoping:
    • #association_scope(target_association) (default returns model_class.unscoped)
  • update code to use #association_scope
@chrisccerami

@keithgabryelski @tenderlove Is this still being worked on or is it abandoned now? Since it's been over 2 years since it's been commented on, perhaps it should be closed? Or can something be done to move this along?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Sep 10, 2012
  1. @keithgabryelski
This page is out of date. Refresh to see the latest.
View
4 activerecord/lib/active_record/attribute_methods.rb
@@ -225,10 +225,10 @@ def attribute_method?(attr_name)
# type casted for use in an Arel insert/update method.
def arel_attributes_with_values(attribute_names)
attrs = {}
- arel_table = self.class.arel_table
+ actual_arel_table = arel_table
attribute_names.each do |name|
- attrs[arel_table[name]] = typecasted_attribute_value(name)
+ attrs[actual_arel_table[name]] = typecasted_attribute_value(name)
end
attrs
end
View
8 activerecord/lib/active_record/connection_adapters/abstract/schema_definitions.rb
@@ -73,6 +73,14 @@ def initialize(base)
@base = base
end
+ #
+ # Builds a SQL check constraint
+ #
+ # @param [String] constraint a SQL constraint
+ def check_constraint(constraint)
+ @columns << Struct.new(:to_sql).new("CHECK (#{constraint})")
@tenderlove Owner

We should move this to ARel (I think). I see why we need this, but I don't like the implementation. ;-)

But it's a DB-aware setting. MySQL has different syntax for partitioning. I think it can be a stub in base adapter with different implementations for each DB.

@tenderlove Owner

I'm specifically complaining about the Struct thing. We should add a node to ARel that handles CHECK

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ end
+
def xml(*args)
raise NotImplementedError unless %w{
sqlite mysql mysql2
View
79 activerecord/lib/active_record/connection_adapters/postgresql/schema_statements.rb
@@ -222,9 +222,7 @@ def client_min_messages=(level)
# Returns the sequence name for a table's primary key or some other specified key.
def default_sequence_name(table_name, pk = nil) #:nodoc:
- result = serial_sequence(table_name, pk || 'id')
- return nil unless result
- result.split('.').last
+ serial_sequence(table_name, pk || 'id')
rescue ActiveRecord::StatementInvalid
"#{table_name}_#{pk || 'id'}_seq"
end
@@ -306,6 +304,81 @@ def pk_and_sequence_for(table) #:nodoc:
nil
end
+ #
+ # Get the next value in a sequence. Used on INSERT operation for
+ # partitioning like by_id because the ID is required before the insert
+ # so that the specific child table is known ahead of time.
+ #
+ # @param [String] sequence_name the name of the sequence to fetch the next value from
+ # @return [Integer] the value from the sequence
+ def next_sequence_value(sequence_name)
+ return execute("select nextval('#{sequence_name}')").field_values("nextval").first.to_i
+ end
+
+ #
+ # Get some next values from a sequence.
+ #
+ # @param [String] sequence_name the name of the sequence to fetch the next values from
+ # @param [Integer] batch_size count of values.
+ # @return [Array<Integer>] an array of values from the sequence
+ def next_sequence_values(sequence_name, batch_size)
+ result = execute("select nextval('#{sequence_name}') from generate_series(1, #{batch_size})")
+ return result.field_values("nextval").map(&:to_i)
+ end
+
+ #
+ # Causes active resource to fetch the primary key for the table (using next_sequence_value())
+ # just before an insert. We need the prefetch to happen but we don't have enough information
+ # here to determine if it should happen, so Relation::insert has been modified to request of
+ # the ActiveRecord::Base derived class if it requires a prefetch.
+ #
+ # @param [String] table_name the table name to query
+ # @return [Boolean] returns true if the table should have its primary key prefetched.
+ def prefetch_primary_key?(table_name)
+ return false
+ end
+
+ #
+ # Creates a schema given a name.
+ #
+ # @param [String] name the name of the schema.
+ # @param [Hash] options ({}) options for creating a schema
+ # @option options [Boolean] :unless_exists (false) check if schema exists.
+ # @return [optional] undefined
+ def create_schema(name, options = {})
+ if options[:unless_exists]
+ return if execute("select count(*) from pg_namespace where nspname = '#{name}'").getvalue(0,0).to_i > 0
+ end
+ execute("CREATE SCHEMA #{name}")
+ end
+
+ #
+ # Drop a schema given a name.
+ #
+ # @param [String] name the name of the schema.
+ # @param [Hash] options ({}) options for dropping a schema
+ # @option options [Boolean] :if_exists (false) check if schema exists.
+ # @option options [Boolean] :cascade (false) drop dependant objects
+ # @return [optional] undefined
+ def drop_schema(name, options = {})
+ if options[:if_exists]
+ return if execute("select count(*) from pg_namespace where nspname = '#{name}'").getvalue(0,0).to_i == 0
+ end
+ execute("DROP SCHEMA #{name}#{' cascade' if options[:cascade]}")
+ end
+
+ #
+ # Add foreign key constraint to table.
+ #
+ # @param [String] referencing_table_name the name of the table containing the foreign key
+ # @param [String] referencing_field_name the name of foreign key column
+ # @param [String] referenced_table_name the name of the table referenced by the foreign key
+ # @param [String] referenced_field_name (:id) the name of the column referenced by the foreign key
+ # @return [optional] undefined
+ def add_foreign_key(referencing_table_name, referencing_field_name, referenced_table_name, referenced_field_name = :id)
+ execute("ALTER TABLE #{referencing_table_name} add foreign key (#{referencing_field_name}) references #{referenced_table_name}(#{referenced_field_name})")
+ end
+
# Returns just a table's primary key
def primary_key(table)
row = exec_query(<<-end_sql, 'SCHEMA').rows.first
View
27 activerecord/lib/active_record/core.rb
@@ -133,8 +133,25 @@ def ===(object)
# class Post < ActiveRecord::Base
# scope :published_and_commented, published.and(self.arel_table[:comments_count].gt(0))
# end
- def arel_table
- @arel_table ||= Arel::Table.new(table_name, arel_engine)
+ def arel_table(arel_attribute_values = {})
+ @arel_tables ||= {}
+
+ if arel_attribute_values.blank?
@tenderlove Owner

Change this to empty?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ key_values = nil
+ else
+ key_values = self.table_partition_key_values(arel_attribute_values)
+ end
+ new_arel_table = @arel_tables[key_values]
+ if new_arel_table.blank?
+ new_arel_table = Arel::Table.new(table_name(*key_values), arel_engine)
+ @arel_tables[key_values] = new_arel_table
+ end
+ return new_arel_table
+ end
+
+ def reset_arel_table
+ @arel_tables ||= {}
+ @arel_tables[nil] = nil
end
# Returns the Arel engine.
@@ -364,6 +381,12 @@ def slice(*methods)
Hash[methods.map { |method| [method, public_send(method)] }].with_indifferent_access
end
+ def arel_table
+ symbolized_attributes = attributes.symbolize_keys
+ table_partition_key_values = Hash[*self.class.table_partition_keys.map{|name| [name,symbolized_attributes[name]]}.flatten]
+ return self.class.arel_table(table_partition_key_values)
+ end
+
private
# Under Ruby 1.9, Array#flatten will call #to_ary (recursively) on each of the elements
View
5 activerecord/lib/active_record/counter_cache.rb
@@ -28,8 +28,9 @@ def reset_counters(id, *counters)
reflection = belongs_to.find { |e| e.foreign_key.to_s == foreign_key && e.options[:counter_cache].present? }
counter_name = reflection.counter_cache_column
- stmt = unscoped.where(arel_table[primary_key].eq(object.id)).arel.compile_update({
- arel_table[counter_name] => object.send(association).count
+ actual_arel_table = arel_table
+ stmt = unscoped.where(actual_arel_table[primary_key].eq(object.id)).arel.compile_update({
+ actual_arel_table[counter_name] => object.send(association).count
})
connection.update stmt
end
View
2  activerecord/lib/active_record/locking/optimistic.rb
@@ -123,7 +123,7 @@ def relation_for_destroy
column = self.class.columns_hash[column_name]
substitute = connection.substitute_at(column, relation.bind_values.length)
- relation = relation.where(self.class.arel_table[column_name].eq(substitute))
+ relation = relation.where(arel_table[column_name].eq(substitute))
relation.bind_values << [column, self[column_name].to_i]
end
View
100 activerecord/lib/active_record/model_schema.rb
@@ -50,6 +50,11 @@ module ModelSchema
# If true, the default table name for a Product class will be +products+. If false, it would just be +product+.
# See table_name for the full rules on table/class naming. This is true, by default.
config_attribute :pluralize_table_names
+
+ def table_name
+ symbolized_attributes = attributes.symbolize_keys
+ return self.class.table_name(*self.class.table_partition_keys.map{|attribute_name| symbolized_attributes[attribute_name]})
+ end
end
module ClassMethods
@@ -108,7 +113,7 @@ module ClassMethods
# end
# end
# Post.table_name # => "special_posts"
- def table_name
+ def table_name(*table_partition_key_values)
reset_table_name unless defined?(@table_name)
@table_name
end
@@ -131,7 +136,7 @@ def table_name=(value)
@table_name = value
@quoted_table_name = nil
- @arel_table = nil
+ reset_arel_table
@sequence_name = nil unless defined?(@explicit_sequence_name) && @explicit_sequence_name
@relation = Relation.new(self, arel_table)
end
@@ -167,6 +172,97 @@ def inheritance_column=(value)
@explicit_inheritance_column = true
end
+ #
+ # Returns an array of attribute names (strings) used to fetch the key value(s)
+ # the determine this specific partition table.
+ #
+ # @return [String] the column name used to partition this table
+ # @return [Array<String>] the column names used to partition this table
+ def table_partition_keys
+ return []
+ end
+
+ #
+ # The specific values for a partition of this active record's type which are defined by
+ # {#self.table_partition_keys}
+ #
+ # @param [Hash] values key/value pairs to extract values from
+ # @return [Object] value of partition key
+ # @return [Array<Object>] values of partition keys
+ def table_partition_key_values(values)
+ symbolized_values = values.symbolize_keys
+ return self.table_partition_keys.map{|key| symbolized_values[key.to_sym]}
+ end
+
+ #
+ # This scoping is used to target the
+ # active record find() to a specific child table and alias it to the name of the
+ # parent table (so activerecord can generally work with it)
+ #
+ # Use as:
+ #
+ # Foo.from_partition(KEY).find(:first)
+ #
+ # where KEY is the key value(s) used as the check constraint on Foo's table.
+ #
+ # @param [*Array<Object>] partition_field the field values to partition on
+ # @return [Hash] the scoping
+ def from_partition(*partition_field)
+ table_alias_name = table_alias_name(*partition_field)
+ from("#{table_name(*partition_field)} AS #{table_alias_name}").
+ tap{|relation| relation.table.table_alias = table_alias_name}
+ end
+
+ #
+ # This scope is used to target the
+ # active record find() to a specific child table. Is probably best used in advanced
+ # activerecord queries when a number of tables are involved in the query.
+ #
+ # Use as:
+ #
+ # Foo.from_partitioned_without_alias(KEY).find(:all, :select => "*")
+ #
+ # where KEY is the key value(s) used as the check constraint on Foo's table.
+ #
+ # it's not obvious why :select => "*" is supplied. note activerecord wants
+ # to use the name of parent table for access to any attributes, so without
+ # the :select argument the sql result would be something like:
+ #
+ # SELECT foos.* FROM foos_partitions.pXXX
+ #
+ # which fails because table foos is not referenced. using the form #from_partition
+ # is almost always the correct thing when using activerecord.
+ #
+ # Because the scope is specific to a class (a class method) but unlike
+ # class methods is not inherited, one must use this form (#from_partitioned_without_alias) instead
+ # of #from_partitioned_without_alias_scope to get the most derived classes specific active record scope.
+ #
+ # @param [*Array<Object>] partition_field the field values to partition on
+ # @return [Hash] the scoping
+ def from_partitioned_without_alias(*partition_field)
+ table_alias_name = table_name(*partition_field)
+ from(table_alias_name).
+ tap{|relation| relation.table.table_alias = table_alias_name}
+ end
+
+ def table_alias_name(*partition_field)
+ return table_name(*partition_field)
+ end
+
+ #
+ # partitioning needs to be able to specify if
+ # we should prefetch the primary key (to determine
+ # the specific table we will insert in to we
+ # need to know the partition key values.
+ #
+ # this needs to be on the model NOT the connection
+ #
+ # for the simple case we just pass the question on to
+ # the connection
+ def prefetch_primary_key?
+ connection.prefetch_primary_key?(table_name)
+ end
+
def sequence_name
if base_class == self
@sequence_name ||= reset_sequence_name
View
21 activerecord/lib/active_record/persistence.rb
@@ -113,7 +113,7 @@ def save!(*)
# callbacks, Observer methods, or any <tt>:dependent</tt> association
# options, use <tt>#destroy</tt>.
def delete
- self.class.delete(id) if persisted?
+ self.class.from_partition(*self.class.table_partition_key_values(attributes)).delete(id) if persisted?
@destroyed = true
freeze
end
@@ -303,9 +303,9 @@ def reload(options = nil)
fresh_object =
if options && options[:lock]
- self.class.unscoped { self.class.lock.find(id) }
+ self.class.unscoped { self.class.lock.from_partition(*self.class.table_partition_key_values(attributes)).find(id) }
else
- self.class.unscoped { self.class.find(id) }
+ self.class.unscoped { self.class.from_partition(*self.class.table_partition_key_values(attributes)).find(id) }
end
@attributes.update(fresh_object.instance_variable_get('@attributes'))
@@ -372,7 +372,7 @@ def relation_for_destroy
substitute = connection.substitute_at(column, 0)
relation = self.class.unscoped.where(
- self.class.arel_table[pk].eq(substitute))
+ arel_table[pk].eq(substitute))
relation.bind_values = [[column, id]]
relation
@@ -390,13 +390,24 @@ def update(attribute_names = @attributes.keys)
attributes_with_values = arel_attributes_with_values_for_update(attribute_names)
return 0 if attributes_with_values.empty?
klass = self.class
- stmt = klass.unscoped.where(klass.arel_table[klass.primary_key].eq(id)).arel.compile_update(attributes_with_values)
+ actual_arel_table = arel_table
+ # This is pretty hacky: we adjust the attributes so they are connected to the correct arel_table
+ # we can do this in two places and I've chosen here (which seems less intrusive).
+ # Alternatively we could hook into any attribute change (model.created_at = Time.now.utc) and
+ # adjust all arel_tables in all attributes when any of this model's partition key values change.
+ # That seems like a lot of work.
+ attributes_with_values = Hash[*attributes_with_values.map{|k,v| [actual_arel_table[k.name], v]}.flatten]
+ stmt = klass.unscoped.where(actual_arel_table[klass.primary_key].eq(id)).arel.compile_update(attributes_with_values)
klass.connection.update stmt
end
# Creates a record with values matching those of the instance attributes
# and returns its id.
def create
+ if self.id.nil? && self.class.prefetch_primary_key?
+ self.id = connection.next_sequence_value(self.class.sequence_name)
+ end
+
attributes_values = arel_attributes_with_values_for_create(!id.nil?)
new_id = self.class.unscoped.insert attributes_values
View
4 activerecord/lib/active_record/relation.rb
@@ -41,12 +41,12 @@ def insert(values)
if !primary_key_value && connection.prefetch_primary_key?(klass.table_name)
primary_key_value = connection.next_sequence_value(klass.sequence_name)
- values[klass.arel_table[klass.primary_key]] = primary_key_value
+ values[arel_table[klass.primary_key]] = primary_key_value
end
end
im = arel.create_insert
- im.into @table
+ im.into arel_table(Hash[*values.map{|k,v| [k.name,v]}.flatten])
conn = @klass.connection
View
2  activerecord/lib/active_record/relation/predicate_builder.rb
@@ -57,7 +57,7 @@ def self.build(attribute, value)
array_predicates << values_predicate
array_predicates.inject { |composite, predicate| composite.or(predicate) }
when ActiveRecord::Relation
- value = value.select(value.klass.arel_table[value.klass.primary_key]) if value.select_values.empty?
+ value = value.select(value.arel_table[value.klass.primary_key]) if value.select_values.empty?
attribute.in(value.arel.ast)
when Range
attribute.in(value)
Something went wrong with that request. Please try again.