Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make AR#increment! AR#decrement! concurency safe #11410

Merged
merged 1 commit into from Oct 10, 2015
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 4 additions & 0 deletions activerecord/CHANGELOG.md
@@ -1,3 +1,7 @@
* Make AR::Base #increment! and decrement! concurrency safe

*Bogdan Gusiev*

* Don't require a database connection to load a class which uses acceptance
validations.

Expand Down
Expand Up @@ -10,7 +10,7 @@ def handle_dependency
def replace(record)
if record
raise_on_type_mismatch!(record)
update_counters(record)
update_counters_on_replace(record)
replace_keys(record)
set_inverse_instance(record)
@updated = true
Expand All @@ -32,45 +32,37 @@ def updated?
end

def decrement_counters # :nodoc:
with_cache_name { |name| decrement_counter name }
update_counters(-1)
end

def increment_counters # :nodoc:
with_cache_name { |name| increment_counter name }
update_counters(1)
end

private

def find_target?
!loaded? && foreign_key_present? && klass
end

def with_cache_name
counter_cache_name = reflection.counter_cache_column
return unless counter_cache_name && owner.persisted?
yield counter_cache_name
def update_counters(by)
if require_counter_update? && foreign_key_present?
if target && !stale_target?
target.increment!(reflection.counter_cache_column, by)
else
klass.update_counters(target_id, reflection.counter_cache_column => by)
end
end
end

def update_counters(record)
with_cache_name do |name|
return unless different_target? record
record.class.increment_counter(name, record.id)
decrement_counter name
end
def find_target?
!loaded? && foreign_key_present? && klass
end

def decrement_counter(counter_cache_name)
if foreign_key_present?
klass.decrement_counter(counter_cache_name, target_id)
end
def require_counter_update?
reflection.counter_cache_column && owner.persisted?
end

def increment_counter(counter_cache_name)
if foreign_key_present?
klass.increment_counter(counter_cache_name, target_id)
if target && !stale_target?
target.increment(counter_cache_name)
end
def update_counters_on_replace(record)
if require_counter_update? && different_target?(record)
record.increment!(reflection.counter_cache_column)
decrement_counters
end
end

Expand Down
Expand Up @@ -88,21 +88,15 @@ def count_records
end

def update_counter(difference, reflection = reflection())
update_counter_in_database(difference, reflection)
update_counter_in_memory(difference, reflection)
end

def update_counter_in_database(difference, reflection = reflection())
if reflection.has_cached_counter?
owner.class.update_counters(owner.id, reflection.counter_cache_column => difference)
owner.increment!(reflection.counter_cache_column, difference)
end
end

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how come this method is removed while the equivalent in has_many_through_association was kept?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I simply moved method to the place where it is used: only in has_many_through_association but not has_many_association.

def update_counter_in_memory(difference, reflection = reflection())
if reflection.counter_must_be_updated_by_has_many?
counter = reflection.counter_cache_column
owner[counter] ||= 0
owner[counter] += difference
owner.increment(counter, difference)
owner.send(:clear_attribute_change, counter) # eww
end
end
Expand Down
12 changes: 7 additions & 5 deletions activerecord/lib/active_record/persistence.rb
Expand Up @@ -332,24 +332,26 @@ def increment(attribute, by = 1)
# Saving is not subjected to validation checks. Returns +true+ if the
# record could be saved.
def increment!(attribute, by = 1)
increment(attribute, by).update_attribute(attribute, self[attribute])
increment(attribute, by)
change = public_send(attribute) - (attribute_was(attribute.to_s) || 0)
self.class.update_counters(id, attribute => change)
clear_attribute_change(attribute) # eww
self
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this changes the behavior from the documentation, which says that these methods return true on success, but now it seems to always return self?
EDIT: the documentation also says that it passes through the attribute setter, which doesn't seem to be the case either.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. /cc @bogdan

end

# Initializes +attribute+ to zero if +nil+ and subtracts the value passed as +by+ (default is 1).
# The decrement is performed directly on the underlying attribute, no setter is invoked.
# Only makes sense for number-based attributes. Returns +self+.
def decrement(attribute, by = 1)
self[attribute] ||= 0
self[attribute] -= by
self
increment(attribute, -by)
end

# Wrapper around +decrement+ that saves the record. This method differs from
# its non-bang version in that it passes through the attribute setter.
# Saving is not subjected to validation checks. Returns +true+ if the
# record could be saved.
def decrement!(attribute, by = 1)
decrement(attribute, by).update_attribute(attribute, self[attribute])
increment!(attribute, -by)
end

# Assigns to +attribute+ the boolean opposite of <tt>attribute?</tt>. So
Expand Down
9 changes: 9 additions & 0 deletions activerecord/test/cases/persistence_test.rb
Expand Up @@ -120,6 +120,15 @@ def test_increment_attribute_by
assert_equal 59, accounts(:signals37, :reload).credit_limit
end

def test_increment_updates_counter_in_db_using_offset
a1 = accounts(:signals37)
initial_credit = a1.credit_limit
a2 = Account.find(accounts(:signals37).id)
a1.increment!(:credit_limit)
a2.increment!(:credit_limit)
assert_equal initial_credit + 2, a1.reload.credit_limit
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we also verify that the in-memory objects have the right value (without a reload) after updating?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This patch doesn't fix that. We can not make everything perfect from one patch. I would prefer to apply it afterwards.

end

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"test increment concurrency" needs to be more precise so future readers of this code understand the guarantee we're making. This is testing the I in ACID, that #increment! isn't vulnerable to stale reads.

Does this single test really cover all the code changes above? It's difficult to determine but seems unlikely.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jeremy Maybe the changes fix some bugs when counter cache is only updated in DB but not on instance, but it is hard to assume where it happens and if it really happens. We have no identity map and we can not be sure we updated all instances in memory with a proper counter value.

So I would consider all these changes a refactoring that just reuses code.

Renamed test name to be more precise about use case.

def test_destroy_all
conditions = "author_name = 'Mary'"
topics_by_mary = Topic.all.merge!(:where => conditions, :order => 'id').to_a
Expand Down