Added test cases and a fix for rails/rails#10865 - double incrementing of counter cache. #11594

Closed
wants to merge 2 commits into
from

Conversation

Projects
None yet
8 participants

jimjh commented Jul 25, 2013

Assuming Topic has_many Replies and Reply belongs_to Topic, the counter cache was incremented twice if reply is saved after reassignment, as follows:

reply.topic = topic  # triggers BelongsToAssocation#replace
reply.save           # triggers BelongsTo belongs_to_counter_cache_after_update

The problem is caused by duplicate work.

Consider the following various ways to update a foreign key:

Case 1: new, assignment, save

example

This is not affected because it is not an update and BelongsTo belongs_to_counter_cache_after_update is not triggered.

Case 2: new, assignment, save, assignment, save

example

This is not affected because the @_after_create_counter_called flag does its work in BelongsTo.

Case 3: find/create, assignment, save

(Refer to new test case in this pull request.)

In the simplistic case, the save is superfluous, but because BelongsToAssocation#replace and belongs_to_counter_cache_after_update are both triggered, the counter is incremented twice.

Case 4: find/create, append, save

(Refer to new test case in this pull request.)

reply = Reply.find(1)
topic.replies << reply
reply.save

Similarly, the save is superfluous. This is not affected because << does not trigger BelongsToAssocation#replace.

Case 5: find/create, append, assignment, save

(Refer to new test case in this pull request.)

reply.topic = t1    # not saved, but counter updated
t2.replies << t2    # save! called, previous change needs to be undone

Fix

This commit adds @_dirty_but_updated_counter_cache flag to reply if it has already triggered an increment, but has not been saved yet. Thus, when the reply is actually saved, the superfluous increment does not occur.

Note that because of Case 5, the flag cannot be a simple boolean flag.

Jiunn Haur Lim Added test cases and a fix for rails/rails#10865.
Assuming Topic has_many Replies and Reply belongs_to Topic, the counter cache
was incremented twice if reply is saved after reassignment, as follows:

    reply.topic = topic  # triggers BelongsToAssocation#replace
    reply.save           # triggers belongs_to_counter_cache_after_update

The problem is caused by duplicate work.

This commit adds a flag to the reply if it has already triggered an increment,
but has not been saved yet. Thus, when the reply is actually saved, the
superfluous increment does not occur.

Note the following edge case:

    reply.topic = t1    # not saved, but counter updated
    t2.replies << t2    # save! called, previous change needs to be undone
52e30db

joshuaclayton referenced this pull request in thoughtbot/factory_girl Sep 19, 2013

Closed

counter_cache seems to be duplicating #566

@xaviershay xaviershay commented on the diff Sep 22, 2013

...st/cases/associations/belongs_to_associations_test.rb
+
+ topic.replies << reply
+ assert_equal 1, topic.reload.replies_count
+
+ assert reply.save
+ assert_equal 1, topic.reload.replies_count
+ end
+
+ def test_belongs_to_counter_with_reassigning_after_find
+ topic = Topic.create("title" => "t1")
+ reply = Reply.create("title" => "r1", "content" => "r1")
+ reply = Reply.find(reply.id)
+
+ reply.topic = topic
+
+ assert reply.save
@xaviershay

xaviershay Sep 22, 2013

Contributor

I just hit this bug using exactly this test case on v4.0.0. Thanks for the patch!

Owner

rafaelfranca commented Sep 22, 2013

@tenderlove could you review this one?

tenderlove was assigned Sep 22, 2013

@xaviershay xaviershay commented on the diff Sep 22, 2013

.../lib/active_record/associations/builder/belongs_to.rb
end
end
end
+
+ def _belongs_to_counter_cache_record_change(reflection, from_key, to_key)
+ model = reflection.klass
+ cache_column = reflection.counter_cache_column
+ if to_key && model.respond_to?(:increment_counter)
+ model.increment_counter(cache_column, to_key)
+ end
+ if from_key && model.respond_to?(:decrement_counter)
+ model.decrement_counter(cache_column, from_key)
+ end
@xaviershay

xaviershay Sep 22, 2013

Contributor

Pretty sure you need to sort these operations by the primary key of the model, or some other deterministic ordering. Otherwise you could deadlock between two transactions each transferring a model in opposite directions, since this code is locking the parent records in a different order.

@xaviershay

xaviershay Sep 22, 2013

Contributor

First version that sprung to mind, lambas might be excessive though:

ops = []
if to_key && model.respond_to?(:increment_counter)
  ops << [to_key, lambda { model.increment_counter(cache_column, to_key) }]
end
if from_key && model.respond_to?(:decrement_counter)
  ops << [from_key, lambda { model.decrement_counter(cache_column, from_key) }]
end

ops.sort_by(&:first).map(&:last).each(&:call)
@xaviershay

xaviershay Sep 22, 2013

Contributor

Looks like this logic is duplicated in #update_counters as well (existing code, not in your change) that would have to be updated also. So my objection shouldn't block this patch, since it matches existing behaviour. We can fix up the potential dead locks in a subsequent patch.

@jimjh

jimjh Sep 22, 2013

When does increment_counter obtain the lock, and when is it released?

@xaviershay

xaviershay Sep 22, 2013

Contributor

At least in MySQL, when you run an update statement it implicitly places a write lock on the row, which is released at end of transaction. Other databases are similar.

@tenderlove tenderlove commented on the diff Sep 23, 2013

.../active_record/associations/belongs_to_association.rb
end
+
+ owner.instance_variable_set(:@_dirty_but_updated_counter_cache, dirty_flag)
@tenderlove

tenderlove Sep 23, 2013

Owner

Why do we need to do this?

@jimjh jimjh commented on the diff Sep 24, 2013

.../lib/active_record/associations/builder/belongs_to.rb
- elsif attribute_changed?(foreign_key) && !new_record? && association.constructable?
- model = reflection.klass
- foreign_key_was = attribute_was foreign_key
- foreign_key = attribute foreign_key
-
- if foreign_key && model.respond_to?(:increment_counter)
- model.increment_counter(cache_column, foreign_key)
- end
- if foreign_key_was && model.respond_to?(:decrement_counter)
- model.decrement_counter(cache_column, foreign_key_was)
+ return
+ when !attribute_changed?(foreign_key) || new_record? || !association.constructable?
+ return
+ when dirty && dirty == { from: foreign_key_from, to: foreign_key_to }
+ @_dirty_but_updated_counter_cache = false
+ return
@jimjh

jimjh Sep 24, 2013

@tenderlove @_dirty_but_updated_counter_cache is used here as dirty. It indicates that the counter cache has been updated, but the record itself has not been saved. When the record is saved and belongs_to_counter_cache_after_update is triggered, it needs to know that it should not update the counter_cache again.

More details are in the description of this pull request.

jimjh commented Nov 9, 2013

Feedback is always appreciated.

Experienced the same issue here... Is this really a bug or something?

Contributor

nwjsmith commented May 13, 2015

I believe @sgrif fixed this bug in 23bb8d7

Member

sgrif commented May 13, 2015

Yes, I believe you're correct.

sgrif closed this May 13, 2015

According to the AR changelog, this fix was released with 4.2.1, but I can still reproduce this issue in 4.2.5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment