Batch touch parent records #19324

arthurnn · 2015-03-13T17:31:22Z

Batch the parent touches. So we doing something like post.comments.each(&:save!) it will only touch the Post parent once, and only before the transaction is sent to the DB.

[fixes #18606]

review @dhh @jeremy

This is heavily inspired from Jeremy's script https://gist.github.com/jeremy/84134ad08f2a137aa1ef . Now that we have the before_commit hooks and the touch(*, time:)

arthurnn · 2015-03-13T17:31:38Z

this is still missing more tests and CHANGELOG, but I want to validate the idea first.

jeremy · 2015-03-13T17:34:54Z

activerecord/lib/active_record/touch_later.rb

+      @_touch_time = current_time_from_proper_timezone
+
+      @_defer_touch_attrs.each { |attr| write_attribute attr, @_touch_time }
+      clear_attribute_changes @_defer_touch_attrs


Would keep #surreptitiously_touch to encapsulate this bit of trickery. Easier to understand

TIL a new word!

arthurnn · 2015-03-14T17:49:00Z

One issue about this implementation is that if we have a 3 level belongs_to, once we touch the grand-child , that will touch_later the child, which will not touch the parent, until the transaction is about to be committed. I am not sure how to fix that, @jeremy any ideas?

arthurnn · 2015-03-28T03:55:40Z

@jeremy Thanks a lot for your review . ❤️ 💛 💚
I did the changed you suggested, also added tests for the touch_later module.
Let me know what else you think I am missing..

arthurnn · 2015-03-30T17:01:34Z

(travis was not running on this PR, so I closed and opened again)

dhh · 2015-04-08T16:42:36Z

Is this good to go? Would be nice to have it committed then. (cc @jeremy)

[fixes rails#18606] Make belongs_to use touch over touch_later when running the callbacks. Add more tests and small method rename Thanks Jeremy for the feedback.

Batch touch parent records

dhh · 2015-04-08T17:34:08Z

🤘

dhh · 2015-06-25T12:23:30Z

This breaks great-grandparent touching chains, like:

Comment.belongs_to :message, touch: true
Message.belongs_to :project, touch: true
Project.belongs_to :account, touch: true
Account

When we do Comment.create!, it'll only trigger a touch on its parent (message) and grandparent (project), not the great-grandparent (account). This is because we're not doing a full save on the Project, so the touch: true callback that Project has against Account is never triggered.

I've committed a skipped, but failing test for this in 5f5e6d9.

arthurnn · 2015-09-13T16:41:05Z

thanks @dhh , I will take a look at it.

brodock · 2015-10-27T15:00:10Z

Is it rails 5 only?

kaspth · 2015-10-27T15:33:30Z

@brodock Rails 5

The problem was that when saving an object, we would call touch_later on the parent which wont be saved immediteally, and it wont call any callbacks. That was working one level up because we were calling touch, during the touch_later commit phase. However that still didnt solve the problem when you have a 3+ levels of parents to be touched, as calling touch would affect the parent, but it would be too late to run callbacks on its grand-parent. The solution for this, is instead, call touch_later upwards when the first touch_later is called. So we make sure all the timestamps are updated without relying on callbacks. This also removed the hard dependency BelongsTo builder had with the TouchLater module. So we can still have the old behaviour if TouchLater module is not included. [fixes 5f5e6d9] [related #19324]

related to #19324 and #18606 [skip ci]

daniel-rikowski · 2016-07-04T14:18:35Z

Is there a way to disable this feature? I'm changing the schema search path during a transaction and when it finishes, the queued touch operations try to access tables, which are no longer available...

arthurnn · 2016-07-13T18:01:50Z

@daniel-rikowski I think that sounds probably like a bug. Do you mind opening a new issue with a script that simulates this situation?
thanks

daniel-rikowski · 2016-07-13T22:20:52Z

I don't think it's a bug. A missing feature perhaps... Rails would have to monitor changes to the schema search path (is that even possible?) and when it changes the delayed touch must be performed beforehand. Much too specialized if you ask me.

Anyway, I created a runnable script which demonstrates the problem: https://gist.github.com/daniel-rikowski/fea2a87dbab975bd5d72556f0a426e34 (You'll need PostgreSQL and an existing database)

If you still think this is a bug I'll gladly open a new issue.

pirtlj · 2019-07-19T17:51:05Z

@dhh just discovered this great feature today, works so well with caching of api compound documents. Thanks for all the work <3

I've found the internal `without_transaction_enrollment` callbacks which have not been newly used over five years, when I tried to work reverting rails#9068 (rails#36049 (comment)). I think that we will never make that callbacks public, since the mechanism of `without_transaction_enrollment` is too implementation specific, at least before rails#9068, records in a transaction had enrolled all into the transaction. That callbacks was introduced at rails#18936 to make `touch_later` rails#19324, but I think that the internal callbacks is overkill to just make the `touch_later` only, and invoking the extra callbacks also have a little overhead even if we haven't used that. So I think we can remove the internal callbacks for now, until we will come up with a good use for that callbacks.

jeremy reviewed Mar 13, 2015
View reviewed changes

arthurnn mentioned this pull request Mar 13, 2015

Fix before_commit when updating a record on the callback #19325

Closed

arthurnn force-pushed the batch_touch branch from 7b53e91 to 5ab0774 Compare March 14, 2015 16:25

carlosantoniodasilva added the activerecord label Mar 16, 2015

jjb mentioned this pull request Mar 17, 2015

ActiveRecord touch causing deadlocks #19372

Closed

arthurnn force-pushed the batch_touch branch from 6df86ff to 143b7b4 Compare March 28, 2015 03:53

arthurnn closed this Mar 30, 2015

arthurnn reopened this Mar 30, 2015

Batch touch parent records

8971389

[fixes rails#18606] Make belongs_to use touch over touch_later when running the callbacks. Add more tests and small method rename Thanks Jeremy for the feedback.

arthurnn force-pushed the batch_touch branch from 143b7b4 to 8971389 Compare April 8, 2015 17:31

arthurnn pushed a commit that referenced this pull request Apr 8, 2015

Merge pull request #19324 from arthurnn/batch_touch

991875f

Batch touch parent records

arthurnn merged commit 991875f into rails:master Apr 8, 2015

arthurnn deleted the batch_touch branch April 8, 2015 17:32

dhh added a commit that referenced this pull request Jun 25, 2015

Add pending test for the great-grandparent touching bug from #19324

5f5e6d9

md5 mentioned this pull request Sep 2, 2015

Difficulty importing large datasets codeforamerica/ohana-api#361

Closed

sgrif mentioned this pull request Sep 24, 2015

touch does not check lock_version #21698

Closed

arthurnn added a commit that referenced this pull request Mar 7, 2016

Add changelog for batch touch records.

f5b553c

related to #19324 and #18606 [skip ci]

mtuckergh mentioned this pull request Mar 12, 2017

can't write unknown attribute :time in Rails 5 godaddy/activerecord-delay_touching#13

Closed

kaspth mentioned this pull request Jul 2, 2017

:touch option should only touch an associated record once per transaction #8759

Closed

kamipo mentioned this pull request May 1, 2020

Remove internal without_transaction_enrollment callbacks #39101

Merged

kennyadsl mentioned this pull request Feb 15, 2021

Should Spree::Product still after_touch Spree::Taxon? Might be causing Deadlocks. solidusio/solidus#3931

Open

sauloperez mentioned this pull request Mar 29, 2021

Update Exchange touch on Enterprise to use #touch_later openfoodfoundation/openfoodnetwork#7236

Merged

p8 mentioned this pull request Oct 10, 2021

Inconsistent behaviour of ActiveRecord::Base#touch since Rails v5.0 #43410

Closed

albertski mentioned this pull request Feb 25, 2023

Eager loading with relations causes touch (touch_later) option not to always work #47501

Closed

tf mentioned this pull request Nov 23, 2023

Nested associations with touch: true cause additional touch on next transaction #50152

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Batch touch parent records #19324

Batch touch parent records #19324

arthurnn commented Mar 13, 2015

arthurnn commented Mar 13, 2015

jeremy Mar 13, 2015

chancancode Mar 13, 2015

arthurnn Mar 14, 2015

arthurnn commented Mar 14, 2015

arthurnn commented Mar 28, 2015

arthurnn commented Mar 30, 2015

dhh commented Apr 8, 2015

dhh commented Apr 8, 2015

dhh commented Jun 25, 2015

arthurnn commented Sep 13, 2015

brodock commented Oct 27, 2015

kaspth commented Oct 27, 2015

daniel-rikowski commented Jul 4, 2016

arthurnn commented Jul 13, 2016

daniel-rikowski commented Jul 13, 2016

pirtlj commented Jul 19, 2019

Batch touch parent records #19324

Batch touch parent records #19324

Conversation

arthurnn commented Mar 13, 2015

arthurnn commented Mar 13, 2015

jeremy Mar 13, 2015

Choose a reason for hiding this comment

chancancode Mar 13, 2015

Choose a reason for hiding this comment

arthurnn Mar 14, 2015

Choose a reason for hiding this comment

arthurnn commented Mar 14, 2015

arthurnn commented Mar 28, 2015

arthurnn commented Mar 30, 2015

dhh commented Apr 8, 2015

dhh commented Apr 8, 2015

dhh commented Jun 25, 2015

arthurnn commented Sep 13, 2015

brodock commented Oct 27, 2015

kaspth commented Oct 27, 2015

daniel-rikowski commented Jul 4, 2016

arthurnn commented Jul 13, 2016

daniel-rikowski commented Jul 13, 2016

pirtlj commented Jul 19, 2019