New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Extract email preferences into contact_rollups_raw #33557
Conversation
You can add |
test 'bunch of email prefences works' do | ||
3.times {|i| create :email_preference, email: "contact_#{i}@rollups.com"} | ||
ContactRollupsRaw.extract_email_preferences | ||
assert 3, ContactRollupsRaw.count |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You also want to do ContactRollupsRaw.delete_all
to assure the table is empty before inserting to it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't that an assumption of our test framework (cleanup/teardown of database before/after every test)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good question. I don't know what is our current test cleaning strategy and how it behaves in local dev machine, in Drone and in test machine. Would be very helpful for me and the team if you could do some research and share what you find.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Thanks, Ben!
Basic logic to extract email preferences into contact_rollups_raw by executing
bin/cron/build_contact_rollups_v2
.Testing story
Tested locally and on a newly created ad hoc connected to a production clone.
email_preferences
has close to a million rows in production -- some interesting findings:truncate
in raw SQL (in this PR) instead ofdelete_all
in ActiveRecord (not in this PR) takes .3 seconds instead of 14 seconds.find_in_batches
batch size from 1000 to 10000 has minimal impact on execution time (actually was 7 seconds slower, 337 vs 330)General finding is raw SQL dramatically outperforming ActiveRecord.
Reviewer Checklist: