Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Change UpdateReactableJob to UpdateReactableWorker and move to sidekiq #5551

Merged
merged 10 commits into from
Feb 13, 2020
Merged

Change UpdateReactableJob to UpdateReactableWorker and move to sidekiq #5551

merged 10 commits into from
Feb 13, 2020

Conversation

luchiago
Copy link
Contributor

What type of PR is this? (check all applicable)

  • Refactor
  • Feature
  • Bug Fix
  • Optimization
  • Documentation Update

Description

Related Tickets & Documents

#5305

Added to documentation?

  • docs.dev.to
  • readme
  • no documentation needed

@luchiago luchiago requested a review from a team January 16, 2020 22:05
@pr-triage pr-triage bot added the PR: unreviewed bot applied label for PR's with no review label Jan 16, 2020
@rhymes
Copy link
Contributor

rhymes commented Jan 17, 2020

As a high frequency job I think this should be done in two steps: add the worker and then remove the old job in a separate PR

@luchiago
Copy link
Contributor Author

As a high frequency job I think this should be done in two steps: add the worker and then remove the old job in a separate PR

Sure, I will do this.

rhymes
rhymes suggested changes Jan 17, 2020
View reviewed changes
Copy link
Contributor

@rhymes rhymes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @luchiago, thanks for this PR as well, there a few things to fix but we're almost there!

app/workers/reactions/update_reactable_worker.rb Outdated
class UpdateReactableWorker
include Sidekiq::Worker

sidekiq_options queue: :low_priority, retry: 10
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would raise the priority, what do you think @mstruve ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, both of the actions in this worker are simple very fast SQL updates. I'm wondering if we save ourselves the complexity and do them inline without creating a whole job for it. @rhymes thoughts?

spec/models/reaction_spec.rb Outdated Show resolved Hide resolved
spec/workers/reactions/update_reactable_worker_spec.rb Outdated Show resolved Hide resolved
@pr-triage pr-triage bot added PR: reviewed-changes-requested bot applied label for PR's where reviewer requests changes PR: unreviewed bot applied label for PR's with no review and removed PR: unreviewed bot applied label for PR's with no review PR: reviewed-changes-requested bot applied label for PR's where reviewer requests changes labels Jan 17, 2020
mstruve
mstruve suggested changes Jan 17, 2020
View reviewed changes
Copy link
Contributor

@mstruve mstruve left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am wondering if we even need this worker since the commands it is executing are 2 very fast SQL commands. Regardless, if we do decide to keep it we do need to change the without_delay method

app/models/reaction.rb Outdated Show resolved Hide resolved
app/workers/reactions/update_reactable_worker.rb Outdated
class UpdateReactableWorker
include Sidekiq::Worker

sidekiq_options queue: :low_priority, retry: 10
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, both of the actions in this worker are simple very fast SQL updates. I'm wondering if we save ourselves the complexity and do them inline without creating a whole job for it. @rhymes thoughts?

spec/models/reaction_spec.rb Outdated Show resolved Hide resolved
spec/workers/reactions/update_reactable_worker_spec.rb Show resolved Hide resolved
@pr-triage pr-triage bot added PR: reviewed-changes-requested bot applied label for PR's where reviewer requests changes and removed PR: unreviewed bot applied label for PR's with no review labels Jan 17, 2020
@rhymes
Copy link
Contributor

rhymes commented Jan 18, 2020

@mstruve I think it's warranted, the job now does this:

https://github.com/thepracticaldev/dev.to/blob/4f7697880c285286b44880798bc9b2d7fc5d021f/app/jobs/reactions/update_reactable_job.rb#L9-L10

  1. reactable.touch_by_reaction is aliased to comment.save for comments but for articles does this:

https://github.com/thepracticaldev/dev.to/blob/4f7697880c285286b44880798bc9b2d7fc5d021f/app/models/article.rb#L285-L288

which calls Articles::ScoreCalcJob.perform_later(id) and .index!

  1. reactable.sync_reactions_count does two simple queries:

update_column(:positive_reactions_count, reactions.where("points > ?", 0).size)

I'd say it's okay to keep it because of point 1. We can measure its performance in DD I guess and remove it in the future

@luchiago
Copy link
Contributor Author

luchiago commented Jan 19, 2020 •
edited
Loading

@rhymes So, should I raise the priority #5551 discuss? And what about the spec #5551 discuss? It's related to Articles::ScoreCalcJob

@luchiago
Copy link
Contributor Author

luchiago commented Feb 2, 2020 •
edited
Loading

@rhymes could you look at my last comment

@rhymes So, should I raise the priority #5551 discuss? And what about the spec #5551 discuss? It's related to Articles::ScoreCalcJob

@rhymes
Copy link
Contributor

rhymes commented Feb 2, 2020

@luchiago sorry for the delay :((

yes please raise the priority and investigate why that spec is failing for you, thank you!

ps. there's also a conflict up here, I guess it's because in the meantime other PRs have been merged :(

@weekly-digest weekly-digest bot mentioned this pull request Feb 3, 2020
Closed
@luchiago
Copy link
Contributor Author

Sure @rhymes. I will send the changes

@pr-triage pr-triage bot added PR: unreviewed bot applied label for PR's with no review and removed PR: reviewed-changes-requested bot applied label for PR's where reviewer requests changes labels Feb 11, 2020
rhymes
rhymes previously approved these changes Feb 12, 2020
View reviewed changes
Copy link
Contributor

@rhymes rhymes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes @luchiago!

@rhymes rhymes requested a review from mstruve February 12, 2020 09:43
mstruve
mstruve suggested changes Feb 12, 2020
View reviewed changes
Copy link
Contributor

@mstruve mstruve left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost there!

app/models/reaction.rb
@@ -80,7 +80,7 @@ def touch_user
end

def update_reactable
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is called in an after_save callback which means the reaction will not be persisted to the database yet. We need to move this to an after_commit callback

mstruve
mstruve approved these changes Feb 12, 2020
View reviewed changes
Copy link
Contributor

@mstruve mstruve left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much @luchiago for all your help with this project!!! 🤗

@atsmith813
Copy link
Contributor

@luchiago I hopped in here real quick and fixed the failing spec. That was a result of some code that I had put into master so I figured I'd help out.

Thank you so much for taking such a strong initiative on moving these jobs over to workers, it has been a H-U-G-E help!

rhymes
rhymes approved these changes Feb 13, 2020
View reviewed changes
Copy link
Contributor

@rhymes rhymes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Thanks @luchiago and @atsmith813!

@mstruve mstruve merged commit f68bd8b into forem:master Feb 13, 2020
@pr-triage pr-triage bot added PR: merged bot applied label for PR's that are merged and removed PR: unreviewed bot applied label for PR's with no review labels Feb 13, 2020
@atsmith813 atsmith813 mentioned this pull request Feb 13, 2020
Merged
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rhymes rhymes rhymes approved these changes

@mstruve mstruve mstruve approved these changes

@lightalloy lightalloy Awaiting requested review from lightalloy

@maestromac maestromac Awaiting requested review from maestromac

Assignees
No one assigned
Labels
PR: merged bot applied label for PR's that are merged
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@luchiago @rhymes @atsmith813 @mstruve