Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

change sidekiq queueing to bulk push #3536

Merged
merged 1 commit into from Jun 3, 2017

Conversation

Projects
None yet
2 participants
@takayamaki
Copy link
Contributor

takayamaki commented Jun 3, 2017

Now, Mastodon has such a code.

Model.pluck(:id).each do |id|
    Worker.perform_async(id)
end

If Model.pluck(:id) returned large array, this code invoke Worker.perform_async many times.

In this case, under code is better.
Referenced : https://github.com/mperham/sidekiq/blob/master/lib/sidekiq/client.rb#L77

args = Model.pluck(:id).map do |id|
    [id]
end
Sidekiq::Client.push_bulk("class" => Worker, "args" => args)

However, 2nd codes are hard to read.
Therefore, I used sidekiq-bulk gem( https://github.com/aprescott/sidekiq-bulk ).
sidekiq-bulk gem is wrapper of Sidekiq::Client.push_bulk for readability.

Finally, I can write same code like this.

Worker.push_bulk(Model.pluck(:id))
@Gargron

Gargron approved these changes Jun 3, 2017

@Gargron Gargron merged commit 3eedad2 into tootsuite:master Jun 3, 2017

2 checks passed

codeclimate no new or fixed issues
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@takayamaki takayamaki deleted the takayamaki:sidekiq_bulk_push branch Jun 3, 2017

koteitan added a commit to koteitan/mastodon that referenced this pull request Jun 25, 2017

YaQ00 added a commit to YaQ00/mastodon that referenced this pull request Sep 5, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.