Enable push_bulk for various classes

[nitzanav]

Hi,

I personally need a push_bulk method that will accept various worker classes.

I thought that a possible implementation can be implemented in sidekiq/client.rb:

# The code wasn't tested in any way, just edited here on github, so it might not even fail.

# The same exact current code of push_bulk, just without the raw_push
def prepare_push_bulk_payloads(items)
  arg = items['args'].first
  return [] unless arg # no jobs to push
  raise ArgumentError, "Bulk arguments must be an Array of Arrays: [[1], [2]]" if !arg.is_a?(Array)

  normed = normalize_item(items)
  payloads = items['args'].map do |args|
    copy = normed.merge('args' => args, 'jid' => SecureRandom.hex(12), 'enqueued_at' => Time.now.to_f)
    result = process_single(items['class'], copy)
    result ? result : nil
  end.compact
end

def push_bulk(items)
  payloads = prepare_push_bulk_payloads(items)

  raw_push(payloads) if !payloads.empty?
  payloads.collect { |payload| payload['jid'] }
end

# e.g. push_bulk_various_classes(['class' => WorkerA, 'args' => [['a', 1],['b', 2],['c', 3]], 'class' => WorkerB, 'args' => [[:a, 10],[:b, 20],[:c, 30]]]
def push_bulk_various_classes(items)
  payloads = items.flat_map(&method(:prepare_push_bulk_payloads))

  raw_push(payloads) if !payloads.empty?
  payloads.collect { |payload| payload['jid'] }
end

[mperham]

This isn't something I'm looking to add to Sidekiq, this seems like a very specialized API, but there's no reason you can't reopen the class and add it in your app.

[nitzanav]

Do you mean to monkey patch Sidekiq::Client ?

[nitzanav]

Closing issue, since @mperham stated that it will not be implemented.

[mperham]

You can patch it wherever you want.

On Oct 7, 2016, at 16:00, Nitzan Aviram notifications@github.com wrote:

Do you mean to monkey patch Sidekiq::Client ?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.