From Sidekiq to Shoryuken

Although Shoryuken doesn't support the same Sidekiq message format, the migration should be straightforward.

Shoryuken doesn't support Sidekiq message format intentionally because:

• You can't basically move from Sidekiq to Shoryuken (as you could do from Resque to Sidekiq) and still consuming the old messages, because Shoryuken doesn't read messages from Redis, it "reads" from SQS. To migrate from Sidekiq to Shoryuken you will need to keep Sidekiq running until it consumes all pending jobs. The migration plan would be: start using Shoryuken, stop sending jobs to Sidekiq, but keep Sidekiq running until it has old jobs to consume.

• Shoryuken tries to be transparent to SQS as possible, so the message format is the same supported by SQS: string. Anything you can dump as a string (JSON, XML, strings, numbers etc) you can use with Shoryuken. The motivation on that, is because if you use clients implemented in other programming languages or using the aws-sdk directly, Shoryuken will be able to receive their messages. The migration or usage in parallel from other SQS implementations is transparent for Shoryuken, everything will work. Have a look at the body_parser option to dynamic parse formats.

Worker

Sidekiq

class MyWorker
  include Sidekiq::Worker

  sidekiq_options queue: 'my_queue'

  def perform(arg)
    # ...
  end
end

Shoryuken

class MyWorker
  include Shoryuken::Worker

  shoryuken_options queue: 'my_queue', auto_delete: true

  def perform(sqs_msg, arg)
    # ...
  end
end

Configuration file

sidekiq.yml

:concurrency: 25
:pidfile: tmp/pids/sidekiq.pid
:queues:
  - default
  - [myqueue, 2]

shoryuken.yml

:concurrency: 25
:pidfile: tmp/pids/shoryuken.pid
:queues:
  - default
  - [myqueue, 2]

But for Shoryuken, if you are not using IAM roles for EC2 you will need the ENV variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_DEFAULT_REGION exported or the aws key in the configuration file.

aws:
  access_key_id:      ...
  secret_access_key:  ...
  region:             ...

Client usage

Sidekiq

MyWorker.perform_async('test')

Shoryuken

MyWorker.perform_async('test')

Process

Sidekiq

bundle exec sidekiq -r ./my_worker.rb -q my_queue

Shoryuken

bundle exec shoryuken -r ./my_worker.rb -q my_queue