-
-
Notifications
You must be signed in to change notification settings - Fork 277
From Sidekiq to Shoryuken
Although Shoryuken doesn't support the Sidekiq message format, the migration is straightforward.
Shoryuken doesn't support Sidekiq message format intentionally because:
-
You can't basically move from Sidekiq to Shoryuken (as you could do from Resque to Sidekiq) and keep consuming old messages, because Shoryuken doesn't read messages from Redis, it "reads" from SQS. To migrate from Sidekiq to Shoryuken you will need to keep Sidekiq running until it consumes all pending jobs. The migration plan would be: start using Shoryuken, stop sending jobs to Sidekiq, but keep Sidekiq running until all enqueued jobs.
-
Shoryuken tries to be transparent to SQS as possible, so the message format is the same supported by SQS:
string
. Anything you can dump as a string (JSON, XML, strings, numbers etc) you can use with Shoryuken. The motivation on that, is because if you are using SQS clients implemented in other programming languages or using the aws-sdk directly, Shoryuken will still be able to consume their messages. You can use a body_parser to parse any format you want. The migration or usage in parallel of other SQS clients is transparent for Shoryuken, everything will work just fine.
class MyWorker
include Sidekiq::Worker
sidekiq_options queue: 'my_queue'
def perform(arg)
# ...
end
end
class MyWorker
include Shoryuken::Worker
shoryuken_options queue: 'my_queue', auto_delete: true
def perform(sqs_msg, arg)
# ...
end
end
:concurrency: 25
:pidfile: tmp/pids/sidekiq.pid
:queues:
- default
- [myqueue, 2]
:concurrency: 25
:pidfile: tmp/pids/shoryuken.pid
:queues:
- default
- [myqueue, 2]
If you are not using IAM roles for EC2, you will need the ENV variables:
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION
exported or the aws
defined in shoryuken.yml.
aws:
access_key_id: ...
secret_access_key: ...
region: ...
MyWorker.perform_async('test')
MyWorker.perform_async('test')
bundle exec sidekiq -r ./my_worker.rb -q my_queue
bundle exec shoryuken -r ./my_worker.rb -q my_queue