Need clarification on how to migrate from resque to sidekiq

[roydq]

I am testing replacement of resque with sidekiq in a project that I'm working on. We make heavy use of resque, currently running 50+ workers and sidekiq seems like it could really help us to save on server costs.

I've successfully migrated all of our code over and jobs seem to be running fine in our staging environment, but the web UI appears to be jacked up. I mounted sidekiq-web in our rails app, but it's saying some insane stuff, such as 'Busy Workers: 1027'. The UI is showing resque workers that no longer exist...stuff from 6+ months ago.

We also have our own queue monitoring panel and it appears to show jobs that are in queue just fine, but failed jobs do not show up. I also noticed the same behavior when running resque-web locally. Failed jobs never showed up.

I configured my app to use the resque namespace...although I'm not sure what the point in that really was - could this be causing the issues?

[jc00ke]

What happens when you use a different namespace? It sounds very likely that the Sidekiq web UI is picking up on old Resque data.

[roydq]

I'll give it a shot. I just happened to start working on this a couple of days before 2.0 was released, so I'm going to update to that first.

[mperham]

Sidekiq's queues are slightly different from Resque's so not everything works perfectly. Instead of a failure queue, Sidekiq has an automated retry set with its own UI to view and manage jobs which failed and will be retried.

For the busy workers, that's what happens when you kill -9 sidekiq while it is processing jobs. You can just clear the workers set in redis. I'll add a button to do this in the web UI in 2.0.1.

[roydq]

Thanks for the quick response. What I did for now was pull in the failurejobs middleware from 2.0 and disable the retry middleware. Not quite ready to update to the redis 3.0 gem otherwise I would have updated to sidekiq 2.0.

Much of our jobs are queued up by clockwork and we don't have very good error handling right now, so our retry backlog would get insane pretty quick. Definitely want to clean things up to make use of that feature ASAP though.

Thanks for this excellent gem, I'll def be sure to donate for a commercial license once I can verify my code is going to work.