Kyle Drake edited this page Jul 2, 2015 · 26 revisions

Have a great success story in rolling out Sidekiq? Add it here!

Andre Arko, @indirect, Plex:

we replaced 12-15 EC2 medium instances running 4 delayed_job processes each with a single EC2 medium instance running 4 sidekiq workers with 25 concurrency each. The final tally is something like $2500 less per month in EC2 costs.

James Hu, @axsuul, Jellibug:

We process millions of jobs everyday just on the data we get from social media providers like Twitter and Instagram. Sidekiq allows us to do it all on one VPS.

Fredrik Bjork, @fbjork, Banjo:

we went from around 160 dynos down to 10 now.

Jacob Gillespie, @jacobwg, Check a Film:

We load movie release and review information in real-time for any movie using Sidekiq - it's really fast.

Mooktakim Ahmed, @mooktakim, tweet-a-lot.com:

We process all twitter stream posts incredibly fast using only a couple of Sidekiq workers.

Mustafa Turan, @mustafaturan, dakick.com:

We process all event recommendation mailings with sidekiq. We processed all the job with 1/4 time with same number of resque workers and with 1/10 of CPU requirements. Thanks sidekiq!

Kevin Menard, @nirvdrum, Mogotest:

We perform a lot of work as we test Web sites, including lots of I/O wait on external services and some rigorous calculations during data analysis. We segment logical "test runs" (what our customers see) into many subtasks to run in parallel. With Sidekiq we can process 3.5x - 5x as many jobs per EC2 c1.medium than we could with Resque. Plus with a long-lived process, we benefit from JITing on the JVM over the length of a deploy, speeding up our calculations. The switch was a big win all around.

Geoff Wright, @geoffw8, weartolook

We run a lot of jobs: image fetching and processing, mail sending, product classification on lots of different fronts and checking if a product is still available. We had jobs running via Resque on our main app server, we could run ~10 workers before things started affecting the web experience. We have just set up a second, dedicated job server running Sidekiq (32GB RAM, 3.2 Quad-core i7) and I'm yet to find our upper limit. I am currently running 200 jobs, the load average is just over 1 and I'm using around 6.5GB RAM.

Avishai Weiss, @apartable, Apartable

We run several background jobs to process images, update stats and the like. Prior to Sidekiq, I was using Resque, and very quickly hit a wall, but migrating to Sidekiq helped reduce the load on the utility machine tremendously. So far so good!

David Celis, @davidcelis, goodbre.ws

Went from having only two resque workers processing jobs to being able to have 25 workers in one process with a much lower memory footprint.

Vito Botta, @vitobotta, onapp.com

If you were here and were a hot blonde I'd kiss you :D At OnApp we process asynchronously tons of jobs concerning data from all the clouds managed by our clients, plus CDN data, and a lot more. Simply switching from Resque to Sidekiq has made everything so much lighter and faster. Thanks!

Glenn Goodrich, @ruprict, kyck.com

We had 4 Resque queues with multiple workers on heroku and Sidekiq allowed us to shave those by half. After the switch, a bunch of other issues seem to have mysteriously disappeared. Yay!

Marcelo Silveira, @mhfsilveira, 140Proof

At the moment we switched from Resque to Sidekiq, we had 7 busy m1.large boxes running 70 Resque workers. A single Sidekiq process with 25 threads was able to handle all of the load alone. We're now running 2 boxes with 1 process each, just for the sake of redundancy, and planning on reducing the instance type since they're using very few resources.

Piotr Usewicz, @pusewicz, Homestay.com:

We had to quickly spawn more machines to process over 200k image optimisations with various sizes. Adding more Sidekiq workers and making the whole process last much shorter was super easy!

Kyle Drake, @kyledrake, Neocities:

Neocities has used Sidekiq since day one for our endless async worker requirements: Site screenshotting, CDN cache purging, site archiving, thumbnails for images, emails, anti-spam machine learning algorithms, and much more. Simply put, Neocities would not have been possible without Sidekiq. I mean, I literally built another Geocities in an insanely hostile modern web with one person using it, WTF. It's stable, it's reliable, and I can trust it to do the work. It's one of the greatest power toys in Ruby's arsenal, and is the killer app of Tony Arcieri's infrastructure work to improve Ruby concurrency without resorting to the flaming tire fire that is EventMachine. Most importantly, it completely challenges the popular notion that Ruby Can't Scale. Absolutely brilliant.

Previous: Problems and Troubleshooting Next: Related Projects