Would be super helpful to get New Relic instrumentation in Sidekiq. Would it be possible to implement using middleware or would have to be added to the rpm_contrib gem?
Here's the documentation:
I've got plenty on my plate at the moment but this is a perfect example of a great feature which could be spiked by someone looking to contribute.
I'll see what I can do to contribute. Would it be a candidate for a middleware?
Here's the resque instrumentation implementation for reference:
Awesome, let me know what the outcome of the discussion is. Meanwhile I'll try to get it working in my Rails app in an initializer.
Tried this approach:
But I couldn't get it to report to New Relic. Anyone has any ideas?
I had a chat with the NR guys today. Their agent collector code is not thread-safe so we'd need to write a bunch of custom code to collect and aggregate the metrics in a thread-safe manner. It sounds like this won't happen any time soon. Perhaps as Sidekiq gets more market share, there will be some pressure for them to refactor their agent API to be thread-safe.
any new news on this?
The only news is to email firstname.lastname@example.org and ask for thread-safe Ruby instrumentation that can be used in tools like Sidekiq.
Oh heck, looks like I should have started doing that instead of what I'm doing now. I'm copying features of Resque-Status with Sidekiq middleware : http://github.com/utgarda/sidekiq-status. Couldn't think of how to do that using only middleware functionality, that's why it's a plugin. Though I'm trying to keep it tidy and not to mess with Sidekiq code too much. Maybe I should switch :)
Any update on this? Would an Enterprise license help?
NewRelic officially supports Resque and Delayed::Job only. If you want to see official support for Sidekiq, please email their support address and ask for it.
I emailed them @mperham. Wonder what they'll say.
EMailed as well
does this http://blog.newrelic.com/2013/01/17/new-relic-ruby-agent-3-5-5-38-come-and-get-it/ change anything on this?
@13k It's great news because it means that NewRelic is starting to take Ruby threading seriously.
I just realised that my client using heroku is paying $40 per month per worker dyno for newrelic to monitor their sidekiq, but they don't actually support it.
I emailed my New Relic account manager and was told that Sidekiq support is on their roadmap for this quarter. Without making any promises, he suggested that it might be released near the end of February.
+1 email sent also
@mperham Did you get any more info from New Relic during your recent visit? Is the Ruby agent now thread safe, opening up Sidekiq integration as an option?
Waiting in anticipation
Looks like it's working now with newrelic_rpm 126.96.36.199.
@jelder How can you tell and how are you using it?
@dfuentes77 I just noticed that our Sidekiq jobs were visible in NewRelic after upgrading to 188.8.131.52 last week. Previously, only our GirlFriday jobs were visible. We're not doing anything terribly special:
def perform ; end
add_transaction_tracer :perform, :category => :task
cool, this is working for me too.
good to finally see some stats on my sidekiq workers.
Official sidekiq support is not out yet. I will post here when it is available.
I am doing a bit of testing on 184.108.40.206. I will let you guys know if I see any issues. I realize the official support isn't there, but if it works I don't really care. =)
Is there a way to see the queue sizes in New Relic?
@cimm: no there is not. They don't have any special hooks for sidekiq yet. However, I can say that the new newrelic agent does indeed work fine with my jobs. I was able to process several hundred thousand jobs on my staging environment with no issues. HOORAY.
Hey all - we (New Relic) have recently released a beta of the newrelic_rpm gem with Sidekiq support, and we'd love to hear feedback if anyone wants to give it a try:
Feel free to contact me directly with feedback, or open support cases if you run into specific issues. Thanks!
We have begun testing the new gem. We will let you know if we run into any issues.
I was running 220.127.116.11 and everything was fine. I then updated to 18.104.22.168 and I can't see Sidekiq monitoring anymore. Any of you guys had that issue?
@dterror The first version of the newrelic_rpm gem that has official Sidekiq support is 22.214.171.124.beta, so I'd encourage you to update to that (or wait for the official release of newrelic_rpm 3.6.0, which should be soon). Did you have some custom instrumentation in place to catch Sidekiq jobs? If so, it should no longer be necessary with 3.6.0.
@benweint I did have custom instrumentation, good to know that won't be needed anymore. Thanks.
So I have the custom instrumentation, too:
I can remove that now, with 3.6.0? Sweet!
@benweint besides the include, we can remove the add_transaction_tracer :work, :category => :task as well?
add_transaction_tracer :work, :category => :task
Oh, I forgot that! Thanks :D
@plentz Yup, you shouldn't need any additional code in your jobs at all now.
Yeah no custom instrumentation is needed anymore. Not only that, the new gem works very well. Been using it in prod for over a week with zero issues.