Profiling

This page is a work in progress and does not reflect reality... yet.

---

Since the dawn of time, it has been effectively impossible to profile a job running in production Sidekiq because Ruby did not offer a thread-safe profiling API.

Until Ruby 3.3 and Sidekiq TBD! Here's how.

Prepare

Add Vernier to your Gemfile:

gem "vernier", "~> 1.0"

Run

1. Start Sidekiq in production.
2. Open a production console.
3. Pick a job type you wish to profile and kick off a job with a special option:

$ bundle exec rails console
> SomeJob.set(profile: "token").perform_async(123, "mike")

The purpose of the token is to associate profile data to you and makes it easier to find your data if other team members are also profiling their jobs. Typically I just use "mike" as my token.

Data Model

The profiler output is stored in Redis as a Hash with key "token-jid", e.g. "mike-1234567890abcdef".

#{token}-#{jid}:
{
  started_at: <timestamp>,
  token: <token>,
  data: <gzip'd JSON>,
  jid: <jid>,
  type: <jobtype>
}

The profiler output data can be quite large so Sidekiq uses gzip to compress it for storage, be careful not to profile every job and quickly fill up your Redis! Profiling data expires after 12 hours.

If your job retries many times, every execution will overwrite that hash with new data. You may wish to include retry: false to disable automatic retries.

Web UI

The Web UI provides a new Profiles tab with a listing of all profile recordings in Redis. You can click a button and see your profile in the Firefox Viewer or download the profile's raw JSON for further analysis.