Accessing the queue name from #perform?

[xxx]

Hi,

I am currently porting our jobs from ActiveJob to using Sidekiq directly, but we have a need in one case to track the specific queue a job was popped from when it was performed. ActiveJob has the queue_name method to do this, but I can't find an analogue in Sidekiq. I am aware of sidekiq_options_hash['queue'], but that doesn't reflect overrides from set calls, which is what I need for this.

My backup plan is to use server middleware, which does accurate receive the overridden queue name:

class SidekiqServerMiddleware
  include Sidekiq::ServerMiddleware

  def call(job, _msg, queue)
    job.instance_variable_set(:@queue_name, queue)
    yield
  end
end

This works, but I cannot get this middleware to apply in testing. Adding it to the Sidekiq::Testing.server_middleware chain seems to do nothing. (I'm just doing it in rails_helper.rb at the file level) I'd really rather just get the name directly, but I feel very frustrated that this single string that is widely available everywhere else, isn't available in the one place I really need it. I have to do be missing something.

Is there a better way? Currently on Sidekiq 7.3.9.

[mperham]

The issue is that the queue is orthogonal to the job functionality. It is a breach of encapsulation for the job to look at what queue it was placed into. By adding this type of logic to the job, you're making it harder to test and your code more brittle. Does that make sense?

Your middleware approach is correct. It should work if you use the inline test mode?

[xxx]

The issue is that the queue is orthogonal to the job functionality. <...> Does that make sense?

It makes sense but I disagree pretty strongly. I think this touches on a philosophical disagreement that we would never resolve in a github discussion. Doesn't matter for this though.

The middleware is not working for tests in either inline mode (i.e. Sidekiq::Testing.inline! { }), or in fake mode when using drain. I don't get any warnings or errors, so at the moment I'm completely in the dark as to why this is failing, only in tests.

[mperham]

I can't debug text. Show me code.

[xxx]

The issue was using sync: true in the set call in the test, (which I had added earlier before learning about Sidekiq::Testing.server_middleware,) after seeing it in the Setter class. After removing the sync: true, the middleware is applied as expected in both fake and inline modes.

[xxx]

#!/usr/bin/env rspec

require 'bundler/inline'

gemfile do
  source 'https://rubygems.org'
  gem 'sidekiq'
  gem 'rspec'
end

require 'rspec'
require 'sidekiq/testing'


class MyJob
  include Sidekiq::Job

  attr_reader :queue_name

  def perform
    raise 'no queue name' if queue_name.nil?
    puts "performing w/ #{queue_name}"
  end
end

class SidekiqServerMiddleware
  include Sidekiq::ServerMiddleware

  def call(job, _msg, queue)
    job.instance_variable_set(:@queue_name, queue)
    yield
  end
end

Sidekiq::Testing.server_middleware do |chain|
  chain.add SidekiqServerMiddleware
end

describe MyJob do
  it 'calls the middleware and passes' do
    expect do
      described_class.set(queue: 'stuff').perform_async
      described_class.drain
    end.not_to raise_error
  end

  it 'does not call the middleware and barfs' do
    expect do
      described_class.set(queue: 'stuff', sync: true).perform_async
      described_class.drain
    end.to raise_error 'no queue name'
  end
end

[mperham]

Right, sync is a special case. It can’t refer to testing code and utilities so it only wraps the execution in whatever server middleware is configured in the current context.