Calling separate workers within a sidekiq worker in rails app

[joes1876]

I noticed a strange problem when using Sidekiq (https://github.com/mperham/sidekiq) with my rails app. I have a model which connects to a remote mysql db to load some csv data into a table using connection_ninja (https://github.com/cherring/connection_ninja). In my worker class i call the model method that loads the data and that works fine. But if i create a subworker inside my main worker, then the subworker is unable to execute the model method. Example.

Below is my model class

class MyModelClass < ActiveRecord::Base
  @conn = use_connection_ninja(:some_db)
  self.table_name = 'SomeTable'
  attr_accessible :name, :state, :country, :employeeId

  def self.update(file_path)
    sql = "LOAD DATA LOCAL INFILE '#{file_path}'
           INTO TABLE SomeTable
           FIELDS TERMINATED BY ',' ENCLOSED BY '\"'
           LINES TERMINATED BY '\n'
           (name,state,country,employeeId)"
    @conn.connection().execute(sql)
  end
end

If i have the following code in my worker everything works fine

class AWorker 
  include Sidekiq::Worker
  sidekiq_options :queue => :first, :retry => false

  def perform(uploaded_file_path)
      MyModelClass.update(uploaded_file_path) #This works
  end
end

But if create another worker class and call that from AWorker then the sidekiq job for ASubWorker is stuck on Busy

class AWorker 
      include Sidekiq::Worker
      sidekiq_options :queue => :first, :retry => false

      def perform(uploaded_file_path)
          ASubWorker.perform_async(uploaded_file_path) #Does not work
      end
end

class ASubWorker
  include Sidekiq::Worker
  sidekiq_options :queue => :first, :retry => false

  def perform(result_file_path)
    MyModelClass.update(result_file_path)    
  end
end

[mperham]

You need to clean up your connections. Sidekiq only cleans up the default AR connection. Use AR's with_connection in your worker.

[galxy25]

I ran into this same issue, which worker needs to be wrapped in an with_connection call? The perform functionality of the calling worker, or the perform method of the called worker?

[mperham]

@galxy25 Whichever is actually using the connection.