Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sidekiq tries to run in test env #1907

Closed
retgoat opened this issue Aug 22, 2014 · 6 comments
Closed

Sidekiq tries to run in test env #1907

retgoat opened this issue Aug 22, 2014 · 6 comments

Comments

@retgoat
Copy link

retgoat commented Aug 22, 2014

Hi All!
I have a very strange issue: Sidekiq tries to establish connection to Redis in test environment.
I'm using sidekiq 3.2.2 with rspec-sidekiq 2.0.0.beta on rails 4.1.4 with ruby 2.1.2

Here is my test

require 'rails_helper'

describe FetchWorker::FetchWorker do
  it 'runs in fetch queue' do
    expect(subject).to be_processed_in :fetch
  end

  it 'not retry when it fails' do 
    expect(subject).to be_retryable false
  end

  it 'enqueues the new job' do
    FactoryGirl.create(:history, running: false)
    FetchWorker::FetchWorker.perform_async 'FetchWorker', true
    expect(FetchWorker::FetchWorker).to have_enqueued_job('FetchWorker', true)
  end # enqueue job

  describe '#perform' do
    let!(:settleddate) { (Time.now - 2.minutes).iso8601(10) }
    let!(:lock_date) { (Time.now - 5.minutes).iso8601(10) }
    context 'success' do
      before do 
        FactoryGirl.create(:history, running: false, NEW_TRANSACTIONS: lock_date)
        FactoryGirl.create(:abp_transaction, settleddate: settleddate, settledyn: 1, transactiontypeid: 1, accountid: 10966)
      end
      it 'perform worker' do
        user = FactoryGirl.build_stubbed(:user, account_id: 10966)
        worker = FetchWorker::FetchWorker.new
        worker.perform
        expect(user.transactions.count).to eq(1)
      end # it perform worker
    end # context success

    context 'fail' do
      it 'perform worker' do
        FactoryGirl.create(:history, running: false, NEW_TRANSACTIONS: lock_date)
        FactoryGirl.create(:abp_transaction, settleddate: settleddate, settledyn: 1, transactiontypeid: 1, accountid: 10966)
        user = FactoryGirl.build_stubbed(:user, account_id: 10966)
        worker = FetchWorker::FetchWorker.new
        allow(worker).to receive(:process_transactions).and_raise('some error')
        expect{ worker.perform }.to raise_error
      end # it perform worker
    end # context fail
  end # describe perform
end # describe FetchWorker

If I try to run bundle exec rspec with working redis, I'm getting pretty nice output:

retgoat@ubuntu:/media/sf_offside/agent_system$ bundle exec rspec
[rspec-sidekiq] WARNING! Sidekiq will *NOT* process jobs in this environment. See https://github.com/philostler/rspec-sidekiq/wiki/FAQ-&-Troubleshooting
.....................................................................................................................2014-08-22T11:35:52.377Z 5140 TID-4ehb4 INFO: Fetching transactions from 2014-08-22T18:30:52.3447179170+07:00.
2014-08-22T11:35:52.394Z 5140 TID-4ehb4 INFO: Lock released.
.2014-08-22T11:35:52.451Z 5140 TID-4ehb4 INFO: Fetching transactions from 2014-08-22T18:30:52.4160675140+07:00.
.

Finished in 29.88 seconds (files took 17.24 seconds to load)
119 examples, 0 failures
Coverage report generated for RSpec to /media/sf_offside/agent_system/coverage. 872 / 873 LOC (99.89%) covered.

But if I try to run tests with stopped redis I'm getting this:

retgoat@ubuntu:/media/sf_offside/agent_system$ bundle exec rspec
Coverage report generated for RSpec to /media/sf_offside/agent_system/coverage. 377 / 855 LOC (44.09%) covered.
/home/retgoat/.rvm/gems/ruby-2.1.2/gems/redis-3.1.0/lib/redis/client.rb:309:in `rescue in establish_connection': Error connecting to Redis on 127.0.0.1:6379 (ECONNREFUSED) (Redis::CannotConnectError)
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/redis-3.1.0/lib/redis/client.rb:304:in `establish_connection'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/redis-3.1.0/lib/redis/client.rb:85:in `block in connect'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/redis-3.1.0/lib/redis/client.rb:266:in `with_reconnect'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/redis-3.1.0/lib/redis/client.rb:84:in `connect'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/redis-3.1.0/lib/redis/client.rb:326:in `ensure_connected'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/redis-3.1.0/lib/redis/client.rb:197:in `block in process'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/redis-3.1.0/lib/redis/client.rb:279:in `logging'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/redis-3.1.0/lib/redis/client.rb:196:in `process'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/redis-3.1.0/lib/redis/client.rb:102:in `call'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/redis-3.1.0/lib/redis.rb:784:in `block in get'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/redis-3.1.0/lib/redis.rb:37:in `block in synchronize'
    from /home/retgoat/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/redis-3.1.0/lib/redis.rb:37:in `synchronize'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/redis-3.1.0/lib/redis.rb:783:in `get'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/sidetiq-0.6.1/lib/sidetiq/schedulable.rb:65:in `get_schedulable_key'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/sidetiq-0.6.1/lib/sidetiq/schedulable.rb:45:in `recurrence'
    from /media/sf_offside/agent_system/app/workers/fetch_worker.rb:6:in `<class:FetchWorker>'
    from /media/sf_offside/agent_system/app/workers/fetch_worker.rb:2:in `<module:FetchWorker>'
    from /media/sf_offside/agent_system/app/workers/fetch_worker.rb:1:in `<top (required)>'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:247:in `require'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:247:in `block in require'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:232:in `load_dependency'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:247:in `require'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:348:in `require_or_load'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:480:in `load_missing_constant'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:180:in `const_missing'
    from /media/sf_offside/agent_system/spec/workers/fetch_worker_spec.rb:3:in `<top (required)>'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:241:in `load'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:241:in `block in load'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:232:in `load_dependency'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:241:in `load'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/rspec-core-3.0.4/lib/rspec/core/configuration.rb:1058:in `block in load_spec_files'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/rspec-core-3.0.4/lib/rspec/core/configuration.rb:1058:in `each'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/rspec-core-3.0.4/lib/rspec/core/configuration.rb:1058:in `load_spec_files'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/rspec-core-3.0.4/lib/rspec/core/runner.rb:97:in `setup'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/rspec-core-3.0.4/lib/rspec/core/runner.rb:85:in `run'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/rspec-core-3.0.4/lib/rspec/core/runner.rb:70:in `run'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/rspec-core-3.0.4/lib/rspec/core/runner.rb:38:in `invoke'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/gems/rspec-core-3.0.4/exe/rspec:4:in `<top (required)>'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/bin/rspec:23:in `load'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/bin/rspec:23:in `<main>'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/bin/ruby_executable_hooks:15:in `eval'
    from /home/retgoat/.rvm/gems/ruby-2.1.2/bin/ruby_executable_hooks:15:in `<main>'

rails_helper.rb

# This file is copied to spec/ when you run 'rails generate rspec:install'
ENV["RAILS_ENV"] ||= 'test'
require 'spec_helper'
require File.expand_path("../../config/environment", __FILE__)

require 'rspec/rails'
require 'database_cleaner'
require 'factory_girl_rails'
require 'capybara/rails'
require 'capybara/rspec'
require 'capybara/poltergeist'
require 'webmock/rspec'
require 'sidekiq'
require 'sidekiq/testing'
Sidekiq::Testing.fake!

WebMock.disable_net_connect!(:allow_localhost => true)

# Start simplecov
require 'simplecov'
SimpleCov.start

# Requires supporting ruby files with custom matchers and macros, etc, in
# spec/support/ and its subdirectories. Files matching `spec/**/*_spec.rb` are
# run as spec files by default. This means that files in spec/support that end
# in _spec.rb will both be required and run as specs, causing the specs to be
# run twice. It is recommended that you do not name files matching this glob to
# end with _spec.rb. You can configure this pattern with the --pattern
# option on the command line or in ~/.rspec, .rspec or `.rspec-local`.
Dir[Rails.root.join("spec/support/**/*.rb")].each { |f| require f }


# Capybara configuration
Capybara.default_driver = :poltergeist
Capybara.javascript_driver = :poltergeist

Capybara.register_driver :poltergeist do |app|
  Capybara::Poltergeist::Driver.new(app, { js_errors: false, timeout: 520 })
end

# Checks for pending migrations before tests are run.
# If you are not using ActiveRecord, you can remove this line.
ActiveRecord::Migration.maintain_test_schema!

RSpec.configure do |config|
  # Remove this line if you're not using ActiveRecord or ActiveRecord fixtures
  config.fixture_path = "#{::Rails.root}/spec/fixtures"

  # If you're not using ActiveRecord, or you'd prefer not to run each of your
  # examples within a transaction, remove the following line or assign false
  # instead of true.
  config.use_transactional_fixtures = true

  config.include Capybara::DSL
  config.include FactoryGirl::Syntax::Methods
  config.include Devise::TestHelpers, :type => :controller

  config.pattern = "**/*_spec.rb"

  # config.before do
  #   WebMock.disable_net_connect! allow: /http:\/\/127.0.0.1:\d*\/__identify__/
  # end

  config.after(:suite) do
    DatabaseCleaner.clean_with(:truncation)
  end

  config.before(:each) do
    Rails.cache.clear
    Sidekiq::Worker.clear_all
  end

  # RSpec Rails can automatically mix in different behaviours to your tests
  # based on their file location, for example enabling you to call `get` and
  # `post` in specs under `spec/controllers`.
  #
  # You can disable this behaviour by removing the line below, and instead
  # explicitly tag your specs with their type, e.g.:
  #
  #     RSpec.describe UsersController, :type => :controller do
  #       # ...
  #     end
  #
  # The different available types are documented in the features, such as in
  # https://relishapp.com/rspec/rspec-rails/docs
  config.infer_spec_type_from_file_location!
end

RSpec::Sidekiq.configure do |config|
  # Clears all job queues before each example
  config.clear_all_enqueued_jobs = true # default => true

  # Whether to use terminal colours when outputting messages
  config.enable_terminal_colours = true # default => true

  # Warn when jobs are not enqueued to Redis but to a job array
  config.warn_when_jobs_not_processed_by_sidekiq = true # default => true
end

I just have no idea why it happens.

Could you please help with that?

Many thanks.
@mperham
Copy link
Collaborator

mperham commented Aug 22, 2014

https://github.com/tobiassvn/sidetiq/issues/103

@mperham mperham closed this as completed Aug 22, 2014
@retgoat
Copy link
Author

retgoat commented Aug 23, 2014

Thanks, Mike!

@jonsgreen
Copy link

This is happening for me as well but I am not using Sidetiq.

@mperham
Copy link
Collaborator

mperham commented Aug 27, 2014

Post the backtrace.

On Aug 26, 2014, at 20:09, Jonathan notifications@github.com wrote:

This is happening for me as well but I am not using Sidetiq.

—
Reply to this email directly or view it on GitHub.

@jonsgreen
Copy link

I am realizing now that it is probably related to me doing something unconventional in that I am using the batch middleware in my tests. I am within a Sidekiq::Testing.fake! block though. Here is the backtrace nonetheless. I am able to get around this using this hack but I know that you don't recommend mocking Redis. Is there a way that I can avoid needing redis running to test this?

 Failure/Error: ScheduledBatchEmailWorker.drain
 Redis::CannotConnectError:
   Error connecting to Redis on 127.0.0.1:6379 (ECONNREFUSED)
 # /Users/jonathangreenberg/.rvm/gems/ruby-2.1.0@platform/gems/redis-3.1.0/lib/redis/client.rb:309:in `rescue in establish_connection'
 # /Users/jonathangreenberg/.rvm/gems/ruby-2.1.0@platform/gems/redis-3.1.0/lib/redis/client.rb:304:in `establish_connection'
 # /Users/jonathangreenberg/.rvm/gems/ruby-2.1.0@platform/gems/redis-3.1.0/lib/redis/client.rb:85:in `block in connect'
 # /Users/jonathangreenberg/.rvm/gems/ruby-2.1.0@platform/gems/redis-3.1.0/lib/redis/client.rb:266:in `with_reconnect'
 # /Users/jonathangreenberg/.rvm/gems/ruby-2.1.0@platform/gems/redis-3.1.0/lib/redis/client.rb:84:in `connect'
 # /Users/jonathangreenberg/.rvm/gems/ruby-2.1.0@platform/gems/redis-3.1.0/lib/redis/client.rb:326:in `ensure_connected'
 # /Users/jonathangreenberg/.rvm/gems/ruby-2.1.0@platform/gems/redis-3.1.0/lib/redis/client.rb:197:in `block in process'
 # /Users/jonathangreenberg/.rvm/gems/ruby-2.1.0@platform/gems/redis-3.1.0/lib/redis/client.rb:279:in `logging'
 # /Users/jonathangreenberg/.rvm/gems/ruby-2.1.0@platform/gems/redis-3.1.0/lib/redis/client.rb:196:in `process'
 # /Users/jonathangreenberg/.rvm/gems/ruby-2.1.0@platform/gems/redis-3.1.0/lib/redis/client.rb:102:in `call'
 # /Users/jonathangreenberg/.rvm/gems/ruby-2.1.0@platform/gems/redis-3.1.0/lib/redis.rb:691:in `block in setex'
 # /Users/jonathangreenberg/.rvm/gems/ruby-2.1.0@platform/gems/redis-3.1.0/lib/redis.rb:37:in `block in synchronize'
 # /Users/jonathangreenberg/.rvm/rubies/ruby-2.1.0/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
 # /Users/jonathangreenberg/.rvm/gems/ruby-2.1.0@platform/gems/redis-3.1.0/lib/redis.rb:37:in `synchronize'
 # /Users/jonathangreenberg/.rvm/gems/ruby-2.1.0@platform/gems/redis-3.1.0/lib/redis.rb:690:in `setex'
 # /Users/jonathangreenberg/.rvm/gems/ruby-2.1.0@platform/gems/sidekiq-pro-1.7.5/lib/sidekiq/batch.rb:163:in `block in jobs'
 # /Users/jonathangreenberg/.rvm/gems/ruby-2.1.0@platform/gems/connection_pool-2.0.0/lib/connection_pool.rb:58:in `with'
 # /Users/jonathangreenberg/.rvm/gems/ruby-2.1.0@platform/gems/sidekiq-pro-1.7.5/lib/sidekiq/batch.rb:206:in `redis'
 # /Users/jonathangreenberg/.rvm/gems/ruby-2.1.0@platform/gems/sidekiq-pro-1.7.5/lib/sidekiq/batch.rb:162:in `jobs'
 # ./app/workers/blast_worker.rb:59:in `batch_deliver_email'
 # ./app/workers/blast_worker.rb:39:in `block (2 levels) in batch_deliver_blast'
 # ./app/workers/blast_worker.rb:38:in `each'
 # ./app/workers/blast_worker.rb:38:in `each_slice'
 # ./app/workers/blast_worker.rb:38:in `with_index'
 # ./app/workers/blast_worker.rb:38:in `block in batch_deliver_blast'
 # ./app/workers/blast_worker.rb:34:in `each'
 # ./app/workers/blast_worker.rb:34:in `batch_deliver_blast'
 # ./app/workers/scheduled_batch_email_worker.rb:7:in `perform'
 # /Users/jonathangreenberg/.rvm/gems/ruby-2.1.0@platform/gems/sidekiq-3.1.4/lib/sidekiq/testing.rb:156:in `drain'
 # ./spec/models/blast_spec.rb:126:in `block (5 levels) in <top (required)>'
 # /Users/jonathangreenberg/.rvm/gems/ruby-2.1.0@platform/gems/sidekiq-3.1.4/lib/sidekiq/testing.rb:15:in `call'
 # /Users/jonathangreenberg/.rvm/gems/ruby-2.1.0@platform/gems/sidekiq-3.1.4/lib/sidekiq/testing.rb:15:in `__set_test_mode'
 # /Users/jonathangreenberg/.rvm/gems/ruby-2.1.0@platform/gems/sidekiq-3.1.4/lib/sidekiq/testing.rb:29:in `fake!'
 # ./spec/models/blast_spec.rb:123:in `block (4 levels) in <top (required)>'

@mperham
Copy link
Collaborator

mperham commented Aug 27, 2014

rspec-sidekiq has some Batch support, I think it mocks it out so you don't need redis?

@maurercw maurercw mentioned this issue May 22, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mperham @jonsgreen @retgoat