Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Parallel testing and database: parallel databases needed? #9

Closed
corneverbruggen opened this Issue Jul 21, 2011 · 7 comments

Comments

Projects
None yet
2 participants
Contributor

corneverbruggen commented Jul 21, 2011

Thanks for this great tool. It is really fast and efficient. I like it a lot.

I have an issue though, when running tests in Ruby on Rails (3.1) that interact with a (MySQL) database. The default assumption is an empty database for each test. In fact, after each test the database gets wiped out. When running tests sequentially, that is fine. But now that test-loop runs them in parallel, tests fail when the database gets wiped out during a test, or is populated by another test with data that you'd not expect.

As far as I understand, this could only (but properly) be solved by using parallel databases, like e.g. the parallel_tests gem does.

What do you think about this?

Owner

sunaku commented Jul 21, 2011

Using SQLite3 with an in-memory adapter is a simpler alternative.

Contributor

corneverbruggen commented Jul 22, 2011

Yes, but I use database specific features that are not available in SQLite3.

I managed to change test-loop a bit to let each worker run in a separate (MySQL) database. It works fine for unit and functional tests, but integration tests (using capybara) are quite another story. Capybara runs its own server. This process should be forked parallel to the forks of the main process. I guess that would require quite some changes to test-loop?

Owner

sunaku commented Jul 22, 2011

Interesting. Would you mind sharing your modifications? I'd assume it could be done in the before_each_test handler.

I have used Capybara successfully with test-loop and in-memory SQLite3 in the past. I had the following test/integration_test_helper.rb file and manually required it at the top of my integration tests:

# http://techiferous.com/2010/04/using-capybara-in-rails-3/
# http://groups.google.com/group/ruby-capybara/browse_thread/thread/dd92cb571e76733c
# https://gist.github.com/674284

require 'test_helper'
require 'capybara/rails'
require 'database_cleaner'

Capybara.default_driver = :selenium
DatabaseCleaner.strategy = :truncation

class ActionDispatch::IntegrationTest
  include Capybara
  self.use_transactional_fixtures = false

  setup do
    DatabaseCleaner.start
  end

  teardown do
    DatabaseCleaner.clean
  end
end
Contributor

corneverbruggen commented Jul 24, 2011

About the integration tests: turned out I put the DatabaseCleaner.clean call in the wrong place. So got that working now.

About the parallel databases: at first, I did a quick hack on your loop.rb code, but now I tried to manage it through the before_each_test handler. Unfortunately, I did not manage to get it all working without changes to loop.rb. The problem is that the before_each_test has no access to the workers. And to let before_each_test set a variable that determines which database to use, you need some bookkeeping.

You not only need to know the number of maximum concurrent tests, but also how many and which of these parallel processes are running. Because then you can use a database for each separate process.

So I sent you a pull request: it gives the before_each_test the number of the process (0,1,2 or 3 when using max 4 concurrent tests) in a test_env_number, which can then be used to set a TEST_ENV_NUMBER environment variable. That can be used in the test/test_helper.rb to set the database to use. This way, you completely separate the databases of the concurrent tests. Note: you need to set up the database of course. I have the parallel_tests installed and use its rake tasks rake parallel:create and rake parallel:prepare for that.

The .test-loop looks like this then:

# Set the TEST_ENV_NUMBER environmental variable.
Test::Loop.before_each_test.push lambda { |test_file, log_file, test_name, test_env_number|
  # for compatitibilty with parallel_tests gem, use numbers as strings: "", "2", "3", "4"
  ENV['TEST_ENV_NUMBER'] = (test_env_number == 0 ? "" : (test_env_number + 1).to_s)
} 

And my test_helper.rb looks like this:

ENV["RAILS_ENV"] = "test"
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'
require 'capybara/rails'
require 'database_cleaner'

DatabaseCleaner.strategy = :truncation

class ActiveSupport::TestCase
  setup do
    # Make sure we are connected to the right database (ENV['TEST_ENV_NUMBER'] set by test-loop)
    DatabaseCleaner.start
    connection_config = ActiveRecord::Base.connection_config

    current_number = connection_config[:database][/\d+$/]
    if current_number != ENV['TEST_ENV_NUMBER']
      connection_config[:database] = connection_config[:database][/.+(?=#{current_number})/] + ENV['TEST_ENV_NUMBER']
      ActiveRecord::Base.establish_connection connection_config
    end
    DatabaseCleaner.start
  end

  teardown do
    DatabaseCleaner.clean
  end
end
Owner

sunaku commented Aug 6, 2011

Hey @corneverbruggen, I merged your pull request regarding this feature. Could you please verify that the current git master works for testing your Rails app? Thanks.

Owner

sunaku commented Aug 13, 2011

@corneverbruggen please verify that require 'test/loop/parallel_tests' from git master (at 8d1b029) works for you.

Contributor

corneverbruggen commented Aug 15, 2011

Hi @sunaku, yes it works correctly. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment