When running tests, you can now switch between running them against a SQLite or PostgreSQL database. This is accomplished by modifying the unit and acceptance tests so that when they generate and load the test Rails application, database.yml is replaced with content that will configure the database appropriately.
Given this scenario: * Using Rails 4.1 * Gemfile has `gem 'shoulda-matchers', require: false` * spec_helper has `require 'shoulda/matchers'` following `require 'rspec/rails'` * Using Spring to run tests matchers that delegate to assertions in Rails (e.g. `render_template` and `route`) will fail in the wrong way if used. They fail because in order to use these assertions, we expect that the assertions will raise a specific exception, an exception that corresponds to whichever test framework that Rails is using. For Rails versions that used Test::Unit, this is Test::Unit::AssertionFailedError. For current Rails versions, which now use Minitest, this exception is Minitest::Assertion. The problem is that instead of asking Rails which exception class it's using, we are trying to detect this exception class ourselves (for cases in which Rails is not being used). This leads to the wrong class being detected: when using a Rails version that uses Minitest, we choose Test::Unit::AssertionFailedError as the class. This happens using the exact scenario above because even though shoulda-matchers is loaded after rspec-rails, rspec-rails itself defines Test::Unit::AssertionFailedError. Also add Cucumber tests that confirms this exact scenario works.
Since as of commit 2748b75, we no longer install dependencies inside of the Rails app that is generated and used to run all of the tests, we have to require all of the dependencies that the app would install inside of the appropriate Appraisals. This was mostly straightforward except for some workarounds with the turn gem: * Rails 3.1 requires two versions of turn depending on which Ruby version you're using. On 1.9.2, it uses turn 0.9.2; after 1.9.2, it uses ~> 0.9.3. To accommodate this we have to have two versions of the Rails 3.1 appraisal which declare the different turn versions. * Rails 3.1 also loads the turn gem even if, in the Gemfile for the app, turn is declared with `require: false`. This causes a problem while running our tests because turn actually requires minitest/autorun, which adds a hook so when Ruby exits, Minitest tests are run. Because we're already using RSpec, Minitest will try to re-run the `rspec` command we ran within a Minitest environment. This will fail since we are using RSpec-specific command line options to run the tests. Unfortunately there's no way to shut off minitest/autorun after it's been required, so we have to monkey-patch Minitest's #run method so it's a no-op.
`bundle install` doesn't always work -- sometimes it runs into errors making HTTP requests, for whatever reason. This will cause Travis to fail which is pretty annoying. * Travis supplies an executable called `travis_retry` which will automatically retry the command up to 3 times before really failing. Tell Travis to use this when it runs `bundle install` before it runs tests. * In spec_helper, we create a Rails app and use this within the test suite. This will also run `bundle install`. Unfortunately we can't use `travis_retry` for this as it's a function and is not available to us in Ruby-land, so use our own retry logic.
* Thanks to @sikachu! Travis will now runs `bundle install --deployment` if it detects that the project has `Gemfile.lock`. However, there's a problem in Bundler deployment install that cause it not installing `bundler` gem into `vendor/bundle` directory, which causes `require 'bundler'` to fail.
Ruby Enterprise Edition has reached its end of life and should be fully compatible with MRI 1.8.7. Testing on both is wasting resources. See: http://blog.phusion.nl/2012/02/21/ruby-enterprise-edition-1-8-7-2012-02-released-end-of-life-imminent/
…bug in Rubinius Tests now also passed on Rubinius 2.0