Integration specs appear to be broken in RSpec 2

[mhartl]

I'm writing the Ruby on Rails Tutorial book, and the Rails 2.3.8 version of the book uses RSpec integration specs for integration testing. My hope for the Rails 3.0 version is to do the same, but integration testing appears to be broken in RSpec 2. For example, consider this test to check that a user can successfully sign in and out:

require 'spec_helper'

describe "Users" do

    describe "success" do

      before(:each) do
        @user = User.create!(:name => "Example User",
                             :email => "user@example.com",
                             :password => "foobar",
                             :password_confirmationn => "foobar")
      end

      it "should sign a user in and out" do
        visit signin_path
        fill_in :email,    :with => @user.email
        fill_in :password, :with => "foobar"
        click_button
        controller.should be_signed_in
        click_link "Sign out"
        controller.should_not be_signed_in
      end
    end
  end
end

As described here, this exact code works fine with Rails 2.3.8/RSpec 1.3, but it breaks in Rails 3/RSpec 2. Close examination reveals that the user gets created in the before(:each) block, and exists in the database immediately before the call to visit signin_path, but is no longer in the database after the call to visit signin_path. As a result, the form submission (click_button) attempts to sign in a nonexistent user, which of course fails.

I've certainly noticed that Cucumber has gotten a lot of traction in the Rails world, and it seem less likely to break going forward. Although I prefer the pure-Ruby approach of integration specs, I've considered ditching them and converting everything over to Cucumber for the Rails 3 tutorial. Please let me know if you think that's a good idea.

[dchelimsky]

In rspec-2, integration specs live in spec/requests. Is that where you have them? If not, please try changing the directory name and let me know if that solves the problem.

As for Cucumber, I use it myself, but I don't have enough context to make a recommendation for your book.

[mhartl]

They're in spec/requests. In fact, they aren't all broken; I have several passing tests that don't require maintaining state. For example, I have integration specs like

it "should have a Home page at '/'" do
  get '/'
  response.should have_selector('title', :content => "Home")
end

that work just fine. The problem lies in the (non-)persistence of data in the test database between requests.

[mhartl]

I figured out the problem. The issue was that I had set config.cache_classes to false in test.rb. I did this because currently Spork doesn't reload the application files when using Autotest, leading to the absurd situation of not being able to get, say, a failing Users controller spec example to pass by editing the Users controller. (This undermines BDD just a bit.)

Unfortunately, setting config.cache_classes = false also breaks the database support in integration testing. The big breakthrough came when I tried an alternate Cucumber implementation, and it spit out a warning that config.cache_classes = false breaks transactional rollback. That was the hint I needed to start fiddling with the class caching.

By the way, the workaround I came up with is to force-load the application files on each run:

Spork.each_run do
  # Force-load all the app files.
  # This is because (as of this writing) changes to the app/ files
  # aren't loaded when running autotest with spork.
  # We could set config.cache_classes to false in test.rb,
  # but that breaks integration testing.
  Dir["#{Rails.root}/app/**/*.rb"].each { |f| load f }
end

[dchelimsky]

Thanks for posting this info.

[mhartl]

You're welcome. And thanks for the great work on RSpec. In the acknowledgments to the Rails Tutorial book, I express my gratitude to all the Rubyists who have taught and inspired me, and you are one of them.

[MSch]

@mhartl: Thanks for your solution, it got me digging. What you proposed works, but there's a flaw: If you remove a method from a class it won't be removed on reloading.

I've found a different solution that works in that case too:

Spork.prefork do
  # .........

  # Emulate initializer set_clear_dependencies_hook in railties/lib/rails/application/bootstrap.rb
  ActiveSupport::DescendantsTracker.clear
  ActiveSupport::Dependencies.clear
end

Spork.each_run do
  # Dir["#{Rails.root}/app/**/*.rb"].each { |f| load f }
end

# In config/application.rb
module Inviteapp
  class Application < Rails::Application
    # .........

    if Rails.env == 'test'
      initializer :after => :initialize_dependency_mechanism do
        # Work around initializer in railties/lib/rails/application/bootstrap.rb
        ActiveSupport::Dependencies.mechanism = :load
      end
    end
  end
end

[dchelimsky]

You might be interested in http://github.com/timcharper/spork/issues#issue/37.

[mhartl]

Thanks. Yes, that is interesting. With respect to the flaw mentioned by @MSch, I'll give the new solution a try. Having to restart the Spork server on method removal isn't too bad, though, since that's not too common a case.

[mhartl]

@MSch: I tried to implement your workaround, but it failed with the error uninitialized constant ActiveSupport::DescendantsTracker (NameError). Any idea why? I tried including Active Support by hand, but couldn't get it to work.

[MSch]

To get this to work on Rails 3 Beta 4 remove this line from my hack above: ActiveSupport::DescendantsTracker.clear

[gudleik]

doesn't work with rails3 rc

[dchelimsky]

@gudelk - what are you seeing?

[gudleik]

Sorry, my fault. I had some other issues that caused problems.
Tried with a blank rails3-rc project using the solution mentioned above, and it works :)
Tested with rails-3.0.0.rc, rspec-2.0.0.beta.19, spork-0.8.4 and ruby-1.9.2-rc2

[evansagge]

Is this solution still working for Rails 3.0 final? I got the code below in my spec_helper.rb and config/application.rb, but I don't see my Rails code getting reloaded.

# spec/spec_helper.rb
require 'rubygems'      
require 'spork'
require 'active_support/dependencies'

Spork.prefork do
  ActiveSupport::Dependencies.clear
end

Spork.each_run do
end

ENV["RAILS_ENV"] ||= 'test'
require File.dirname(__FILE__) + "/../config/environment" 

Dir["#{File.dirname(__FILE__)}/support/**/*.rb"].each {|f| require f}
Dir["#{File.dirname(__FILE__)}/fixtures/**/*.rb"].each {|f| require f}

RSpec.configure do |config|
end

# config/application.rb
module Workers
  class Application < Rails::Application  
    config.autoload_paths << Rails.root.join('lib')

    config.generators do |g|
      g.orm             :mongoid
      g.test_framework  :rspec
    end

    config.encoding = "utf-8"

    config.filter_parameters += [:password]

    if Rails.env == 'test'
      initializer :after => :initialize_dependency_mechanism do
        # Work around initializer in railties/lib/rails/application/bootstrap.rb
        ActiveSupport::Dependencies.mechanism = :load
      end
    end      
  end
end

[dchelimsky]

Nothing has changed in rspec that should impact this. With spork, I always put everything in the file other in one of the two spork blocks.

[knoopx]

any news on this issue? I'm on rails-3.0.0 and spork doesn't seem to reload any classes, I tried all hacks mentioned here.

[dchelimsky]

knoopx - check the spork tracker