Skip to content
This repository

prevent users from using both capybara and webrat #406

Closed
dchelimsky opened this Issue June 25, 2011 · 30 comments

8 participants

David Chelimsky Justin Ko Andy Lindeman José Valim Jonas Nicklas Myron Marston Steve Brown
David Chelimsky
Owner

Capybara and Webrat use some of the same method names which leads to runtime collisions that reveal themselves in confusing ways. We should raise an error if both libraries are present, but we should also support an override for this so that people who know what they are doing can use Webrat and Capybara in different contexts in the same suite.

José Valim

I came here to open a similar issue. :)

I don't think the issue is between Capybara and Webrat but Capybara and rack-test. Both Rails and Webrat are built on top of rack-test and expose the rack-test API. However, Capybara works differently. For instance, the following request spec won't work with Capybara (regardless if you are using Webrat or simply Rails):

it "should be created with success" do
  get posts_path
  click_link "New Post"

  fill_in "Title", :with => "My first post"
  fill_in "Body",  :with => "Is quite short"
  click_button "Create Post"

  response.status.should be(200)
end

The issue here is that both get and response are rack-test/rails/webrat's methods. While all the others are Capybara's. If you mix them, it will explode. So what is happening when using requests specs with capybara is that it has a mix of methods that won't work together and that is very confusing.

So instead of raising when Webrat and Capybara are loaded together (which actually seems fine as you may be moving from Webrat to Capybara), I propose to have two different test groups. As capybara already includes its DSL for the acceptance group, maybe we could just go with it. The only extra thing we would need to add is Rails url helpers:

config.include Rails.application.routes.url_helpers, :type => :acceptance

I also would like to mention that Webrat and Capybara are somewhat different conceptually as Capybara hides the request and response objects from you, forcing you to stay more on the "browser mindset". If you want to test APIs using Capybara, you will likely enter a world of pain, while Rails/Webrat/rack-test helpers are probably a better tool for the job. Imho, this difference on its own is enough to have two different tests groups. :)

/cc @jnicklas

Jonas Nicklas

This is pretty tricky. As José mentions, there is a valid point to be made about allowing the use of both Capybara and rack-test in the same suite. However, I can't count the number of posts to the mailing list I've got over the years due to this problem, this really does trip a lot of people up pretty badly. It's especially aggravating since this did work with Webrat, so many people are vert used to it working.

It's very difficult for us to detect if someone is using the two separate methods in a nonsensical way together. I see no way of fixing Capybara, and I see few ways of providing an error or a warning which doesn't annoy people who are using these tools together in the way they were meant to be used.

The idea of using :acceptance is rather nice, though I suspect this would break the existing rspec-rails API somewhat, since it currently does include rack-test, right? Also, it would break Capybara a little bit, but that might be okay.

David Chelimsky
Owner

@jnicklas - what do mean it did work with Webrat? I've always seen conflicts if you loaded both libraries, but it's only a problem when you mix the APIs.

@josevalim - Agreed that we need some mechanism to say "this spec uses capybara, and this other one uses rack-test/webrat", but I think it should be more explicit than :integration vs :request vs :acceptance. How about a means of configuring things (i.e. RSpec.configure) that let's the user define the criteria by which to load which lib? It could be based on type or directory, and then each group that might include both would result in an error if rspec couldn't determine which lib to include.

David Chelimsky
Owner

@myronmarston, @justinko - do either of you have any thoughts about this?

Justin Ko

"January 23, 2011" <-- the last date Webrat was updated.

I think we can all agree Capybara is the superior library here - thanks Jonas!

However, that is not enough justification to allow just one or the other. What does justify it, is the very similar API's. To switch from Webrat to Capybara is usually just "find and replace". Nothing like switching from T/U to RSpec :)

I'm strongly in favor of the title of this issue.

José Valim

Thanks everyone for joining the conversation!

In terms of usage, we have two issues today:

  • request includes all Rails related helpers (like routes and stuff), plus rack-test ones (from Rails), plus webrat ones, plus capybara ones. rack-test and webrat do not have compatibility issues, as the latter is built on top of the former;
  • acceptance includes only capybara helpers (without Rails related stuff);

This means that I will likely have conflicts if I use the request test group but I cannot move to acceptance straight away because it lacks Rails helpers. I can certainly work around this issue, but I believe most people cannot. That said, we need to provide two distinct packages:

  • Rails helpers + Capybara
  • Rails helpers + rack-test (+ webrat)

If we can make this work with a configuration option, it would be awesome! But I think we would have to introduce backward incompatible changes in Capybara regardless, as both Capybara and Rspec include stuff in the request example group by default and we should change it to avoid conflicts.

Remember this is not about the best tool out there, because an application may make use of the two packages for testing different stuff. As I said previously, rack-test/webrat is a better tool to test APIs while for everything else I do recommend Capybara.

Myron Marston
Owner

Are the conflicting methods all injected via included modules? If so, would it be workable to change rspec-rails so it does not include them, and force the end user to decide in which example groups they want to include the modules? RSpec already provides a nice API to allow users to configure which modules get included in which example groups via metadata.

Of course, this may break backwards compatibility. I'm a bit out of the loop on what gets included automatically be rspec-rails since I don't do too much with rails these days.

Justin Ko

@myronmarston - they would have to do this:

RSpec.configure do |c|
    c.include Capybara::RSpecMatchers, :type => :view
    c.include Capybara::RSpecMatchers, :type => :helper
    c.include Capybara::RSpecMatchers, :type => :mailer
    c.include Capybara::RSpecMatchers, :type => :controller
    c.include Capybara::DSL, :type => :controller
    c.include Capybara, :type => :request
    c.include Capybara, :type => :controller
end

Justin Ko

@myronmarston - Personally, I would only need to do this :)

RSpec.configure do |c|
    c.include Capybara, :type => :request
end
Justin Ko

@josevalim

I just scoured the rspec google group and github issues. I cannot find one single instance where someone wants to use Webrat and Capybara at the same time. Once they find out using them together causes issues, they throw one or the other out. As I said before, this is not an issue for them because the API's are so similar.

This means that I will likely have conflicts if I use the request test group but I cannot move to acceptance straight away because it lacks Rails helpers

To get Rails helpers into your "acceptance" specs, just add this to spec_helper.rb:

c.include RSpec::Rails::RequestExampleGroup, :example_group => {
    :file_path => /spec\/acceptance/
  }

Then, we would just need to change this line of code to this:

metadata[:type] = :request unless metadata[:type]

Will that work?

P.S. I don't understand why anyone would use Webrat to test API's. Maybe the have_[content|xpath] methods for the response?

José Valim

@justinko I already do such tricks ;) I am just explaining in general why it doesn't work by default and that in any case the user needs to do a (not so obvious) task.

About APIs, you cannot easily set custom headers with Capybara. So if you need to manipulate the request object somehow before doing the request, you need webrat or rack-test instead.

Just to make it clear, when I talk about using Webrat and Capybara together, I mean in different example groups inside rspec. I usually use, spec/request for Webrat and spec/acceptance for Capybara.

David Chelimsky
Owner

I'm OK with breaking compatibility for this issue (it's currently broken, and it's an integration issue) as long as we don't do it silently.

Based on everybody's comments here, here's what I propose:

  1. When a group loads up that can include Capy or Webrat, it raises an error explaining to the user what to do. This would only happen for users who have both libs in their Gemfiles, so already it only affects a subset of all users.

  2. We offer a one liner that lets you map Capy or Webrat:

RSpec.configure do |c|
  c.use_webrat, :type => :api
  c.use_capybara, :type => [:request, :controller, :view, :helper]
end

rspec-rails would then include the correct components for each type.

This would probably conflict with Capybara's RSpec integration, so we'd need to address that somehow - probably a coordinated release. If we do that, I want to do it ONCE, and do it right, so let's take our time on this.

Thoughts?

José Valim

Sounds good to me!

Justin Ko

About APIs, you cannot easily set custom headers with Capybara. So if you need to manipulate the request object somehow before doing the request, you need webrat or rack-test instead.

My point was, for API's, what benefit does Webrat offer you over just using straight Rails integration helpers (rack-test)? Need to set request headers?:

get '/products', {sale: true}, {accept: 'text/plain'}

VS Webrat:

header 'Accept', 'text/plain'
visit '/products', :get, sale: true

With the induction of Capybara, I see absolutely no use case for Webrat. For each individual RSpec example, either use Capybara or rack-test (Rails). People need to stop mixing the two!

Last year, when Capybara was becoming the standard, we got quite a few Webrat/Capybara integration questions. This is because people were in the process of switching from Webrat to Capybara. This process is largely over, hence the minute amount of issues raised about it recently.

Okay I've said enough, anything more is just trolling ;)

David Chelimsky
Owner

Even without Webrat, you can still do this with Capybara:

get '/products'
page.should contain("Products")

This leads to an error because page (reasonably) doesn't know anything about the response from get.

I'm wondering if this is really just a matter of documentation.

José Valim
Justin Ko

@dchelimsky - the point was that Capybara is basically useless for spec'ing API's, because you cannot modify the request object. My point was that Webrat offers nothing for spec'ing API's because it's actually easier to just use the Rails helpers (rack-test).

But yes, I agree beefing up the documentation about this would be a good idea. The current doc says nothing about the problems that arise when using both Webrat and Capybara at the same time.

Justin Ko justinko closed this August 10, 2011
Justin Ko justinko reopened this August 10, 2011
José Valim
David Chelimsky
Owner

@justinko - my last example didn't involve Webrat at all. It's just Capybara sitting on top of the request example group.

Justin Ko

@josevalim @dchelimsky - You would need to include RSpec::Rails::RequestExampleGroup, but currently, including it will override your metadata[:type] value (setting it to :request, which would then include Capybara). We would still need to do the unless metadata[:type] clause I mentioned earlier to keep it as :acceptance.

Justin Ko

spelling fix: "which would then include Webrat"

José Valim
Justin Ko

@josevalim - see my typo fix above :)

José Valim
Justin Ko

@josevalim - ah, my bad. I thought you wanted the rails test helpers with Capybara (since they do not cause conflict if you don't use them together), but also to not include Webrat.

can I have a summary of the current status of possible coexistence of both capybara and webrat? I have a set of tests that worked when both capybara and webrat are used (and both are included in the Gemfile), and another set of tests that only work when webrat is disabled. I would like to know if there is any way to specify a certain test file to use only capybara constructs and exclude webrat from it.

David Chelimsky
Owner

@prusswan there has been no progress on this issue since last august, but there are plenty of suggestions in this thread about how you can get around it.

Steve Brown

Hey there. Short-term, it would be great to get something about this into the documentation. In particular, point by @josevalim about mixing rack-test/rails/webrat's methods with Capybara's:

it "should be created with success" do
  get posts_path
  click_link "New Post"

  fill_in "Title", :with => "My first post"
  fill_in "Body",  :with => "Is quite short"
  click_button "Create Post"

  response.status.should be(200)
end
Andy Lindeman
Owner

I believe this has been addressed with capybara 2.0 (yet to be released) and rspec-rails 2.11.4. See https://github.com/rspec/rspec-rails/blob/master/Capybara.md for more details.

Andy Lindeman alindeman closed this October 26, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.