I did some research into the use of Cucumber by the programmers around me and I looked at six open source Rails projects on github. In nearly everything I saw, programmers wrote 'acceptance tests' almost exclusively with the web_steps or in composite steps built from the web steps. But this defeats the purpose of using Cucumber: such tests are far too low level. In reality they are integration tests. Worse, they are integration tests written in a difficult syntax. Cucumber's advantages vanish in these cases.
According to BDD expert Elizabeth Keogh “If your scenario starts with “When the user enters ‘Smurf’ into ‘Search’ text box…” then that’s far too low-level. However, even “When the user adds ‘Smurf’ to his basket, then goes to the checkout, then pays for the goods” is also too low-level. You’re looking for something like, “When the user buys a Smurf.”
We're all Cuking it wrong, and the problem starts with the bad example set by the web_steps.rb file. I vote to remove it permanently so as to encourage people to write their step definitions in Capybara, saving time and encouraging real acceptance tests over use of Cucumber as an awkward integration test framework.
I've written in depth about it on my blog:Why Bother With Cucumber
This has been discussed before, and we agreed to keep them because we thought it would give people a head start. I see more and more people using bad features resulting from web_steps as a straw man argument (usually unconsciously) against cucumber.
Even in the Cucumber book we tell people to go ahead and delete web_steps.rb.
@mattwynne, @josephwilk, @msassak, @ghnatiuk -what do you think? Yank it for good to save cucumber's reputation?
More and more people writing bad features using web_steps.rb isn't an argument against Cucumber; it's an argument against the use of Cucumber by many people who either find it unnecessary to write acceptance tests or do not understand the point behind them. I can see good reasons for Cucumber, but not when it's used as a wrapper for integration tests.
I haven't read the Cucumber book yet. My main exposure to Cucumber was through the blogosphere, the Rspec book and through two years of full time usage. Web_steps.rb is everywhere.
As much as I dislike web_steps.rb (and have argued that we should remove them), I'm afraid removing them now is only going to upset users. Hell, I thought we could remove the steps method from the world, but lo and behold QA people seem to love it. I think the situation will improve as the project evolves better ways to map from Gherkin to code and as more people realize when and how to use Cucumber, but in the meantime, if you can't be bothered to learn how to use a tool you've selected, then you deserve all the scorn you get. There are still people, I imagine, who see no problem with putting business logic in their views. (And the same applies to elite haxors who attack Cucumber based on the impressions they gather from observing its uninformed use.) Giving a customer scenarios when all they want is a website is dumb. You can't delete dumb from a git repository.
Edit: By elite haxors I don't mean you, @jackkinsella. I mean people who talk trash and don't try to do anything about it or engage with the community at all. I don't think that's what you're doing here.
+1 for killing it.
web_steps is simultaneously an example of how to write step definitions and how to not write step definitions. It is really confusing.
I understand that some example step definitions are really handy for beginners though.
We could include an example_steps.rb file exposing some fictional domain-specific steps. It does not even need to actually run but it can both display how to phrase steps and how to interact with Capybara & friends. That would be a nice way of communicating more about the importance of declarative steps and not pollute the beginner's mind with a too imperative style.
@msassak Can't web_steps stay alive somewhere else for people already using it? We are not going to remove it from existing projects, right? Removing steps is not something that could be reverted by the users. I can imagine how some of them are afraid of losing it for good, even if we consider it to be bad.
Of course it's going to upset users. But it's for their own good :-)
People might read the warning comments, but then they are too lazy to actually delete the stepdefs.
By deleting them we make step_defs an opt-in practice instead of an opt_out one. (I expect someone will fork the code and maintain it somewhere else).
I just emailed the list...
+1 for making it opt-in rather than opt-out. Can we add a generator to cucumber-rails for the web steps?
I was at one point all for just removing the web steps (and other parts of Cucumber that encourage bad practices), but the experience of trying to convince people not to use the steps method and write higher-level scenarios has made me consider the fact that there are at least two distinct kinds of Cucumber users: developers and QAs. The former really dislike the macro-style, but the latter are attracted to Cucumber because of the simplified model of programming it presents. I think the developer workflow is far and away a better way of using Cucumber, but I think we should keep that other type of user in mind. web_steps.rb is really useful for them.
I expect someone who still thinks canned stepdefs is a good idea will make a gem/generator for this. Since I'm not one of those people it will have to be outside the Cucumber organisation's git repos ;-)
I don't think web_steps.rb is useful to anyone - it just appears to be. If there still is demand for it, someone else will satisfy that demand.
+1 for getting rid of them. paths.rb as well?
yeah, paths.rb as well :-) the make no sense without web_steps.rb
Another angle on this is that web_steps.rb actually has no dependency on rails. It could be useful to a team who are using capybara outside of Rails.
If it needs a new home, it would be better moved out to being part of the capybara project, IMO, same way as @bmabey's email_spec supplies it's own step defs. I'm going to be surprised if @jnicklas will want to catch this ball though ;)
Fixed by f027440 - released 1.1.0.
@jackkinsella thanks for bringing this up again - hopefully this will nudge people to write more valuable scenarios (and weed out the people who are too lazy to learn)
Wow excellent :-)
Haha, excellent that this was removed. Good move, long overdue, imo. And no, I'm not going to maintain them in Capybara, I think they are useless.
@aslakhellesoy I think there's a fine line between people who are too lazy to learn, and those that are overwhelmed by the volume of learning necessary to even start, let alone start doing things the "right way" from the beginning.
-1. The web steps are a huge help, especially when combined with support/selectors.rb and support/paths.rb. One of the constant complaints I hear about cucumber is that it's too open ended, and at least this gives a good example of how to write steps and how to use it easily with your app.
TL;DR: I think this is going to do nothing but hurt uptake of Cucumber; at the price of the "power users" feeling better since they don't have to deal with tests "written poorly" from web_steps.rb.
I realize it's over and done, but I also want to voice my -1. I always thought of web_steps.rb as great building blocks for higher level steps, and at worst a great set of references/examples for using capybara.
I don't think web_steps.rb is useful to anyone.
I don't think web_steps.rb is useful to anyone.
I wholeheartedly disagree. I think web_steps are very useful and I'm really disappointed that they'd be ripped out. There's a large (albeit apparently not a majority) camp that uses web_steps.rb regularly and successfully, myself included. I write my cukes with my "dumbest" user in mind and when it comes to walking that user through a particular feature, these building-block steps are vital. This baby-step approach also helps produce meaningful documentation for exactly how a feature does and doesn't work.
It's not because I'm lazy or because I'm not willing to learn the "right way." I apparently just disagree with the majority's right way. When people use your product and have success in a manner you wouldn't expect, I don't think the proper response is to take that out from under them "for their own good."
Oh a fight! I'm thrilled the web_steps.rb are gone; I haven't been using them lately and it's honestly much better.
I wouldn't have minded turning them into mixed-in methods (or full classes) that I can optionally include in my env.rb and use from my custom steps, but this and capybara will suffice.
Let's all also keep in mind that the web_steps were in horrible shape.
The code was simply not idiomatic, and they even encouraged bad
practices (though that's gotten a bit better).
I'm glad they're gone.
These kind of tests can be done in capybara/rspec with a lot less pain & risk of building up crazy-complex and brittle stepdef "libraries".
It's been an interesting learning experience to port tests to that & start over with cucumber, aiming to get the granularity right rather than for test coverage.
I agree with @laserlemon and @msassak in that I find uses for web-steps. I have had the following scenarios in old-school Enterprisey projects.
"We need a full suite of test scripts for the system that can be checked / run by a QA group / clients manually if need be". Now in the old days that mean some-one firing up "shudder" excel or word and writing tedious "Click on this, read that" steps while the devs had their (hopefully) automatic unit tests.
Using cucumber and web_steps and paths I can write these detailed, walk-through scenarios in a relatively natural language which can be read by a human if need be and I can run them automatically myself. This is much better than the bad old days of some poor shmuck in QA having to run the tests manually all the time.
I am guessing that most cucumber users don't have to deal with this kind of corporate... situation and therefore don't see the benefit of these two files.
I am sure that for some clients an acceptance test with steps like "When the user buys a Smurf." is good enough but for others this level of ambiguity is not good enough and I cannot say "Oh, all the details about how you buy a smurf it written in a programming language that a normal user cannot read, sorry."
+1 to the removal. Written about this here but TL;DR: UI changes a lot. Business rules vary less frequently. Features are hard to change (you need to talk to the stakeholder ideally), ruby code is easy to change. Put UI in Ruby code where lots of changes can be handled well. Document and collaborate on the business rules in your scenarios, not how to test the UI.
Besides, Capybara's API is idiomatic and easy to learn: why not use it?
Does anyone have a good document/blog post ready to help developers who suddenly find themselves without these things. To re-educate people on the right way of writing the features. I have the Cucumber book, so I'm not personally bothered (although I do probably still write them in a bad way), but to give the only answer as "buy this book" would a bit harsh to most people.
For me as a beginner in TDD, this discussion is very interesting. I've read the RSpec Book (and I already ordered the Cucumber Book), but it's a lot of new information and a bit hard to use the right way without having much experience. So I highly appreciate your request for a good document/blog, @andyjeffries, and your link to such one, @aslakhellesoy. Keep up the good work, guys!