-
Notifications
You must be signed in to change notification settings - Fork 0
Cucumber Backgrounder
This discussion deals principally with the initial set up and use of cucumber features in a Ruby on Rails (RoR) Project. Discussion of Behaviour Driven (BDD), Test Driven (TDD), and Panic Driven Development (SNAFU) can be found elsewhere. Details regarding installing the Cucumber gem and its recommended support tools for RoR are found at Ruby on Rails.
Given that you have installed the gem called "Cucumber" And you have generated a RoR project called "MyProject" And your session's working directory is called "MyProject" When you run "script/generate cucumber" Then you will create a sub-directory called "features"
The foregoing is written in the style of the feature statements that make up the user interface to cucumber testing. We will return to that topic later. For the moment we deal with the logical arrangement of cucumber files within the context of a RoR project.
The root level of the archetypal RoR project directory tree looks like this:
MyProject
|-- README
|-- Rakefile
|-- app
|-- config
|-- db
|-- doc
|-- lib
|-- log
|-- public
|-- script
|-- test
|-- tmp
`-- vendor
Running script/generate cucumber adds this layout to the existing structure:
|-- features
| |-- step_definitions
| | `-- webrat_steps.rb
| `-- support
| `-- env.rb
We are now ready to begin testing with cucumber.
Cucumber divides testing into two parts, the outward facing features and the inward facing steps. As discussed elsewhere, Feature Introduction are descriptions of desired outcomes (Then) under predefined conditions (Given) and following upon specific events (When). They are typically used in conjunction with end-user input, and in some cases, may be entirely under end-user (in the form of a domain expert) control. Feature files are given the extension .feature.
A source of potential confusion is that the term steps has two, closely related but vitally distinct, meanings depending on context. Inside .feature files, steps are the textual descriptions which form a scenario. These are prefaced with Given, When, Then, and And (note as well that the capitalization of these four keywords/method names is significant). Inside a step_definitions.rb file, steps (which strictly speaking should always be called step definitions) refers to the matcher methods, given exactly the same names (Given, When, Then, or And), each provided with a matcher regexp that corresponds to one or more feature steps.
Step Organisation, keyed by their snippets of text from the feature files, invoke blocks of ruby and rails code that usually contains assertion statements from whatever test system you have installed. Given that cucumber evolved out of RSpec stories it is assumed in the cucumber generator that rspec is available, as is evidenced in the default contents of features/support/env.rb:
# Sets up the Rails environment for Cucumber
ENV["RAILS_ENV"] = "test"
require File.expand_path(File.dirname(__FILE__) + '/../../config/environment')
require 'cucumber/rails/world'
Cucumber::Rails.use_transactional_fixtures
# Comment out the next line if you're not using
# RSpec's matchers (should / should_not) in your steps.
require 'cucumber/rails/rspec'
You are not limited to this testing framework however. If you choose to use Rails built-in TestUnit, or any other testing suite, then you may do that by extending the cucumber working environment as explained in Using Test::Unit.
As detailed above, the default features directory tree is fairly shallow. One can put every feature into a single file in the features directory and every step in a single file in the steps_definition directory (or even in the features directory itself) if one so chooses. One can choose to have one or more feature files for each feature together with one or more step files for each feature file; or any combination thereof. However, cucumber is programmed with the flexibility to support a much more expressive directory structure. For instance:
|-- features
| |-- entities
| | |-- entity.feature
| | `-- step_definitions
| | |-- anything.rb
| | `-- entity_steps.rb
| |-- locations
| | |-- location.feature
| | `-- step_definitions
| | `-- location_steps.rb
| |-- sites
| | `-- step_definitions
| |-- step_definitions
| | `-- webrat_steps.rb
| `-- support
| `-- env.rb
In this case the bland initial setup has been divided into sub-directories informed by model-centric testing. However this could equally well have been broken up in to model/controller/view hierarchies:
|-- features
| |-- models
| | |-- entities
| | | |-- entity.feature
| | | `-- step_definitions
| | | |-- anything.rb
| | | `-- entity_steps.rb
| |-- views
| | | |-- entity_new
| | | `-- step_definitions
| | | `-- entity_new_steps.rb
| |-- step_definitions
| | `-- webrat_steps.rb
| `-- support
| `-- env.rb
Consider, however, that the cucumber feature generator will generate the files it produces in conformance with the default layout, with the manage_frooble.feature file placed in ./features and the frooble_steps.rb file in ./features/step_definitions. Also be aware that, regardless of the directory structure employed, cucumber effectively flattens the features directory tree when running tests. By this I mean that anything ending in .rb under the start point for a cucumber feature run is searched for feature matches. So that a step contained in features/models/entities/step_definitions/anything.rb can be used in a feature file contained in features/views/entity_new. It is also worth noting that step files can be called anything so long as they end in .rb.
Constructing ones first tests, or features as BDD purists prefer, is often accompanied by what can only be described as writer’s block. While detailed discussion of feature writing and step construction are provided elsewhere (Given-When-Then), perhaps the easiest thing to do for the first time tester/behaviourist is to use cucumber’s built-in generator to create a feature scaffold and then modify it.
script/generate feature Frooble name color description
exists features/step_definitions
create features/manage_froobles.feature
create features/step_definitions/frooble_steps.rb
A Feature is typically contained in its own file (ending in
.feature). Each Feature consists of multiple Scenarios. Each Scenario consists of three sections, Given, When and Then. Each section consists of one or more clauses that are used to match test steps. The conventional arrangement is:
Feature: Some terse yet descriptive text
In order to/that some business value is realized
A some actor
Should obtain some beneficial outcome which furthers the goal
To Increase Revenue | Reduce Costs | Protect Revenue (pick one)
Scenario: Some determinable business situation
Given some condition to meet
And some other condition to meet
When some action by the actor
And some other action
And yet another action
Then some testable outcome is achieved
And something else we can check happens too
Scenario: A different situation
...
For cucumber the key words use here are Feature, Scenario, Given, When, Then, and And. Feature is used to provide identification of the test group when results are reported. Scenario is used in the same fashion. Given, When, Then, and And are cucumber methods that take as their argument the string that follows. These are the steps that cucumber will report as passing, failing or pending based on the results of the corresponding step matchers in the step_definitions files. These methods are also all equivalent:
def Given(name)
create_step('Given', name, *caller[0].split(':')[1].to_i)
end
def When(name)
create_step(‘When’, name, *caller0.split(‘:’)1.to_i)
end
def Then(name)
create_step(‘Then’, name, *caller0.split(‘:’)1.to_i)
end
def And(name)
create_step(‘And’, name, *caller0.split(‘:’)1.to_i)
endThe string from each one of these method is compared against the matchers contained in the step_definitions.rb files. A step matcher looks much like this:
Given /there are (\d+) froobles/ do |n|
Frooble.transaction do
Frooble.destroy_all
n.to_i.times do |n|
Frooble.create! :name => "Frooble #{n}"
end
end
end
The significant thing here is that the method (Given) takes as its argument a regexp bounded by /. For example, in the features given above we had the statement: And some other action. This could be matched by any of the following step matchers in any step_definitions.rb file found under the features directory.
Given /some other action/ do
When /some (.*) action/ do |match|
Then /(*.) other action/ do |match|
And /some other act.*/ do
Given /(.*) other (.*)/ do |first,second|
Which raise an interesting point. If you had all of these matchers defined somewhere in step_definitions.rb files then cucumber would tell you that it found multiple matches for a single feature step and force you to distinguish them.
Note that you are not required to use a matched subpattern, (.* ), in the following block.
It is considered better form by some to surround with double quotation marks, ", elements in the feature step clauses that are meant to be taken as values. This is only a convention and one not followed by many. However, if you choose to follow this road then you must adjust your step files accordingly. For example:
Given Some determinable "business" situationGiven /determinable "(.*)" situation/ do |s|Finally, you can have steps call steps. For example:
And /some "(.*)" action/ do |a|
...
end
When /in an invoiced non-shipped situation/ do
Given "some \"invoiced\" action"
Then "some \"non-shipped\" action"
...
end
Worthy of note is that, inside the step_definitions.rb files, one must enclose the arguments to other Given/When/Then/And methods with a string delimiter. Because of this, if you have adopted the practice of demarcating parameter values present in the features.feature file steps with double quote marks, then you must escape these parameters when calling another definition_steps.rb matcher from inside the step_definitions.rb files. You should also be cautious not to include the quote marks in the step_definitions parameter matchers, for “(.)” is not the same as (.*). If you use quotes in the .feature file steps and do not account for them in your corresponding step_definition.rb matcher methods then you will obtain variables that contain leading and trailing quotes as part of their values.
Scenario: Quotes surround value elements
Given some "required" action
# step_definitions
And /some (.*) action/ do |a|
a => "required"
And /some "(.*)" action/ do |a|
a => required
Begin with
rake db:migrate
rake db:test:prepare
Testing with cucumber can be performed in several fashions. Be aware that Rake features, cucumber features, and autotest with ENV AUTOFEATURE=true do not necessarily produce the same results given the same tests.
Running rake features from the command line provides the easiest and most reliable method to run cucumber tests. The rake script provided with cucumber performs much of the background magic required to get the test database and requisite libraries properly loaded.
Running cucumber directly from the command line requires the creation of a cucumber.yml file in the project root directory, even if that file remains empty. When cucumber is run from the command line it is usually necessary to provide both the directory name containing the root directory of the tree containing feature files and the directory name containing references to the necessary library files. In the typical project cucumber features -r features will normally suffice.
Finally, running autotest with the environment variable AUTOFEATURE=true will run ALL tests, including those in /test and (if present) /spec. As this will load all test fixtures it may leave the test database in an indefinite state when cucumber features are run. It is wise, as always, to write cucumber steps so that they do not depend upon an empty database or themselves place the database in the requisite state.
For those of you that have used growl or snarl to provide desktop notifiers from autotest be advised that as of this writing cucumber did not hook into the :red :green notifier capability of autotest. So, no popups when a step fails. However, ther exisits a project to add this to cucumber. See Cucumber_Growler.
Recall that cucumber is an integration test harness. It is designed to exercise the entire application stack from view down to the database. It is certainly possible to write features and construct steps in such a fashion as to conduct unit tests. No great harm will result. However, this is not considered best practice by many and may prove impossible for very large projects due to the system overhead it causes and the commensurate delay in obtaining test results.
Cucumber comes preconfigured with webrat test steps. If webrat is installed as a plugin in your project then these are available to you for view tests “out of the box”. If, instead, you have webrat installed as a gem and you wish to use the gem to perform the tests then you need add this to the features/support/env.rb file:
require 'webrat' if !defined?(Webrat)
The best place to go for help, that I know of, is the RSpec Forum at Ruby-Forum.com. There you can find access to a number of forums and gateways to mailing lists that are related to Ruby and Ruby on Rails.
2008 November 28 – J. B. Byrne