Support Scenario Outlines

[alisterscott]

Scenario Outlines are not currently supported.

For example

  Scenario Outline: Search for Watir
    Given I am on the <search engine> Home Page
    When I search for "Watir"
    Then I should see at least <expected number of> results
    Scenarios:
      | search engine | expected number of  |
      | Google        | 100,000             |
      | Bing          |  30,000             |

This gives:

Scenario: Search for Watir
    ?  Given I am on the <search engine> Home Page                  
    ~  When I search for "Watir"                                    
    ~  Then I should see at least <expected number of> results      

        Step 'I am on the <search engine> Home Page' not found

        You can define it with: 

          Given  'I am on the <search engine> Home Page' do
            raise 'step not implemented'
          end

even though I created the valid step, it doesn't do the substitution of the value.

[josepjaume]

Hi Alister,

We haven't (still) implemented scenario outlines for a reason: They usually are a "test" smell. We've found out that if you try to be as abstract as possible in your feature definitions, there's barely no situation where scenario outlines are needed. If you're using them, you're probably testing something that should be on a unit test.

For example, I've found myself testing the same scenario against different user roles using scenario outlines and that's wrong. Unit tests should guarantee that, and features should always run on the happy path.

Also, scenario outlines trend to be less readable than standard feature definitions. In addition, there's a principle we wanted to encourage with spinach: "When you write a feature, you shouldn't be thinking of any semantics more than the language in which you're writing them". Having to think about <blah> and putting them just below your scenario feels kinda unnatural.

For example, in your case I would better write two scenarios for that instead of an outline.

  Scenario: Search for Watir on Google
    Given I search for "Watir" on google
    Then I should see at least 100,000 results

  Scenario: Search for Watir on Bing
    Given I search for "Watir" on Bing
    Then I should see at least 30,000 results

So here's why we haven't implemented scenario outlines to the moment. If you can prove us wrong, please feel free to troll us :D

What do you think?

[mrcasals]

IMHO and experience, scenario outlines are a pain in the ass to debug. Imagine you have something like:

Scenario: User adds a post
  Given I am logged in as a <user_type>
  And I am at the post creation page
  When I add a post
  Then I should see the post in my post list

Examples:
  | user_type |
  | admin |
  | editor |

So at first you have them both working, but you change something on your code and now the editor, for some reason you don't know, can't add a post. So you try to debug: add a binding.pry or a save_and_open_page in the last step... The step will be executed twice, and there's no way to help it (unless your remove the admin user_type in the examples table, but then the scenario outline is useless.

Even so, when you run cucumber features and one of the outlines fails, you can't know in which step... So a real pain in the ass :)

[divins]

+1 to @mrcasals

[txus]

I think we won't support scenario outlines. I consider them a code smell, since they usually indicate you should be pushing some tests down to lower test layers (functional, unit). Integration tests should be mainly happy path tests.