New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gherkin Format Support #2750

Merged
merged 87 commits into from Feb 10, 2016

Conversation

Projects
None yet
7 participants
@DavertMik
Member

DavertMik commented Jan 29, 2016

For someone this is long awaited feature, for others an unexpected one.
We bring Gherkin support to Codeception 2.2. This means that you can run your BDD-scenarios as tests.

In Codeception we were targeted developers audience so instead of Gherkin syntax we had PHP DSL. However, it is good to describe tests, and not for actual features. But what if we can mix flexibility and simplicity of PHP DSL for describing tests and Gherkin for features narrative.

So if you expect that bringing Gherkin format to Codeception is just about writing tests in this kind of way:

Given I am on page "/"
When I click "Login"
And I fill field "Username" with "davert"
And I fill field "Password" with "123456"
And I click "Login"
Then I should see "Welcome, Davert"

Well, nope, this won't work in Codeception. At least by default. We expect that BDD will be used in its right way for describing a business rules, and not on testing UI.

So here goes the idea: you describe business specification in Gherkin format, then you implement steps using Codeception's $I (actor) object. So the example above should be reworked to something like that:

Inside a feature file:

Given I have account for "davert"
When I log in
Then I should see "Welcome, Davert" notification

Inside an actor file (AcceptanceTester) or a context file (see below):

<?php
class AcceptanceTester extends \Codeception\Actor 
{
  use _generated\AcceptanceTesterActions;

  protected $userName;
  protected $password = '123456';
  /**
   * @Given I have account for :name
   */
  public function prepareUser($name)
  {
      $this->userName = $name;
      // let's assume user is already in database ;)
  }  

  /**
   * @When I log in
   */
  public function logIn() 
  {
    $I = $this;
    $I->amOnPage('/');
    $I->fillField('Username', $this->userName);
    $I->fillField('Password', $this->password); // standard password
    $I->click('Login', '#login-form'); // let's use UI-specific locators
  }

  /**
   * @Then I should see :text notification
   */
  public function seeOnPage($text)
  {
    // we just wrap standard action into a feature step
    $this->see($text, '#notification');
  }
}
?>

As always, step by step execution is shown when running with --steps option:

 Given I have account for "davert" 
 When I log in
  I am on page '/'
  I fill field 'Username', 'davert'
  I fill field 'Password', '123456'
  I click 'Login', '#login-form'
 Then I should see "Welcome, Davert" notification
  I see 'Welcome, Davert', '#notification'

With @Given, @When or @Then annotation any method of actor object can become a feature step.
You can use either string with placeholders or regex for a step definition.

Here is another example. We can use tables to define multiline data. Supported in Codeception.

Feature: ToDo items
  In order to get my tasks done
  As a motivated person
  I need to be able to organize my todo list

  Scenario: show current todos
    Given there are todo items:
      | Task             | State   |
      | make Gherkin PR  | opened  |
      | update docs      | opened  |
      | create examples  | closed  |
    When I open my todos
    Then I should see 2 todo items

Features

  • executing tests from *.features file
  • defining feature steps in actor file
  • regex step definitions
  • placeholder step definitions
  • tables
  • gherkin tags => Codeception groups
  • multiple contexts
  • select context for a tag or role
  • list step definitions in console
  • generate snippets for undefined steps
  • describe features in other languages
  • gherkin examples
  • ???

Implementation Details

(for core team)

This PR includes lots of non-obvious changes. The overall Gherkin implementation requires "behat/gherkin" library and 2 files: Codeception\Test\Gherkin and its loader Codeception\Test\Loader\Gherkin.

However, lot of work was done to improve console output so all test formats would look nicely.
Also all improvements to output was also added to this branch (and not to 2.1 or master) for not to get into merge conflicts.

Step, and MetaStep classes were changed to handle feature steps of Gherkin.

Roadmap

This feature is about to land into 2.2. A new guide on Behavior-Driven Development with Codeception is coming by that time.

Usage

Start creating first features:

codecept g:gherkin <suite> <name>

To fetches empty steps from feature files of suite and print (generate) code snippets

codecept gherkin:snippets

To print all defined feature steps:

codecept gherkin:steps

P.S. Appveyor tests may fail due to their technical issues

DavertMik added some commits Oct 1, 2015

refactored, Cest and Cept files to extend lightweight Codeception\Lib…
…\Test class instead of PHPUnit_Framework_TestCase
@jadb

This comment has been minimized.

Show comment
Hide comment
@jadb

jadb Jan 30, 2016

Just saying thanks! 👍

jadb commented Jan 30, 2016

Just saying thanks! 👍

DavertMik added some commits Feb 1, 2016

@DavertMik

This comment has been minimized.

Show comment
Hide comment
@DavertMik

DavertMik Feb 3, 2016

Member

Just about to get it merged today. Last chance to review )

Member

DavertMik commented Feb 3, 2016

Just about to get it merged today. Last chance to review )

Show outdated Hide outdated codeception.yml
@@ -31,6 +31,7 @@
"symfony/dom-crawler": ">=2.4|<3.1"
},
"require-dev": {
"behat/gherkin": "~4.4.0",

This comment has been minimized.

@Naktibalda

Naktibalda Feb 3, 2016

Member

why is it in require-dev?

@Naktibalda

Naktibalda Feb 3, 2016

Member

why is it in require-dev?

This comment has been minimized.

@Naktibalda

Naktibalda Feb 3, 2016

Member

Add behat/gherkin to suggest if you don't want to add it to require.

@Naktibalda

Naktibalda Feb 3, 2016

Member

Add behat/gherkin to suggest if you don't want to add it to require.

if ($this->isDetailed($test)) {
return;
}
if ($this->output->isInteractive()) {
$this->output->write("\x0D");

This comment has been minimized.

@Naktibalda

Naktibalda Feb 3, 2016

Member

I had to lookup this one.
Why do you print \r here?

@Naktibalda

Naktibalda Feb 3, 2016

Member

I had to lookup this one.
Why do you print \r here?

This comment has been minimized.

@DavertMik

DavertMik Feb 3, 2016

Member

Let me explain. In interactive mode we can control the caret on a line. For a test we can print currently started test:

- CurrentTest

and when test finishes we replace that line with result + test name + time:

✓CurrentTest (0.1s) 

so we use \r character to print that line on top of previous one

@DavertMik

DavertMik Feb 3, 2016

Member

Let me explain. In interactive mode we can control the caret on a line. For a test we can print currently started test:

- CurrentTest

and when test finishes we replace that line with result + test name + time:

✓CurrentTest (0.1s) 

so we use \r character to print that line on top of previous one

@@ -3,6 +3,7 @@
$I->wantTo('run skipped test');
$I->amInPath('tests/data/sandbox');
$I->executeCommand('run skipped SkipMeCept.php');
$I->seeShellOutputMatches("~\(SkipMeCept\)[\s\.]*?Skipped~");
$I->seeInShellOutput("S Skip it");

This comment has been minimized.

@Naktibalda

Naktibalda Feb 3, 2016

Member

Shouldn't it be "I skip it"?

@Naktibalda

Naktibalda Feb 3, 2016

Member

Shouldn't it be "I skip it"?

This comment has been minimized.

@DavertMik

DavertMik Feb 3, 2016

Member

No, S is a status character, first in line

@DavertMik

DavertMik Feb 3, 2016

Member

No, S is a status character, first in line

{
$this->seeFileFound('c3.php');
$this->seeFileFound('composer.json');
$this->seeInThisFile('codeception/c3');
}
/**
* @Given I have terminal opened

This comment has been minimized.

@Naktibalda

Naktibalda Feb 3, 2016

Member

Is it mandatory to have a Given line in the feature file?
This one is so useless that it would be better to skip it.

@Naktibalda

Naktibalda Feb 3, 2016

Member

Is it mandatory to have a Given line in the feature file?
This one is so useless that it would be better to skip it.

This comment has been minimized.

@DavertMik

DavertMik Feb 3, 2016

Member

there is no real difference between @Given, @When and @Then but they should be present in annotation to become a gherkin step

@DavertMik

DavertMik Feb 3, 2016

Member

there is no real difference between @Given, @When and @Then but they should be present in annotation to become a gherkin step

{
$pattern = 'hello :name, are you from :place?';
$regex = $this->loader->makePlaceholderPattern($pattern);
$this->assertEquals('/hello "(.*?)", are you from "(.*?)"\?/', $regex);

This comment has been minimized.

@Naktibalda

Naktibalda Feb 3, 2016

Member

Why are the quotes added around placeholders?

@Naktibalda

Naktibalda Feb 3, 2016

Member

Why are the quotes added around placeholders?

This comment has been minimized.

@DavertMik

DavertMik Feb 3, 2016

Member

there are 2 step formats: regular string with placeholders and regex
so yes, we convert placeholders to a compatible regex.
In regex string we use quotes to highlight the variables.

@DavertMik

DavertMik Feb 3, 2016

Member

there are 2 step formats: regular string with placeholders and regex
so yes, we convert placeholders to a compatible regex.
In regex string we use quotes to highlight the variables.

@@ -0,0 +1,24 @@
<?php
class GherkinCest

This comment has been minimized.

@Nitpick-CI

Nitpick-CI Feb 3, 2016

Each class must be in a namespace of at least one level (a top-level vendor name)

@Nitpick-CI

Nitpick-CI Feb 3, 2016

Each class must be in a namespace of at least one level (a top-level vendor name)

@micaweb

This comment has been minimized.

Show comment
Hide comment
@micaweb

micaweb Feb 4, 2016

This is awesome Mr. Davert! I have like forever that 'Herkin language to Gherkin' gist bookmarked. Thank you so much!

micaweb commented Feb 4, 2016

This is awesome Mr. Davert! I have like forever that 'Herkin language to Gherkin' gist bookmarked. Thank you so much!

@@ -0,0 +1,24 @@
<?php
class GherkinCest

This comment has been minimized.

@Nitpick-CI

Nitpick-CI Feb 4, 2016

Each class must be in a namespace of at least one level (a top-level vendor name)

@Nitpick-CI

Nitpick-CI Feb 4, 2016

Each class must be in a namespace of at least one level (a top-level vendor name)

DavertMik added some commits Feb 5, 2016

@@ -0,0 +1,24 @@
<?php
class GherkinCest

This comment has been minimized.

@Nitpick-CI

Nitpick-CI Feb 9, 2016

Each class must be in a namespace of at least one level (a top-level vendor name)

@Nitpick-CI

Nitpick-CI Feb 9, 2016

Each class must be in a namespace of at least one level (a top-level vendor name)

@@ -0,0 +1,24 @@
<?php
class GherkinCest

This comment has been minimized.

@Nitpick-CI

Nitpick-CI Feb 10, 2016

Each class must be in a namespace of at least one level (a top-level vendor name)

@Nitpick-CI

Nitpick-CI Feb 10, 2016

Each class must be in a namespace of at least one level (a top-level vendor name)

@@ -0,0 +1,24 @@
<?php
class GherkinCest

This comment has been minimized.

@Nitpick-CI

Nitpick-CI Feb 10, 2016

Each class must be in a namespace of at least one level (a top-level vendor name)

@Nitpick-CI

Nitpick-CI Feb 10, 2016

Each class must be in a namespace of at least one level (a top-level vendor name)

DavertMik added a commit that referenced this pull request Feb 10, 2016

@DavertMik DavertMik merged commit a2d3855 into master Feb 10, 2016

3 of 5 checks passed

continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
continuous-integration/travis-ci/push The Travis CI build is in progress
Details
continuous-integration/appveyor/branch AppVeyor build succeeded
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
semaphoreci The build passed on Semaphore.
Details

@DavertMik DavertMik deleted the gherkin branch Mar 26, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment