Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PLANET-5167: Introduction of features descriptions as tests #81

Merged
merged 1 commit into from Jul 15, 2020

Conversation

lithrel
Copy link
Member

@lithrel lithrel commented Jun 30, 2020

Refs:

Follow-up of PoC: PoC for BDD tests #77

A short reminder about Gherkin

This PR implements tests in a Gherkin format, run by Codeception.
By implementing reusable sentences and making available a documented list of those, we can try to reach a point where a tester (non-coder) writes a feature file that is playable as is, without any technical intervention. Even if this goal is not fully reached, this might greatly help with tests readability and organization for the whole team.

Tests look like this (keywords hilighted in red, parameters quoted):

Scenario: Create a new post
    Given I am on a new post page
    When I add a title "Test title"
    And I publish the post
    Then a message "Post published." is displayed

Implementation looks like this (sentences are matched by annotations @Given, @When, @Then):

    /**
     * @example I add a title "My title !"
     * 
     * @When I add a title :arg1
     */
    public function iAddATitle(string $title): void
    {
        $I = $this->tester;
        $I->fillField(EditorSelector::TITLE_FIELD(), $title);
    }

What changed since #77

  • Selectors moved into constants. I'm using myclabs/phpenum for this:
    • it makes it easy to add variable parts to those strings, as everything is called like a static method
    • it gives some helpers to list all keys and values
    • it allows to type constants, making it possible to lock usage of some functions to only a specific type of constant, and fail faster and with a more obvious error message (for example, GutenbergEditor::addBlock can only take BlockSection and BlockName as params and will say so if failing, which is more explicit than failing because the selector was not found in the page)
  • Method names follow more closely the sentence they describe. I'm not completely satisfied by what codeception generates, but we can also make our own name generator if we feel the need to.
  • Steps (methods described with Given When and Then) were moved to Step\ namespace
  • Page\ namespace is used for interactions with pages that are not steps but that can be used by steps
    • In those interactions, we can more tightly limit parameter types and make tests wrongly implemented fail faster
  • 2 commands were added to facilitate development and sharing of vocabulary

Commands

p4:selectors

  • lists the available selectors, ordered by class. Class name should be explicit enough. Classes currently have to be added to the command code, we should add a file scan later if needed.
  • variable parts of the selectors are highlighted.

Screenshot 2020-06-29 at 14 49 31

p4:steps <suite> [-i]

  • lists the available steps, with examples if possible (examples are method annotations @example)
  • sorted by Step files, with a short description (description taken from class comments)
  • option -i displays the class::method implementing the sentence

Screenshot 2020-06-29 at 14 49 57

Test

Tests passed in the CI :https://app.circleci.com/pipelines/github/greenpeace/planet4-base-fork/452/workflows/8e5b6c62-e0c4-437e-a4f5-5217fea3dac0/jobs/5075/steps.
Run it locally with tests/vendor/bin/codecept run -vv acceptance editor.feature

Review guide

  • tests/__support/Command
    The two commands added, started by copying Codeception\Command\GherkinSteps
  • tests/__support/Selector
    All selectors, grouped by zone as Enum
  • tests/__support/Step
    All steps, grouped by topic/zone
  • tests/__support/Page
    Pages interactions not declared as steps (for internal use, type hinting, etc.)
  • Added commands to codeception.yml, added steps to acceptance.suite.yml

Next

  • In time we should automate an export of p4:steps to make it always available and up to date for test writers.
  • Add string interpolation to constants instead of just %s
  • Some scenarios need context to continue (a post id just created for example), find a clean way to handle that. (It is in part why there is no public site check at the moment, I'm waiting to have a few more tests to see how to group that)
  • Tests are slow, some scenarios should be grouped by chaining when-then in the same scenario
  • Commands code might deserve a bit of cleanup
  • Still did not solve this copy/paste problem (not a priority)

@lithrel lithrel added the WIP Work in progress label Jun 30, 2020
@lithrel lithrel force-pushed the planet-5167-bdd branch 2 times, most recently from f4cf5c2 to 31260dc Compare June 30, 2020 14:24
@lithrel lithrel added Review and removed WIP Work in progress labels Jul 1, 2020
@lithrel lithrel self-assigned this Jul 9, 2020
Copy link
Member

@comzeradd comzeradd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work 💯

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
2 participants