Best Practices

Best practices

As in each framework, things in Bobcat also can be done in many ways. This page will guide you about most common mistakes and tell you how to develop good, readable and reliable tests.

Do not write assertions inside page objects

This is not proper way to indicate whether some test case should pass or not as it's not page object's responsibility. Page object should describe only a contract between a user and the page. It should be a tool to control it and to use it. When we are talking about test cases and generally about testing we should base on Gherkin features. Gherkin feature is a file that describes a functionality to be tested. Features consist of one or more scenarios. Each scenario has it's steps which can pass or fail. Each step is backed up by it's implementation that we should create. Let's take a look at the example. The following scenario is checking if the user is getting error message after providing wrong credentials:

@serviceLogin
Feature: Login
...
  Scenario: Fail to login with invalid credentials
    Given I have opened login page
    And I am not logged in
    When I enter following credentials "invalid", "user"
    And I press login button
    Then Authorization error message should appear
...

For more information about Gherkin language and syntax please visit Cucumber documentation. Since we have our steps defined in the feature file - we can implement them:

import com.google.inject.Inject;
import cucumber.api.java.en.Given;
import cucumber.api.java.en.Then;
import cucumber.api.java.en.When;
import cucumber.runtime.java.guice.ScenarioScoped;
...
@ScenarioScoped
public class LoginPageSteps {

  @Inject
  private LoginPage loginPage;

  ...

  @Given("^I have opened login page$")
  public void iHaveOpenedLoginPage() {
    assertTrue(loginPage.openLoginPage().loginPageIsDisplayed());
  }

  @When("^I enter following credentials \"(.+)\", \"(.+)\"$")
  public void iEnterFollowingCredentials(String login, String password) {
    loginPage.getLoginBox().enterLogin(login).enterPassword(password);
  }

  @When("^I press login button$")
  public void iPressLoginButton() {
    loginPage.getLoginBox().clickSignIn();
  }

  @Then("^Authorization error message should appear$")
  public void errorMessageShouldAppear() {
    assertTrue(loginPage.getLoginBox().isErrorMessageVisible());
  }
  ...
}

As we can see, the loginPage object is used here only to control the login page itself. It does not (and should not) contain any logic regarding our test cases. Instead we should put that logic in our scenario steps implementation and use the Page Object as a tool for controlling a page as the end user would.

But ok - this is not really convenient to create number of methods, each for separate scenario step with much information in annotations.

Agreed! There is a tool that allow you to generate it automatically using feature files!

Generating scenario steps implementation

Creating scenario steps implementation manually is a painful process. To avoid that you can use Cucumber plugin for your IDE that will generate all method stubs for you! Please refer to the documentation of available plugin for:

• IntelliJ IDEA
• Eclipse

Use Cucumber Tags

Cucumber Tags are really useful - you can run or disable specified features and scenarios using these. A tag can be placed on Feature or Scenario level. Let's take a look at the example:

@serviceLogin
Feature: Login
...
  @loginFailScenario	
  Scenario: Fail to login with invalid credentials
    Given I have opened login page
    And I am not logged in
    When I enter following credentials "invalid", "user"
    And I press login button
    Then Authorization error message should appear

As we can see, we have two Cucumber Tags defined here: @serviceLogin which is on the Feature level and @loginFailScenario which is placed on Scenario level. Now we can use it in our Test Runner to configure tags we want to run.

import com.cognifide.qa.bb.cumber.Bobcumber;
import cucumber.api.CucumberOptions;
...
@RunWith(Bobcumber.class)
@CucumberOptions(
    features = "src/main/features/",
    plugin = {"pretty", "html:target/cucumber-html-report/example",
        "json:target/example.json"},
    tags = {"@serviceLogin"},
    glue = "com.cognifide.qa"
)
public class ExampleTest {
  // This class is empty on  purpose - it's only a runner for cucumber tests.
}

This code snippet tells the Test Runner that each item annotated with @serviceLogin should be run. If we want to run only one scenario with login fail, we can provide @loginFailScenario here - then only this scenario will be executed. We could also annotate things with marker annotation such as @disabled. Then in our runner it should be prepended with "~" sign. This tells the runner to skip executions of elements tagged with @disabled annotation. Example:

@serviceLogin
Feature: Login
...
  @disabled	
  Scenario: Fail to login with invalid credentials
    Given I have opened login page
    And I am not logged in
    When I enter following credentials "invalid", "user"
    And I press login button
    Then Authorization error message should appear

...
@RunWith(Bobcumber.class)
@CucumberOptions(
    features = "src/main/features/",
    plugin = {"pretty", "html:target/cucumber-html-report/example",
        "json:target/example.json"},
    tags = {"@serviceLogin", "~@disabled"},
    glue = "com.cognifide.qa"
)
public class ExampleTest {
  // This class is empty on  purpose - it's only a runner for cucumber tests.
}

In this example the entire feature annotated with @serviceLogin will be executed except the login fail scenario which is annotated with @disabled tag.

Ok, but things like that need changes in code. Is there any possibility to manage it dynamically to configure it on demand example on Jenkins?

Of course! You can use Maven parameters to achieve that. For example - if you want to run only Login Fail Scenario you could provide the following parameter:

... -Dcucumber.options="--tags@loginFailScenario"

Tags are not the only parameter that you can provide like that - you can handle all Cucumber options like that: features, glue etc.

Write specific scenario steps

Remember that your scenario steps should be specific. Do not write steps like And I click on the carousel. It can confuse either you or your IDE plugin which is helpful in finding implementation of your steps. When it finds multiple implementation of the same scenario steps it can be really confusing for developers.

Do not use BobcatWait#sleep()

This freezes the entire thread. Whenever possible - use the BobcatWait#withTimeout which is more flexible and works on the WebDriver level.

Use Cucumber transformers

Transformers are really helpful when it comes to improving readability of our scenario steps. Let's take a look at the two example use cases:

Proper handling of URLs

Things like URLs can change really often. Hardcoding them in a scenarios is not a good practice. We can put them in a properties file instead and refer to the property key in a scenario. Let's take a look at the example:

Properties file:

example.product=/some/example/product.html

Feature file:

...
And I am on example.product page
...

Scenario step implementation:

@Given("^I am on ([^\"]*) page$")
 public void I_am_on_page(String pageNameProperty) {
  String path = properties.getProperty(pageNameProperty);
  ...
 }

Ok, but putting some property keys in the scenario steps reduces it's readability and introduces some technical aspects which are not wanted here...

That's of course true and you can use Cucumber's @Transform annotation and implement your PropertyTransformer to get rid of technical stuff.

Example implementation of Property Transformer:

import cucumber.api.Transformer;
 ...
public class PropertyTransformer extends Transformer<String> {
 @Override
 public String transform(String value) {
  return value.trim().toLowerCase().replace(" ", "."); 
 }
}

Now all we need to do is to add proper annotation on a method parameter.

@Given("^I am on ([^\"]*) page$")
public void I_am_on_page(@Transform(PropertyTransformer.class) String pageName) {
 String path = properties.getProperty(pageName);
 ...
}

With such annotated parameter, we could write our scenario step like this:

...
And I am on Example Product page
...

The Property Transformer will transform the "Example Product" string into example.product key by trimming, lowercasing and replacing spaces with periods from the parameter string variable. It can improve scenarios readability a lot!

Using transformers as normalizers

As we all know, computers tend to start counting from zero. This is not natural for humans. Let's say that we have a table that represents bullet list elements, where each one has it's ordinal number. When we start our numbering from one it will be more readable for no-technical people and improve readability of our scenarios. But we have to map these numbers with appropriate indexes of our collection, holding this elements in our code. We can implement a number normalizer to handle it. Example:

@When("^I click \"([^\"]*)\" bullet in a product's options bullet list $")
 public void I_click_bullet_in_carousel(@Transform(NumberNormalizer.class) int bulletNumber) {
  productPage.clickBullet(bulletNumber);
  assertThat("Bullet is not active", productPage.isBulletActive(bulletNumber), is(true));
 }

Where our NumberNormalizer may look like this:

public class NumberNormalizer extends Transformer<Integer> {
 @Override public Integer transform(String value) {
  return Integer.parseInt(value) - 1;
 }
}

These are only two use cases to show you some examples for using Cucumber's @Transform annotation. It can be used in a lot of other use cases though. Bear in mind that readability of your scenarios is really important, and Transformers can be really helpful here.

Use expected conditions

When your page makes an AJAX requests or has some animation you should use ExpectedConditions to wait for them. For example, when clicking on some header on the page, you can wait until page URL contains a # sign.

import com.cognifide.qa.bb.utils.WebElementUtils;
import com.cognifide.qa.bb.expectedconditions.UrlExpectedConditions
...
	@Inject
	private WebElementUtils webElementUtils;
  ...
	@When("^I click \"([^\"]*)\" product header, the page scrolls to the clicked section$")
	 public void I_click_product_header_page_scrolls(String headerName) {
	  productPage.clickHeader(headerName);
	  boolean result = webElementUtils.isConditionMet(UrlExpectedConditions.pageUrlContains("#"), 5);
	  ...
	 }

Put comments in your assertions

Whenever making assertions in your tests - add a proper comment. When something goes wrong it can give you a readable advice what failed. It is better than analyzing an assertion exception stacktrace.

Avoid boolean assertions if possible

What's wrong with boolean assertions? When making a boolean assertion on some object we can loose the information about the difference between them. For example:

@Then("^Search result header is \"([^\"]*)\" in Example Product Page$")
 public void Search_result_header_is_in_Example_Product_Page(String title) {
  assertThat("Expected search result header (" + title + ") does not appear",
    exampleProductPage.isSearchResultTitleEquals(title), is(true));
 }

When this assertion fails, we only get information that assertion should return true but it returned false. It is better to write assertion in a way that will show you the expected value and the actual value present instead of returning logical result only.