This structure is generated when intializing a new test project running the npx etaf install
command. See the
technical documentation for
more details.
conf/ Configuration
| +-- realm/ Realm configuration files
doc/ Documentation
| +-- Framework.md This documentation
log/ Log files (Selenium, browser, etc.)
node_modules/ All dependencies (node modules) installed via npm install
output/ Test execution reports such as JUnit, Allure, etc. reports
| +-- errorShots/ Error screenshots taken when scenarios have failed
src/ All source code
| +-- features/ Scenarios written in Gherkin
| +-- step_definitions/ Translation of the Gherkin scenarios
| +-- support/ Source code implementing step definitions
| +-- business-object/ Implementing business flows
| +-- component-object/ Parts of the page objects that are reused on several pages
| +-- data/ Containing data objects that are data sets for each scenario
| +-- helper/ Any other pieces of code that help make the code more readable
| +-- page-object/ Page objects (see design pattern)
| +-- hooks.js Cucumber.js hooks
| +-- world.js Global libs and helpers available for each scenario execution
test/ Unit tests
.gitignore Files to be ignored by Git (logs, node_modules, etc.)
.jscsrcs Configuration of the JSCS linter
.npmrc npm configuration specific to this project
package.json npm configuration for the project, including dependencies. Used by npm on npm install
README.md Getting started documentation
wdio.conf.js webdriver.io configuration. Used by npx etaf run
wdio.local.conf.js webdriver.io configuration (inherits from wdio.conf.js) to run test locally. Used by npx etaf run wdio.local.conf.js
wdio.debug.conf.js webdriver.io configuration (inherits from wdio.local.conf.js) for debugging purposes. Used by npx etaf run wdio.debug.conf.js
The framework follows the Push how down good practice which consists in pushing the complexity as down as possible in the stack. A good example on how the complexity should be pushed down can be found here.
The different layers are the following (from top to bottom):
Gherkin Scenarios
|
Step Definitions
|
Business Objects
|
Page/Component Objects
Each layer can only interact with the layer right below it:
- Step definitions can only interact with Business objects
- Business objects can only interact with Page/Component objects
Details of each layer are described below.
Gherkin is a Business Readable, Domain Specific Language that lets you describe software’s behaviour without detailing how that behaviour is implemented.
Example:
Feature: Some terse yet descriptive text of what is desired
Textual description of the business value of this feature
Business rules that govern the scope of the feature
Any additional information that will make the feature easier to understand
Scenario: Some determinable business situation
Given some precondition
And some other precondition
When some action by the actor
And some other action
And yet another action
Then some testable outcome is achieved
And something else we can check happens too
Please read the Gherkin documentation to learn the syntax. It is fairly simple to understand and get started.
-
Each scenario should be unitary and therefore should not rely on previously executed scenario(s) to pass.
-
Scenario should remain at a functional/business level, they should not make any reference to the web page:
Bad, using CSS selectors
When I open the login menu And I set #ta-header-username to john.doe@email.com And I set #ta-header-password to mypassword And I click on #ta-header-login-submit
Good
When I login as a customer
Bad, describing web page layout
Then I should see an error message on top of the page stating that the login/password is invalid
Good
Then I should be told that I have provided wrong credentials
-
Make sure to target the feature that you want to test. The
When
steps should only describe the tested feature/action. In the example below, we want to test the logout, not the login. The fact that the user is logged in should be part of the test requirements (Given
step), not part of the tested feature (When
step). Thus, there should be only one singleẀhen
step per scenario.Bad
Scenario Successful logout Given I am on the homepage When I log in as john.doe@email.com/mypassword And I log out Then I should be a visitor
Good
Scenario Successful logout Given I am a customer When I log out Then I should be a visitor
-
Each step should be written with the tense related to the step type. Read this interesting blog post.
-
Given
sentences should use the present or past (present perfect) tense to state the context. -
When
sentences should use the present tense with an action verb to describe the actions. -
Then
sentences should use the conditional tense to list assertions. They should start withI should
.Examples:
Scenario: Unsuccessful login with invalid username Given I am a visitor When I log in as a customer Then I should be a visitor And I should be told that I have provided wrong credentials
-
-
Each step should start with
I ...
-
Features and scenarios should be tagged. Tagging provides the ability to only run a subset of features and/or scenarios. Tags can be set to features and scenarios by adding
@tagname
on the line prior toFeature
orScenario
keywords.The following tags are allowed:
@smoke
(on scenarios only): To tag scenarios that are the most important from a business point of view. The objective is to run those scenarios before merging a new piece of code and blocking the merge if some tests fail (thanks to a CI ideally).@nrt
(on features and/or scenarios): To tag features/scenarios that represent all required scenarios to validate to ensure no regression and guarantee that new developments do not break any existing features.@feature
(on features only): To tag features related to the same functional area. For example:@registration
,@login
, etc. The objective is for the developer to be able to run tests related to the features he/she is updating to guarantee that he/she does not break anything.
@nrt @login Feature: Login / Logout from the user menu @smoke Scenario: Successful login with valid credentials Given I ...
Always set the context in the following order:
- Set the page from where to start the test case
I am on the home page
I am on the registration page
I am on the my account addresses page
- ...
- Set the user
I am a visitor
=> When I should be a visitor, meaning not logged in.I am a customer
=> When I should be a customer, meaning logged in, with no specific configuration.I am a customer who xxx
=> When I should be a customer with specific configuration.I am a customer who has registered 2 addresses
I am a customer with products in my cart
- ...
- Set other specific pieces of context
I go to the xxx page
I go to the homepage
I go to the registration page
- ...
I should be a visitor
=> To assert that I should not be logged in.I should be a customer
=> To assert that I should be logged in.- ...
Translation of each Gherkin sentence into executable JavaScript code.
Structure:
Given
steps should contain actions to reach the required state for the test (functions that are action verbs) followed by assertions of those actions (functions starting withsee
)When
steps should only contain actions (functions that are action verbs)Then
steps should only contain assertions (functions starting withsee
)
Example:
'use strict';
require('src/support/business-object/user.bo');
const { defineSupportCode } = require('cucumber');
defineSupportCode(function ({ Before, Given, When, Then }) {
let customer = { firstName: 'John', lastName: 'Doe', email: 'john.doe@email.com' };
Before({ tags: '@data-customer-personal-information-update-name' }, function () {
customer = { firstName: 'John2', lastName: 'Doe2', email: 'john.doe2@email.com' };
});
Given(/^I am a customer$/, function () {
return browser
.loginAs(customer.email, 'password')
.seeCustomer();
});
Given(/^I am a visitor$/, function () {
return browser
.forceLogout()
.seeVisitor();
});
When(/^I log in from (login page|menu) with valid credentials$/, function (location) {
return login(location, { usernameField: 'login@email.com', passwordField: 'password' });
});
When(/^I log out$/, function () {
return browser.logout();
});
Then(/^I should be a customer$/, function () {
return browser.seeCustomer();
});
...
- Code should be understandable by anybody (not only developers). Thus, it should contain as few logic as possible. Most of the logic should be hidden in business objects.
- Step definitions should only interact with business objects, not with page/component objects.
- Data set of scenarios should not be hard coded (use data files to be stored in the
src/support/data
folder instead). - Step definitions should not contain any technical assertions (such as
.should.eventually.be...
,.should.eventually.equal...
), the business objects should.
Bad
When(/^I register without a machine$/, function () {
browser
.setValue('#ta-registration-firstName', 'Hillary')
.setValue('#ta-registration-lastName', 'Trump')
.click('#ta-continue-bottom');
});
Good
When(/^I register without a machine$/, function () {
const dataObject = require('src/support/data/registration/without-machine/us.data');
return browser.register(dataObject);
});
Second piece of code is intelligible, data are not hard coded, it interacts only with a Business object and returns a Promise.
Objects that represent the core business. Methods should be as close as possible to the business, using the commonly shared vocabulary.
Example:
'use strict';
class User {
constructor(userComponent, loginPage) {
this.userComponent = userComponent;
this.loginPage = loginPage;
}
// Action function
login(data) {
return this.userComponent.login(data);
}
// Assertion function
async seeCustomerWithName(firstName, lastName) {
const customer = await this.userComponent.getLoggedUser();
return Promise.all([
customer.firstname.should.equal(firstName),
customer.lastname.should.equal(lastName),
]);
...
}
const user = new User(
require('src/support/component-object/user.component'),
require('src/support/page-object/login.page')
);
browser.addCommand('login', function (data) {
logger.info('logs in with user: ' + data.usernameField, {
file: __filename,
method: 'browser.login',
});
return user.login(data);
});
...
module.exports = user;
- Action function should be an action verb
- Assertion function should start with
see
- Add commands to the
browser
viaaddCommand
API to be able to easily chain promises in step definitions. - All technical assertions (such as
.should.eventually.be...
,.should.eventually.equal...
) should be used in business objects only. - Handle [functional logging)(#logging)
A page object represents a web page and define elements to interact with (e.g. fields, buttons...) and possible actions (e.g. submit form).
A component object is similar to a page object but for parts of web page that are reused in several pages, such as login block or cart summary block in header.
Example:
'use strict';
class LoginPage {
get pageElements() {
return {
loginButton: '#ta-login-page-login-submit',
registerButton: '#registerBtn',
usernameField: '#emailField',
passwordField: '#passwordField',
emailErrorMessageLabel: '#email-error-message',
};
}
clickOnRegisterButton() {
return browser.click(this.pageElements.registerButton);
}
async getWrongCredentials() {
let credentials = {
password: '',
};
credentials.errorMessage = await browser
.waitForText(this.pageElements.emailErrorMessageLabel)
.getText(this.pageElements.emailErrorMessageLabel);
credentials.login = await browser.getValue(this.pageElements.usernameField);
return credentials;
}
...
}
module.exports = new LoginPage();
- Page/Component objects should not contain any technical assertions (such as
.should.eventually.be...
,.should.eventually.equal...
), the business objects should. - Each page/component object should define a
pageElements
getter listing all the HTML elements that will be interacted with. Thus, a change in the web interface (id field change for instance) will only require a change in thepageElements
getter. All the HTML elements should be accessed throughpageElements
. - Functions returning the value of a page element should comply to the following naming convention:
getXXX
- Functions returning whether an object is visible or not should comply to the following naming convention:
isXXXVisible
- Functions just performing a single browser interaction should comply to the following naming convention:
clickXXXButton
,clickXXXLink
, etc. - Functions performing several browser interactions should start with an action verb:
deleteSecondaryAddress
,addAddress
, etc.
Even though the tests are automatically executed, it is important to be able to re-run them manually if needed. Especially for investigation in case of failure.
In order to do this, a log file is generated in log/functional-logs.log
for each execution.
Note that the file is re-created for each execution to not be polluted by previous runs.
The logging is done at the Business Object level, meaning all browser.addCommand
should include the following logging:
logger.info('Functional explanation of what is being executed.', {
file: __filename, // This never changes
method: 'browser.methodName',
});
Here is an example:
/**
* @alias Cart.proceedToCheckout
* @memberOf browser
* @method proceedToCheckoutFromMiniCart
*/
browser.addCommand('proceedToCheckoutFromMiniCart', function () {
logger.info('Click on the "Proceed to checkout" button from the mini cart.', {
file: __filename,
method: 'browser.proceedToCheckoutFromMiniCart',
});
return cart.proceedToCheckoutFromMiniCart();
});
To log data (that is used to fill in form for instance), you can use JSON.stringify
:
/**
* @alias FastRegistration.fastRegister
* @memberOf browser
* @method fastRegister
*/
browser.addCommand('fastRegister', function (data) {
logger.info('Fast register with the following data:\n' + JSON.stringify(data, null, 2), {
file: __filename,
method: 'browser.fastRegister',
});
return fastRegistration.fastRegister(data);
});
On top of the browser commands provided by WebdriverIO, additional commands are made available through the framework
such as fillInForm
to automatically fill a form from a data object, selectRadioButton
, etc. See
documentation for more details.
The faker module can be used to generate random and properly formatted data to fill in forms.
The faker library can be accessed through the global faker
object.
const firstName = faker.name.firstName();
const lastName = faker.name.lastName();
The app-module-path module can be used to make all the path relative to the root directory of your test project.
For instance to include src/support/helper/helper.js
from any other file whatever its location, the syntax will always
be the same: const helper = require('src/support/helper/helper');
Extensively use async / await to make your code more readable and maintainable.
- All configuration and helper modules
- All business objects; however, not all single functions should be tested. It is useless to test functions with no logic such as:
seeVisitor() {
return this.userComponent.isVisitor().should.eventually.be.true;
}
seeCustomer() {
return this.userComponent.getLoggedUser().should.be.fulfilled;
}
But it is useful to test functions with logic such as:
seeCustomerWithName(firstName, lastName) {
const user = await this.userComponent.getLoggedUser();
return Promise.all([
user.firstname.should.equal(firstName),
user.lastname.should.equal(lastName),
]);
}
An interesting blog post detailing how to use spies, stubs and mocks. with the Sinon library.
- Use Spies - if you simply want to watch and verify somethings happens in your test case.
- Use Stubs - if you simply want to specify how something will work to help your test case.
- Use Mocks - if you want to both of the above on a single dependency in your test case.
Always use sandbox to set and reset the context through beforeEach
and afterEach
:
let sandbox;
beforeEach(function () {
sandbox = sinon.sandbox.create();
});
afterEach(function () {
sandbox.restore();
});
- Install the
NodeJS
plugin - Update Mocha configuration to include the test-common file while running your tests. Go to Run >
Edit Configurations... > Defaults > Mocha and set the
Extra Mocha options
as follow: