Benchmark of web-applications testing tools

Here we compare Cypress.io vs Nightwatch.js vs Playwright vs Selenium WebDriver vs WebdriverIO, all in their Node.js versions.

We present in this repo some concrete examples of automated tests implemented in differents tools.

Constraints

• all implemented tests will use the Chrome browser already installed ¹ on the machine
• Browser is in headless mode ²
• all tests written in TypeScript.

List of tests scenarios

See this page: "List of tests scenarios"

Install

Execute in your terminal this command: npm init

Run tests

All at once, with all frameworks

Execute in your terminal this command: npm test

Run the test set 5 times?

for ((n=0;n<5;n++)); do npm test; done

Run all tests with a specific framework

tool	command	detailed script
Cypress	npm test:cy	tests/with-cypressio/run.sh
Nightwatch	npm test:nw	tests/with-nightwatch/run.sh
Playwright	npm test:pw	tests/with-playwright/run.sh
Selenium+Jest	npm test:se	tests/with-selenium-jest/run.sh
WebdriverIO	npm test:wi	tests/with-webdriverio/run.sh

Is this an article?

No. But the code present in this repository (and its execution) will be used in this article: Benchmark of web-applications testing tools.

---

Notes:

Footnotes

1. forced via --browser argument CLI command for Cypress.io, in configuration for Nightwatch and Playwright, and inside tests code for Selenium ↩

2. forced for Selenium, default mode for other ↩