Note 2019-06-19: this tutorial is slightly outdated.
KITE Tests are implemented in a framework that follows the Page Object Model design pattern (POM), and organises a Test in Pages, Steps and Checks. Using Page Object Model, all element locators are being managed in separate directory and can be updated easily without any modification to the test scenarios. The best practice to this model is that no WebDriver API should be call in the test class, it should be manage in Pages, especially for exception/error handling.
A Test consists of a succession of Steps and Checks:
- Steps are actions, for example, navigate to a page or click a button. Steps are not expected to fail, they either 'Pass' or 'Break' (any unexpected error).
- Checks are assertations, for example, validate that a video is playing. Checks can 'Pass' (video is playing), 'Fail' (video is blank/still) or 'Break' (e.g. page timeout).
Pages are where we place the element locators and the functions to interact with them.
This tutorial will guide you step-by-step in writing your first KITE Test. The sample test will open the https://meet.jit.si/ page on a web browser (Chrome and Firefox), check the sent and received videos, collect the WebRTC statistics and take a screenshot.
- Create the Test
- Adding the first Step: open https://meet.jit.si/ page
- Adding the MainPage (Jitsi page)
- Adding the Checks :
- Sent Video Check
- Received Videos Check
- Adding more Steps:
- Collect WebRTC Stats
- Taking a Screenshot
You can find the full source code of this tutorial here.
To create the test, execute the following commands:
On Windows:
cd %KITE_HOME%
kite_init jitsiTutorial
c
On Linux/Mac:
cd $KITE_HOME
kite_init jitsiTutorial
c
This will create KITE-JitsiTutorial-Test folder containing all the basic files and sub-folders used in all KITE Tests, for both Java and Javascript.
This tutorial is about the Javascript KITE Test and all the javascript files are located in KITE-JitsiTutorial-Test/js.
It will also compile the project, which will setup npm, nodejs and trigger npm install for the js/ folder.
Let's have a closer look at the files and folders generated in KITE-JitsiTutorial-Test/js.
The file js/JitsiTutorial.js is the entry point of the test. It will be run by KITE by calling node JitsiTutorial.js <params of the tests>.
At the top of this file, we will find three classes imported from kite-common:
const {TestUtils, WebDriverFactory, KiteBaseTest} = require('kite-common');
TestUtils contains a set of common functions that are used by all KITE Tests. WebDriverFactory allows the creation of the WebDriver. KiteBaseTest this is the parent class of the test, it will enable the generation of the Allure Report.
The following code gets the capabilities required to create the webdrivers and the payload which is the test specific configuration passed to the test by the KITE Engine. We can pass any kind of information to the test via this payload object in the config file. It can also get the command line arguments.
const kiteConfig = await TestUtils.getKiteConfig(__dirname);
The kite_init script generated 3 subfolders:
- pages/
- steps/
- checks/
This is where we will respectively implement the Pages, Steps and Checks of the test, following the Page Object Model design pattern.
The node_modules should have already been created during the compilation process by npm install.
The KITE config file of our javascript test can be found at KITE-JitsiTutorial-Test/configs/js.jitsiTutorial.config.json
This file contains all the information for the tests.
name: name of the configuration file. grids: the address and the type of the selenium hub. tests: a array containing all the tests to be run. Each test contains all the necessary information for its progress. In each test we can find:
- name: name of the test
- description: description of the test
- tupleSize: number of browsers participating/launched at the same time during the test
- testImpl: name of the JS test file
- payload contains name:values pairs to be used in the test.
clients: List of all the browsers that will be tested. This list will be used to create tuples (with provided tupleSize). Each tuple will be used to run in one test case.
Once done, we can already run the test with:
r configs\js.jitsiTutorial.config.json
At this stage, the test only launches a webbrowser, opens https://google.com and does a random check.
KITE test is not based on any unit test framework, so Allure has no support for it. However, KITE test has its own report system, inspired by Allure reporting, and can be served by Allure.Open the Allure Report with:
a
The report should look like this:
The file /steps/OpenUrlStep.js was generated by the kite_init script. It already contains everything required for this Step.
TestStep is the parent class that enables the Allure Report generation.
The constructor of each Step must contain the WebDriver object. It is needed for the Step to control the web browser and execute what it needs to do.
Next, there is the attribute this.page which contains an instance of the MainPage.
The constructor can include other variables and objects, specifically needed for the Step.
⚠ To create a Step or a Check, we must implement two functions, stepDescription() and step(), which are abstract functions of the parent class TestStep.
-
stepDescription() returns a string which will be displayed in the report.
-
step() is asynchronous because it executes WebDriver functions which are asynchronous. Therefore, we use
async/awaitto make it synchronous.
Example:
async step() {
await this.page.open(this);
}
step() will call open() in /pages/MainPage.js which use TestUtils.open(url) to open the browser to the url.
But, you have to change the url in /configs/js.jitsiTutorial.config.json in the payload:
"url": "https://meet.jit.si/"
Now we're going to add the code that allows joining a meeting.
Similarly to the Steps which are located in the steps/ folder, the Pages are located in the pages/ folder.
The kite_init script created two files: pages/MainPage.js and pages/index.js.
We can now edit pages/MainPage.js and add the HTML locators and functions that allows interactions with the web page.
The HTML locators will be represented the class By from Selenium-webdriver.
Here, we'll use the text input to choose our meeting room;
const meetingRoom = By.id('enter_room_field') where enter_room_field is the id of the HTML object.
In MainPage class, add:
async enterRoom(roomId) {
let meeting = await this.driver.findElement(meetingRoom);
await meeting.sendKeys(roomId); // Fill out the field and add some random numbers
await meeting.sendKeys(Key.ENTER); // Press ENTER to enter in the room
}
The variable stepInfo is a reference to the Step object (= this).
To interact with a HTML element, for example the button or the text input, we're using the selenium-webdriver API.
The full API doc can be found here: Selenium-webdriver
Key.XXX where XXX is an enumeration of keys like "ENTER" that can be used: Key.
Now, we are going to modify /steps/OpenUrlStep.js to join our metting room.
In step(), add: `await this.page.enterRoom("I am a random room" + this.uuid);
To obtain:
async step() {
await TestUtils.open(this);
await this.page.enterRoom("I am a random room" + this.uuid);
}
See OpenUrlStep.js.
And the report should now look like:
Now, we're going to add a Check. A Check is a kind of Step that asserts a condition and can 'Fail'. All Checks are places in the checks/ folder.
We're goind to create a file named checks/SentVideoCheck.js.
Once the file has been created, we'll add a reference to checks/index.js so it's available to any code that requires the folder checks/:
____ checks/index.js ____
...
exports.SentVideoCheck = require('./SentVideoCheck');
...
The objective of this Check is to validate that the video is being sent. We'll need some elements in the pages/MainPage.js to access the <video> elements.
Open the file pages/MainPage.js.
To add the <video> elements, we're simply add:
const videos = By.css('video'); => This allows us to get all the HTML elements with tag <video> into our videos array.
Next, we're going to add a synchronous function called videoCheck() where we'll implement the video check logic.
async videoCheck(stepInfo, index) {
...
},
stepInfo is a reference to the Check object. index is the index of the video to be checked (0 for the first video, 1 for the 2nd, etc...)
First, we'll declare our variables:
async videoCheck(stepInfo, index) {
let checked; // Result of the verification
let i; // iteration indicator
let timeout = stepInfo.timeout;
stepInfo.numberOfParticipant = parseInt(stepInfo.numberOfParticipant) + 1; // To add the first video
}
Then we'll wait for all the videos. So, we're going to use TestUtils.waitForVideos() from kite-common.
async videoCheck(stepInfo, index) {
let checked; // Result of the verification
let i; // iteration indicator
let timeout = stepInfo.timeout;
stepInfo.numberOfParticipant = parseInt(stepInfo.numberOfParticipant) + 1; // To add the first video
// Waiting for all the videos
await TestUtils.waitForVideos(stepInfo, videos);
stepInfo.numberOfParticipant --; // To delete the first video
}
We add parseInt(stepInfo.numerOfParticipant) + 1 because there is an other video that does not need to be checked. But, we remove it right after.
Next we'll check that the video is actually playing, meaning that it isn't blank (all the pixels of the video frame are black/white) or still, which means that the same image is still displayed after a second interval. For that we'll use the utility function TestUtils.verifyVideoDisplayByIndex() from kite-common:
async videoCheck(stepInfo, index) {
let checked; // Result of the verification
let i; // iteration indicator
let timeout = stepInfo.timeout;
stepInfo.numberOfParticipant = parseInt(stepInfo.numberOfParticipant) + 1; // To add the first video
// Waiting for all the videos
await TestUtils.waitForVideos(stepInfo, videos);
stepInfo.numberOfParticipant --; // To delete the first video
// Check the status of the video
// checked.result = 'blank' || 'still' || 'video'
i = 0;
checked = await TestUtils.verifyVideoDisplayByIndex(stepInfo.driver, index + 1);
while(checked.result === 'blank' || typeof checked.result === "undefined" && i < timeout) {
checked = await TestUtils.verifyVideoDisplayByIndex(stepInfo.driver, index + 1);
i++;
await TestUtils.waitAround(1000);
}
i = 0;
while(i < 3 && checked.result != 'video') {
checked = await TestUtils.verifyVideoDisplayByIndex(stepInfo.driver, index + 1);
i++;
await TestUtils.waitAround(3 * 1000); // waiting 3s after each iteration
}
return checked.result;
}
We need to add index + 1 to skip the large video in our .
The asynchronous function TestUtils.waitAround() allows to wait for a given time.
To make the check robust to poor connections, we decided to repeat it 3 times at 3 seconds interval. We could make the checks much stricter by doing it only once, which would cause it to fail more easily in case of low framerate.
Now that we completed the implementation in pages/MainPage.js, we're going to edit checks/SentVideoCheck.js.
As previously mentioned, a Check is a kind of Step and inherits from the the framework's TestStep class.
At the top of the file, we add the following require:
const {TestStep, KiteTestError, Status} = require('kite-common');
Then we implement the SentVideoCheck Class and its constructor:
class SentVideoCheck extends TestStep {
constructor(kiteBaseTest) {
super();
this.driver = kiteBaseTest.driver;
this.timeout = kiteBaseTest.timeout;
this.numberOfParticipant = kiteBaseTest.numberOfParticipant;
this.page = kiteBaseTest.page;
// Test reporter if you want to add attachment(s)
this.testReporter = kiteBaseTest.reporter;
}
}
Update the stepDescription():
stepDescription() {
return "Check the first video is being sent OK";
}
Then the step(), where we call this.page.videoCheck() for the first video in the page (the sender's video).
We compare the result and if it's not 'video' we throw a KiteTestError with the Status.FAILED, that will be reported accordingly in the Allure Report.
async step() {
try {
let result = await this.page.videoCheck(this, 0);
if (result != 'video') {
this.testReporter.textAttachment(this.report, "Sent video", result, "plain");
throw new KiteTestError(Status.FAILED, "The video sent is " + result);
}
} catch (error) {
console.log(error);
if (error instanceof KiteTestError) {
throw error;
} else {
throw new KiteTestError(Status.BROKEN, "Error looking for the video");
}
}
}
To finish with this file, at the end, add:
module.exports = SentVideoCheck;
See SentVideoCheck.js.
Lastly, we return to the main Test class file: JitsiTutorial.js and add our check there:
At the top of the file, we import the class
const {SentVideoCheck} = require('./checks');
And add the check to the testScript() function:
this.driver = await WebDriverFactory.getDriver(this.capabilities, this.remoteUrl);
this.page = new MainPage(this.driver);
let openUrlStep = new OpenUrlStep(this);
await openUrlStep.execute(this);
// New check
let sentVideoCheck = new SentVideoCheck(this);
await sentVideoCheck.execute(this);
Once here, you can remove checks/MyFirstCheck.js, remove it from JitsiTutorial.js and checks/index.js.
See JitsiTutorial.js.
Now, our test is able to check the sentVideo.
We can run the test again and see the report like mentioned above.
Our report should now look like this:
Now, we are going to add an other Check. It will check that we received all the remote videos.
First, we have to change the value of tupleSize in /configs/js.jitsiTutorial.config.json and set it at 2 because we'll need 2 browsers.
We're goind to create a file named checks/ReceivedVideoCheck.js.
Once the file has been created, we will add a reference to checks/index.js.
____ checks/index.js ____
exports.VideoSentCheck = require('./VideoSentCheck');
The objective of this Check is to validate that all the remode videos are being received. Fortunately, we already have everything to do it.
Open the file checks/ReceivedVideoCheck.js.
Like the last check, at the top of the file, we add the following require:
const {TestStep, KiteTestError, Status} = require('kite-common');
Then we implement the ReceivedVideoCheck Class and its constructor:
class ReceivedVideoCheck extends TestStep {
constructor(kiteBaseTest) {
super();
this.driver = kiteBaseTest.driver;
this.timeout = kiteBaseTest.timeout;
this.numberOfParticipant = kiteBaseTest.numberOfParticipant;
this.page = kiteBaseTest.page;
// Test reporter if you want to add attachment(s)
this.testReporter = kiteBaseTest.reporter;
}
}
Update the stepDescription():
stepDescription() {
return "Check the other videos are being received OK";
}
Then the step(), where we call this.page.videoCheck() for each remote video.
We compare every result and if one is not 'video', then we throw a KiteTestError with the Status.FAILED, that will be reported accordingly in the Allure Report.
async step() {
let result = "";
let tmp;
let error = false;
try {
for(let i = 1; i < this.numberOfParticipant; i++) {
tmp = await this.page.videoCheck(this, i);
result += tmp;
if (i < this.numberOfParticipant) {
result += ' | ';
}
if (tmp != 'video') {
error = true;
}
}
if (error) {
this.testReporter.textAttachment(this.report, "Received videos", result, "plain");
throw new KiteTestError(Status.FAILED, "Some videos are still or blank: " + result);
}
} catch (error) {
console.log(error);
if (error instanceof KiteTestError) {
throw error;
} else {
throw new KiteTestError(Status.BROKEN, "Error looking for the video");
}
}
}
The for loop, starting at index 1, allows to check every received video. Indeed, the one of index 0 is the video sent.
At the end of the file, we shall add the following line to export this module and make it available to JitsiTutorial.js:
module.exports = ReceivedVideoCheck;
See ReceivedVideoCheck.js.
Finally, we add this check as the previous one in JitsiTutorial.js.
We modify:
const {SentVideoCheck} = require('./checks');
by
const {SentVideoCheck, ReceivedVideoCheck} = require('./checks');
and add in testScript():
this.driver = await WebDriverFactory.getDriver(this.capabilities, this.remoteUrl);
this.page = new MainPage(this.driver);
let openUrlStep = new OpenUrlStep(this);
await openUrlStep.execute(this);
let sentVideoCheck = new SentVideoCheck(this);
await sentVideoCheck.execute(this);
// New check
let receivedVideoCheck = new ReceivedVideoCheck(this);
await receivedVideoCheck.execute(this);
See JitsiTutorial.js.
Now, our test is able to every videos (sent and received).
We can run the test again like mentioned above and the report will look like this:
#CONFIG FILE
Before we start, we have to modify our config file, configs/js.jitsiTutorial.config.json.
Indeed, this step requires some information to work properly.
So, in payload, we add :
"getStats" : {
"enabled": true,
"statsCollectionTime": 2,
"statsCollectionInterval": 1,
"peerConnections": ["window.peerConnections[0]"],
"selectedStats" : ["inbound-rtp", "outbound-rtp", "candidate-pair"]
}
To obtain:
"payload":{
"url": "https://meet.jit.si/",
"getStats" : {
"enabled": true,
"statsCollectionTime": 2,
"statsCollectionInterval": 1,
"peerConnections": ["window.peerConnections[0]"],
"selectedStats" : ["inbound-rtp", "outbound-rtp", "candidate-pair"]`
}
}
Now, we are going to create a step from scratch. Remember, all Steps are places in the steps/ folder.
We're going to create a file named steps/GetStatsStep.js.
Once the file has been created, we'll also add a reference to steps/index.js.
____ steps/index.js ____
exports.GetStatsStep = require('./GetStatsStep');
Open the file checks/GetStatsStep.js..
At the top of the file, we add the following require:
const {TestUtils, TestStep, KiteTestError, Status} = require('kite-common');
Then, in this file, we implement the GetStatsStep class and its constructor:
class GetStatsStep extends TestStep {
constructor(kiteBaseTest) {
super();
this.driver = kiteBaseTest.driver;
this.statsCollectionTime = kiteBaseTest.statsCollectionTime;
this.statsCollectionInterval = kiteBaseTest.statsCollectionInterval;
this.selectedStats = kiteBaseTest.selectedStats;
this.peerConnections = kiteBaseTest.peerConnections;
this.page = kiteBaseTest.page;
// Test reporter if you want to add attachment(s)
this.testReporter = kiteBaseTest.reporter;
}
}
this.statsCollectionTime: duration of data collection
this.statsCollectionTime: interval of data collection
this.selectedStats: the data we want to collect
this.peerConnections: array of peer connections used to get stats;
Update the stepDescription():
stepDescription() {
return 'Getting WebRTC stats via getStats';
}
Then the step():
async step() {
try {
let rawStats = await this.page.getStats(this);
let summaryStats = TestUtils.extractJson(rawStats, 'both');
// // Data
this.testReporter.textAttachment(this.report, 'GetStatsRaw', JSON.stringify(rawStats, null, 4), "json");
this.testReporter.textAttachment(this.report, 'GetStatsSummary', JSON.stringify(summaryStats, null, 4), "json");
} catch (error) {
console.log(error);
throw new KiteTestError(Status.BROKEN, "Failed to getStats");
}
}
Do not forget, at the end of this file, add:
module.exports = GetStatsStep;
See GetStatsStep.js.
The this.page.getStats() allows to get stats. We are going to create it right after in MainPage(); The TestUtils.extractJson() allows to make a summary of the data collected.
Now, we are going to create getStats(). Open the file /pages/MainPage.js.
First, we need a script to get the peer connection. Here is the script:
window.peerConnections = [];
map = APP.conference._room.rtc.peerConnections;
for(var key of map.keys()){
window.peerConnections.push(map.get(key).peerconnection);
}
So, we create a function, outside our class, to get this script:
const getPeerConnectionScript = function() {
return "window.peerConnections = [];"
+ "map = APP.conference._room.rtc.peerConnections;"
+ "for(var key of map.keys()){"
+ " window.peerConnections.push(map.get(key).peerconnection);"
+ "}";
}
Then, in MainPage class:
async getStats(stepInfo) {
await stepInfo.driver.executeScript(getPeerConnectionScript());
let stats = await TestUtils.getStats(stepInfo, 'kite', stepInfo.peerConnections[0]);
return stats;
}
See MainPage.js.
The stepInfo.driver.executeScript(getPeerConnectionScript()) will execute our script to get the peer connection more easily. The TestUtils.getStats() get stats from the peer connection.
Finally, we add this step in JitsiTutorial.js.
We modify:
const {OpenUrlStep} = require('./steps');
by
const {OpenUrlStep, GetStatsStep} = require('./steps');
and add in testScript():
this.driver = await WebDriverFactory.getDriver(this.capabilities, this.remoteUrl);
this.page = new MainPage(this.driver);
let openUrlStep = new OpenUrlStep(this);
await openUrlStep.execute(this);
let sentVideoCheck = new SentVideoCheck(this);
await sentVideoCheck.execute(this);
let receivedVideoCheck = new ReceivedVideoCheck(this);
await receivedVideoCheck.execute(this);
// New step
if (this.getStats) {
let getStatsStep = new GetStatsStep(this);
await getStatsStep.execute(this);
}
See JitsiTutorial.js.
Like this, it's easier to enable or disable this step using the config file.
Now, we can get stats from the peer connection.
We can run the test again like mentioned above and the updated report should be now:
Now, we're going to create a new step to take a screenshot. This step is already created in kite-common in steps/ folder.
To use it, simply add the following in our main JS file:
-
Modify:
const {TestUtils, TestStep, KiteTestError, Status} = require('kite-common');byconst {TestUtils, TestStep, KiteTestError, Status, ScreenshotStep} = require('kite-common'); -
Add the step:
let screenshotStep = new ScreenshotStep(this);
await screenshotStep.execute(this);
in testScript():
this.driver = await WebDriverFactory.getDriver(this.capabilities, this.remoteUrl);
this.page = new MainPage(this.driver);
let openUrlStep = new OpenUrlStep(this);
await openUrlStep.execute(this);
let sentVideoCheck = new SentVideoCheck(this);
await sentVideoCheck.execute(this);
let receivedVideoCheck = new ReceivedVideoCheck(this);
await receivedVideoCheck.execute(this);
if (this.getStats) {
let getStatsStep = new new GetStatsStep(this);
await getStatsStep.execute(this);
}
// New step
let screenshotStep = new ScreenshotStep(this);
await screenshotStep.execute(this);
This step uses the asynchronous function TestUtils.takeScreenshot() from kite-common, to take a screenshot.
Useful information:
Sometimes, a browser can perform the steps faster than others, that's why the asynchronous function waitAllSteps()
allows to synchronize your browsers. You can use it in testScript() like this:
await this.waitAllSteps()
See: JitsiTutorial.js. js.jitsitutorial.config.json.
We can also do the same as getStats to enable or disable this step more easily. We just have to add "takeScreenshotForEachTest": true, in configs/js.jitsiTutorial.config.json in the payload.
We can run the test like mentioned above.
Finally, the report should be now:
Congratulations, you've implemented your first KITE Test in javascript!