Skip to content

Files

Latest commit

 

History

History
267 lines (208 loc) · 10.4 KB

playwright-visual-regression.md

File metadata and controls

267 lines (208 loc) · 10.4 KB
id title sidebar_label description keywords url site_name slug
playwright-visual-regression
Getting started with Smart UI using Playwright
Playwright
Master Playwright Visual Regression with our step-by-step guide. Seamlessly integrate SmartUI, enhance your testing skills, and unlock a world of possibilities.
Visual Regression
Visual Regression Testing Guide
Visual Regression Test Automation
Visual Regression Automation Testing
Running Visual Regression Tests
Visual Regression Testing Online
Run Visual Regression
Visual Regression Run Specific Test
Visual Regression Testing Environment
How to Run Visual Regression Tests
LambdaTest
playwright-visual-regression/

import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import NewTag from '../src/component/newTag';

This documentation will act as your step-by-step guide in to perform Playwright test with SmartUI.

  • Basic understanding of Playwright is required.
  • Go to SmartUI and login along with your credentials.
  • Copy LT_USERNAME and LT_ACCESS_KEY credentials from Access Key button on the top right of the dashboard.
export LT_USERNAME="YOUR_USERNAME" 
export LT_ACCESS_KEY="YOUR ACCESS KEY"
set LT_USERNAME="YOUR_USERNAME" 
set LT_ACCESS_KEY="YOUR ACCESS KEY"

The following steps will guide you in running your first Visual Regression test on LambdaTest platform -

Step 1: Create a SmartUI Project

The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps:

  1. Go to Projects page
  2. Click on the new project button
  3. Select the platform as Web for executing your PlayWright tests.
  4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation.
  5. Click on the Submit.

Step 2: Configure your test with Playwright Desired Capabilities

Once you have created a SmartUI Project, you can generate screenshots by running automation scripts. Follow the below steps to successfully generate screenshots -

  1. Please clone the following sample Github repo
git clone https://github.com/LambdaTest/playwright-sample
  1. Install the node modules using the command
npm i
  1. Set up the LambdaTest credentials by using the commands below in the terminal.The account details are available on your LambdaTest Profile page.

For macOS:

export LT_USERNAME=LT_USERNAME
export LT_ACCESS_KEY=LT_ACCESS_KEY

For Linux:

export LT_USERNAME=LT_USERNAME
export LT_ACCESS_KEY=LT_ACCESS_KEY

For Windows:

set LT_USERNAME=LT_USERNAME
set LT_ACCESS_KEY=LT_ACCESS_KEY
  1. Edit the required capabilities in your test file playwright-smartui.js.
const { chromium } = require('playwright')
const { expect } = require('@playwright/test');

(async () => {
  const capabilities = {
    'browserName': 'Chrome', // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit`
    'browserVersion': 'latest',
    'LT:Options': {
      'platform': 'Windows 10',
      'build': 'Playwright Sample Build',
      'name': 'Playwright Sample Test',
      'user': process.env.LT_USERNAME,
      'accessKey': process.env.LT_ACCESS_KEY,
      'network': true,
      'video': true,
      'console': true,
      "smartUIProjectName": "<projectName>" //Add the required Smart UI Project name
    }
  }

  const browser = await chromium.connect({
    wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}`
  })

  const page = await browser.newPage()

  await page.goto('https://www.bing.com')
  // Add the following command in order to take screenshot in SmartUI 
 await page.evaluate((_) => {},
`lambdatest_action: ${JSON.stringify({ action: "smartui.takeScreenshot", arguments: { fullPage: true, screenshotName: "<Your Screenshot Name>" } })}`); // Add a relevant screenshot name here

  const element = await page.$('[aria-label="Enter your search term"]')
  await element.click()
  await element.type('LambdaTest')
  await element.press('Enter')
  const title = await page.title()

  try {
    expect(title).toEqual('LambdaTest - Search')
    // Mark the test as completed or failed
    await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify({ action: 'setTestStatus', arguments: { status: 'passed', remark: 'Title matched' } })}`)
  } catch {
    await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify({ action: 'setTestStatus', arguments: { status: 'failed', remark: 'Title not matched' } })}`)
  }

  await browser.close()
})()
  • Execute the test using the following command
node playwright-smartui.js

Fetching Screenshot Status and Results using WebHook

To retrieve the status and results of a captured screenshot, you can utilize the provided webhook. Follow the steps below to fetch this information:

 // Add this code snippet within your script to fetch the screenshot status during runtime


response = await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify({action: 'smartui.fetchScreenshotStatus', arguments: { screenshotName: "" }})}`)

console.log(response)

A sample response for the same can be seen below

screenshotStatus response:  {
  screenshotsData: [
    {
      screenshotName: '<Your Screenshot Name>',
      screenshotURL: '<Link to the screenshot captured>',
      screenshotStatus: 'Approved', // current status of the screenshot
      approvedBy: 'system', // Approver details
      misMatchPercentage: 31.8, // percentage mismatch of the screenshot
      threshold: 5000, // Threshold set for the screenshot
      browserName: 'chrome', // Browser used for capturing the screenshot
      resolution: '1920x1080' // Resolution of the screenshot
    },
    
  buildId: '<Your Build ID>',
  projectName: '<Your Project Name>'
}

The following are the description of the parameters:

Parameter Description
screenshotName Name of the screenshot for which the result is to be fetched.
screenshotURL URL of the screenshot captured.
screenshotStatus Status of the screenshot captured.
approvedBy Approver of the screenshot
misMatchPercentage Percentage of mismatch in the screenshot captured.
threshold Threshold set for the screenshot captured.
browserName Browser used for capturing the screenshot.
resolution Resolution of the screenshot captured.
buildId ID of the build in which the screenshot was captured.
projectName Name of the project in which the screenshot was captured.

By including the above code, you will be able to obtain the status of the captured screenshot. If you specify a particular Screenshot Name, the response will contain the result for that specific screenshot.

:::note

Please note that the screenshot name should be provided within the screenshotName argument. :::

GitHub App Integration With Playwright

Experience effortless collaboration and streamlined testing workflows with SmartUI's GitHub app integration for Playwright users. Now, you can seamlessly link your GitHub repositories to your SmartUI accounts. This integration empowers teams to effortlessly share test scripts, review and track changes, and provide feedback within the familiar GitHub environment.

To add this integration, you can add the following capability:

const capabilities: {
  ...
   "smartUIProjectName": "<projectName>"
   "github": {
    "url": "https://api.github.com/repos/OWNER/REPO/statuses/commitId"
   }
  ...
}

:::tip For step by step guide for Github Integration click here :::

For additional information about Playwright framework please explore the documentation here

Advanced Options for Screenshot Comparison

Build Configuration - If you have multiple screenshots running the same test suite and want to run the comparison for the same test suite, want to add a build as a baseline from your test suite or need to access more SmartUI Build Config Options, click here.

Handling Dynamic Data - In case if you have any dynamic elements that are not in the same position across test runs, you can ignore or select a specific area to be removed from the comparison. For accessing such HTML DOM Config and Options, click here.