Skip to content

Commit

Permalink
chore(docs): Adding Playwright docs (#3516)
Browse files Browse the repository at this point in the history
* chore(docs): Adding Playwright docs

* chore(docs): Edit suggestions

* chore(docs): Edit suggestions
  • Loading branch information
xoscar committed Jan 11, 2024
1 parent c0344a5 commit bd6d3a1
Show file tree
Hide file tree
Showing 7 changed files with 335 additions and 0 deletions.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 2 additions & 0 deletions docs/docs/tools-and-integrations/overview.mdx
Expand Up @@ -18,6 +18,8 @@ Tracetest can be integrated and used with other tools. See below which integrati

- [Cypress](/tools-and-integrations/cypress) is a JavaScript end-to-end testing framework. It is used for testing web applications by simulating user interactions within the browser. Cypress provides a fast, reliable, and easy-to-use testing environment for developers.

- [Playwright](/tools-and-integrations/playwright) is an open-source automation framework developed by Microsoft that enables cross-browser automation for web applications. It provides a set of APIs and libraries for automating interactions with web browsers such as Chrome, Firefox, and Microsoft Edge.

- [Keptn](/tools-and-integrations/keptn) is a powerful tool to automate the lifecycle of your application running on Kubernetes. With this integration, you can run Tracetest alongside Keptn test tasks to validate systems managed by Keptn.

- [K6](/tools-and-integrations/k6) is a powerful tool to run load tests against any type of services (REST, GRPC, GraphQL, etc). It is widely used by Developers, Site Reliability Engineers and Software Engineers in Test/QA teams to find potential issues when testing real life scenarios in both controlled environments and production.
Expand Down
328 changes: 328 additions & 0 deletions docs/docs/tools-and-integrations/playwright.mdx
@@ -0,0 +1,328 @@
---
id: playwright
title: Trace-Based End to End Testing with Playwright and Tracetest
description: Tracetest can be integrated and used with Playwright. This guide shows running Tracetest tests from Playwright.
keywords:
- tracetest
- trace-based testing
- observability
- distributed tracing
- testing
- playwright
- end to end testing
- end-to-end testing
- integration testing
image: https://res.cloudinary.com/djwdcmwdz/image/upload/v1698686403/docs/Blog_Thumbnail_14_rsvkmo.jpg
---

[Tracetest](https://tracetest.io/) is a testing tool based on [OpenTelemetry](https://opentelemetry.io/) that permits you to test your distributed application. It allows you to use the trace data generated by your OpenTelemetry tools to check and assert if your application has the desired behavior defined by your test definitions.

[Playwright](https://playwright.dev/) is an open-source automation framework developed by Microsoft that enables cross-browser automation for web applications. It provides a set of APIs and libraries for automating interactions with web browsers such as Chrome, Firefox, and Microsoft Edge.

## Why is this important?

The Tracetest integration for Playwright enables your current Playwright tests to easily capture a full distributed trace from your OpenTelemetry instrumented frontend and backend system. You can embed a Tracetest in this Playwright test, and allow trace-based testing assertions to be applied across this entire flow, enabling true end-to-end tests across your entire system.

## How It Works

The following is a high-level sequence diagram of how Playwright and Tracetest interact with the different pieces of the system.

```mermaid
sequenceDiagram
Playwright->>+Scripts: Execute tests
Scripts->>+@tracetest/playwright: Configures library
@tracetest/playwright-->>-Scripts: Ok
Scripts->>+Scripts: Visits website
Scripts->>+@tracetest/playwright: captures and injects traceparent meta tag
@tracetest/playwright-->-Scripts: Ok
Scripts->>+Scripts: Executes test
Scripts->>+@tracetest/playwright: Runs test
@tracetest/playwright-->-Scripts: Ok
Scripts->>@tracetest/playwright: Waits for results and shows the summary
```

## The `@tracetest/playwright` npm Package

The [`@tracetest/playwright`](https://www.npmjs.com/package/@tracetest/playwright) npm package is a Playwright plugin that allows you to run trace-based testing using Tracetest and Playwright. It is a wrapper around the Tracetest API that allows you to configure, orchestrate and run tests from Playwright.

## Requirements

**Tracetest Account**:

- Sign up to [`app.tracetest.io`](https://app.tracetest.io) or follow the [get started](/getting-started/installation) docs.
- Create an [environment](/concepts/environments).
- Create an [environment token](/concepts/environment-tokens).
- Have access to the environment's [agent API key](/configuration/agent).

**Pokeshop Demo:** Clone the official [Tracetest Pokeshop Demo App Repo](https://github.com/kubeshop/pokeshop) to your local machine.

**Docker**: Have [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) installed on your machine.

## Project Structure

The project is built with Docker Compose.

### Pokeshop Demo App

The [Pokeshop Demo App](/live-examples/pokeshop/overview) is a complete example of a distributed application using different backend and front-end services, implementation code is written in Typescript.

The `docker-compose.yml` file in the root directory is for the Pokeshop Demo app and the OpenTelemetry setup.
And the `docker-compose.e2e.yml` includes the [Tracetest Agent](/concepts/agent).

Finally, the Playwright E2E tests can be found in `playwright/e2e/1-getting-started`.

## The Playwright Setup

The Playwright setup is pretty straightforward, it was generated by installing the Playwright dependency and configuring it from the UI after running `playwright open`. The configuration script looks like this:

```typescript
import { defineConfig, devices } from "@playwright/test";
import { config } from "dotenv";

config();
export default defineConfig({
testDir: "./playwright",
fullyParallel: true,
forbidOnly: !!process.env.CI,
retries: process.env.CI ? 2 : 0,
workers: process.env.CI ? 1 : undefined,
reporter: "html",
use: {
baseURL: process.env.POKESHOP_DEMO_URL || "http://localhost:3000",
trace: "on-first-retry",
},
projects: [
{
name: "chromium",
use: { ...devices["Desktop Chrome"] },
},
],
});
```

### The `home.spec.ts` Test Script

The `home.spec.ts` test script contains three different tests based on the Pokeshop Demo UI fetures, which are:

1. Create a Pokemon
2. Import a Pokemon (using an async process)
3. Delete a Pokemon

### Tracetest Library Setup

If you go to the `package.json` file you will find the inclusion of a Tracetest package for Playwright `@tracetest/playwright`.
The first thing the test script does is import the package, grab the Tracetest API token from the environment variables and create the Tracetest instance.

```typescript
import Tracetest from "@tracetest/playwright";
const { TRACETEST_API_TOKEN = "" } = process.env;
const tracetest = Tracetest();
```

Afterward, during the `beforeAll` hook, the test script **configures** the Tracetest instance with the API token.

```typescript
test.beforeAll(async () => {
await tracetest.configure(TRACETEST_API_TOKEN);
});
```

Then, during the `beforeEach` hook, the script **captures** the document to inject the `traceparent` to the meta tag.

```typescript
test.beforeEach(async ({ page }, { title }) => {
await page.goto("/");
await tracetest.capture(title, page);
});
```

Lastly, during the `afterEach` hook, the script **runs** the test with an optional definition string.

```typescript
test.afterEach(async ({}, { title, config }) => {
await tracetest.runTest(title, config.metadata.definition ?? "");
config.metadata.definition = "";
});
```

**OPTIONAL**: If you want to wait for the test to finish and break the Playwright execution based on a failed Tracetest test, you can add the `after` hook and call the `summary` method.

```typescript
test.afterAll(async ({}, testInfo) => {
testInfo.setTimeout(60000);
await tracetest.summary();
});
```

The rest of the test script is the Playwright test definitions for the test cases mentioned above. The complete test script looks like this:

```typescript
import { test, expect } from "@playwright/test";
import Tracetest from "@tracetest/playwright";
const { TRACETEST_API_TOKEN = "" } = process.env;
const tracetest = Tracetest();

test.describe.configure({ mode: "serial" });

test.beforeAll(async () => {
await tracetest.configure(TRACETEST_API_TOKEN);
});

test.beforeEach(async ({ page }, { title }) => {
await page.goto("/");
await tracetest.capture(title, page);
});

test.afterEach(async ({}, { title, config }) => {
await tracetest.runTest(title, config.metadata.definition ?? "");
config.metadata.definition = "";
});

// optional step to break the playwright script in case a Tracetest test fails
test.afterAll(async ({}, testInfo) => {
testInfo.setTimeout(60000);
await tracetest.summary();
});

test("Playwright: creates a pokemon", async ({ page }) => {
expect(await page.getByText("Pokeshop")).toBeTruthy();

await page.click("text=Add");

await page.getByLabel("Name").fill("Charizard");
await page.getByLabel("Type").fill("Flying");
await page
.getByLabel("Image URL")
.fill("https://upload.wikimedia.org/wikipedia/en/1/1f/Pok%C3%A9mon_Charizard_art.png");
await page.getByRole("button", { name: "OK", exact: true }).click();
});

const definition = `
type: Test
spec:
id: UGxheXdyaWdodDogaW1wb3J0cyBhIHBva2Vtb24=
name: "Playwright: imports a pokemon"
trigger:
type: playwright
specs:
- selector: span[tracetest.span.type="http"] span[tracetest.span.type="http"]
name: "All HTTP Spans: Status code is 200"
assertions:
- attr:http.status_code = 200
- selector: span[tracetest.span.type="database"]
name: "All Database Spans: Processing time is less than 100ms"
assertions:
- attr:tracetest.span.duration < 2s
outputs:
- name: MY_OUTPUT
selector: span[tracetest.span.type="general" name="Tracetest trigger"]
value: attr:name
`;

test("Playwright: imports a pokemon", async ({ page }, { config: { metadata } }) => {
// set the definition for the tracetest test
metadata.definition = definition;
expect(await page.getByText("Pokeshop")).toBeTruthy();

await page.click("text=Import");

await page.getByLabel("ID").fill(Math.floor(Math.random() * 101).toString());
await page.getByRole("button", { name: "OK", exact: true }).click();
});

test("Playwright: deletes a pokemon", async ({ page }) => {
await page.locator('[data-cy="pokemon-list"]');

await page.locator('[data-cy="pokemon-card"]').first().click();
await page.locator('[data-cy="pokemon-card"] [data-cy="delete-pokemon-button"]').first().click();
});
```

### Setting the Environment Variables

Copy the `.env.example` file to `.env` and add the Tracetest API token and agent tokens to the `TRACETEST_API_TOKEN` and `TRACETEST_AGENT_API_KEY` variables.

### Starting the Pokeshop Demo App

To start the Pokeshop Demo App, run the following command from the root directory:

```bash
docker compose up -f docker-compose.yml -f docker-compose.e2e.yml
```

### Running the Tests

Next, you can run the tests by using both the Playwright CLI and the Playwright UI.

#### Using the Playwright CLI

To run the tests using the Playwright CLI, run the following command from the root directory:

```bash
npm run pr:run
```

You should see the following output:

```bash

> pokeshop@1.0.0 pr:run
> playwright test


Running 3 tests using 1 worker
[chromium] › home.spec.ts:30:5 › Playwright: creates a pokemon
Test `Playwright: creates a pokemon` started
Find the results at https://app.tracetest.io/organizations/ttorg_2179a9cd8ba8dfa5/environments/ttenv_231b49e808c29e6a/test/UGxheXdyaWdodDogY3JlYXRlcyBhIHBva2Vtb24=/run/41
[chromium] › home.spec.ts:65:5 › Playwright: imports a pokemon
Test `Playwright: imports a pokemon` started
Find the results at https://app.tracetest.io/organizations/ttorg_2179a9cd8ba8dfa5/environments/ttenv_231b49e808c29e6a/test/UGxheXdyaWdodDogaW1wb3J0cyBhIHBva2Vtb24=/run/41
[chromium] › home.spec.ts:76:5 › Playwright: deletes a pokemon
Test `Playwright: deletes a pokemon` started
Find the results at https://app.tracetest.io/organizations/ttorg_2179a9cd8ba8dfa5/environments/ttenv_231b49e808c29e6a/test/UGxheXdyaWdodDogZGVsZXRlcyBhIHBva2Vtb24=/run/43
Test `Playwright: creates a pokemon` finished
Test `Playwright: imports a pokemon` finished
Test `Playwright: deletes a pokemon` finished

Successful: 3
Failed: 0

[SUCCESSFUL] Playwright: creates a pokemon - https://app.tracetest.io/organizations/ttorg_2179a9cd8ba8dfa5/environments/ttenv_231b49e808c29e6a/test/UGxheXdyaWdodDogY3JlYXRlcyBhIHBva2Vtb24=/run/41
[SUCCESSFUL] Playwright: imports a pokemon - https://app.tracetest.io/organizations/ttorg_2179a9cd8ba8dfa5/environments/ttenv_231b49e808c29e6a/test/UGxheXdyaWdodDogaW1wb3J0cyBhIHBva2Vtb24=/run/41
[SUCCESSFUL] Playwright: deletes a pokemon - https://app.tracetest.io/organizations/ttorg_2179a9cd8ba8dfa5/environments/ttenv_231b49e808c29e6a/test/UGxheXdyaWdodDogZGVsZXRlcyBhIHBva2Vtb24=/run/43

3 passed (47.9s)

To open last HTML report run:

npx playwright show-report
```

#### Using the Playwright UI

To run the tests using the Playwright UI, run the following command from the root directory:

```bash
npm run pr:open
```

Then, click the play button at the top.

![Playwright UI](./img/playwright-scripts.png)

You should see the three tests running and passing.
And by looking at the console log you can find the statuses and results of the tests.

![Playwright Results](./img/playwright-open-results.png)

Then, you can follow any of the result links in the console log to the Tracetest App and see the results of the tests.

The Tracetest library uses the test name for the trace-based tests. That way you can identify them more easily and it also fills some of the metadata directly from the Playwright execution.

![Tracetest App](./img/playwright-tracetest-tests.png)

Lastly, you can now create assertions based on the trace data that was captured from the browser to the backend services.

Starting with the click events and the fetch request from the client side, to the HTTP requests and database queries from the backend services, including async processes like the one showcased during the import pokemon test.

![Tracetest App](./img/playwright-tracetest-import-pokemon-test.png)
5 changes: 5 additions & 0 deletions docs/sidebars.js
Expand Up @@ -342,6 +342,11 @@ const sidebars = {
id: "tools-and-integrations/cypress",
label: "Cypress",
},
{
type: "doc",
id: "tools-and-integrations/playwright",
label: "Playwright",
},
{
type: "doc",
id: "tools-and-integrations/keptn",
Expand Down

0 comments on commit bd6d3a1

Please sign in to comment.