Cypress Docs (#3487)

* chore(docs): Cypress docs * initial cypress docs draft * Adding it to the overview * Edit suggestions * Edit suggestions
kubeshop · Jan 3, 2024 · 5ed8eee · 5ed8eee
1 parent 1dab58a
commit 5ed8eee
Show file tree

Hide file tree

Showing 7 changed files with 357 additions and 0 deletions.
diff --git a/docs/docs/tools-and-integrations/cypress.mdx b/docs/docs/tools-and-integrations/cypress.mdx
@@ -0,0 +1,349 @@
+---
+id: cypress
+title: Trace-Based End to End Testing with Cypress and Tracetest
+description: Tracetest can be integrated and used with Cypress. This guide shows running Tracetest tests from Cypress.
+keywords:
+  - tracetest
+  - trace-based testing
+  - observability
+  - distributed tracing
+  - testing
+  - cypress
+  - end to end testing
+  - end-to-end testing
+  - integration testing
+image: https://res.cloudinary.com/djwdcmwdz/image/upload/v1698686403/docs/Blog_Thumbnail_14_rsvkmo.jpg
+---
+
+[Tracetest](https://tracetest.io/) is a testing tool based on [OpenTelemetry](https://opentelemetry.io/) that permits you to test your distributed application. It allows you to use the trace data generated by your OpenTelemetry tools to check and assert if your application has the desired behavior defined by your test definitions.
+
+[Cypress](https://www.cypress.io/) is a JavaScript end-to-end testing framework. It is used for testing web applications by simulating user interactions within the browser. Cypress provides a fast, reliable, and easy-to-use testing environment for developers.
+
+## Why is this important?
+
+Cypress is currently one of the top end to end testing frameworks, it is a great tool in its own right that allows you to replicate most of the production challenges you might encounter by allowing developers to test the system from the user perspective. But, as with all of the tools that only test the UI portion application, you can only run validations against what the user would see while going through the flows.
+
+## How It Works
+
+The following is a high-level sequence diagram of how Cypress and Tracetest interact with the different pieces of the system.
+
+```mermaid
+sequenceDiagram
+  Cypress->>+Scripts: Execute tests
+  Scripts->>+@tracetest/cypress: Configures library
+  @tracetest/cypress-->>-Scripts: Ok
+  Scripts->>+Scripts: Visits website
+  Scripts->>+@tracetest/cypress: captures and injects traceparent meta tag
+  @tracetest/cypress-->-Scripts: Ok
+  Scripts->>+Scripts: Executes test
+  Scripts->>+@tracetest/cypress: Runs test
+  @tracetest/cypress-->-Scripts: Ok
+  Scripts->>@tracetest/cypress: Waits for results and shows the summary
+```
+
+## The `@tracetest/cypress` npm Package
+
+The `@tracetest/cypress` npm package is a Cypress plugin that allows you to run Trace-based testing using Tracetest and Cypress. It is a wrapper around the Tracetest API that allows you to configure, orchestrate and run tests from Cypress.
+
+## Requirements
+
+**Tracetest Account**:
+
+- Sign up to [`app.tracetest.io`](https://app.tracetest.io) or follow the [get started](/getting-started/installation) docs.
+- Create an [environment](/concepts/environments).
+- Create an [environment token](/concepts/environment-tokens).
+- Have access to the environment's [agent API key](/configuration/agent).
+
+**Pokeshop Demo:** Clone the official [Tracetest Pokeshop Demo App Repo](https://github.com/kubeshop/pokeshop) to your local machine.
+
+**Docker**: Have [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) installed on your machine.
+
+## Project Structure
+
+The project is built with Docker Compose.
+
+### Pokeshop Demo App
+
+The [Pokeshop Demo App](/live-examples/pokeshop/overview) is a complete example of a distributed application using different backend and front-end services, implementation code is written in Typescript.
+
+The `docker-compose.yml` file in the root directory is for the Pokeshop Demo app and the OpenTelemetry setup.
+And the `docker-compose.e2e.yml` includes the [Tracetest Agent](/concepts/agent).
+
+Finally, the Cypress E2E tests can be found in `cypress/e2e/1-getting-started`.
+
+## The Cypress Setup
+
+The Cypress setup is pretty straightforward, it was generated by installing the Cypress dependency and configuring it from the UI after running `cypress open`. The configuration script looks like this:
+
+```typescript
+import { defineConfig } from "cypress";
+import { config } from "dotenv"; // loads the environment variables
+
+config();
+
+module.exports = defineConfig({
+  chromeWebSecurity: false,
+  e2e: {
+    baseUrl: process.env.POKESHOP_DEMO_URL || "http://localhost:3000",
+    env: {
+      TRACETEST_API_TOKEN: process.env.TRACETEST_API_TOKEN, // used to authenticate with Tracetest
+    },
+    setupNodeEvents() {
+      // implement node event listeners here
+    },
+  },
+});
+```
+
+### The `home.cy` Test Script
+
+The `home.cy` test script contains three different tests based on the Pokeshop Demo UI fetures, which are:
+
+1. Creates a Pokemon
+2. Imports a Pokemon (using an async process)
+3. Deletes a Pokemon
+
+### Tracetest Library Setup
+
+If you go to the `package.json` file you will find the inclusion of a Tracetest package for Cypress `@tracetest/cypress`.
+The first thing the test script does is import the package, grab the Tracetest API token from the environment variables and create the Tracetest instance.
+
+```typescript
+import Tracetest from "@tracetest/cypress";
+const TRACETEST_API_TOKEN = Cypress.env("TRACETEST_API_TOKEN") || "";
+const tracetest = Tracetest();
+```
+
+Afterward, during the `before` hook, the test script **configures** the Tracetest instance with the API token.
+
+```typescript
+before((done) => {
+  tracetest.configure(TRACETEST_API_TOKEN).then(() => done());
+});
+```
+
+Then, during the `beforeEach` hook, the script **captures** the document to inject the `traceparent` to the meta tag.
+
+```typescript
+beforeEach(() => {
+  cy.visit("/", {
+    onBeforeLoad: (win) => tracetest.capture(win.document),
+  });
+});
+```
+
+Lastly, during the `afterEach` hook, the script **runs** the test with an optional definition string.
+
+```typescript
+afterEach((done) => {
+  const definition = Cypress.env("definition") || "";
+  tracetest.runTest(definition).then(() => done());
+});
+```
+
+**OPTIONAL**: If you want to wait for the test to finish and break the Cypress execution based on a failed Tracetest test, you can add the `after` hook and call the `summary` method.
+
+```typescript
+after((done) => {
+  tracetest.summary().then(() => done());
+});
+```
+
+The rest of the test script is the Cypress test definitions for the test cases mentioned above. The complete test script looks like this:
+
+```typescript
+import Tracetest from "@tracetest/cypress";
+const TRACETEST_API_TOKEN = Cypress.env("TRACETEST_API_TOKEN") || "";
+const tracetest = Tracetest();
+
+// trace-based tests can take a little bit longer than regular cypress scripts, this why we set the `defaultCommandTimeout` to 1 minute
+describe("Home", { defaultCommandTimeout: 60000 }, () => {
+  before((done) => {
+    tracetest.configure(TRACETEST_API_TOKEN).then(() => done());
+  });
+
+  beforeEach(() => {
+    cy.visit("/", {
+      onBeforeLoad: (win) => tracetest.capture(win.document),
+    });
+  });
+
+  afterEach((done) => {
+    const definition = Cypress.env("definition") || "";
+    tracetest.runTest(definition).then(() => done());
+  });
+
+  // uncomment to wait for trace tests to be done
+  after((done) => {
+    tracetest.summary().then(() => done());
+  });
+
+  it("create a pokemon", () => {
+    cy.get('[data-cy="create-pokemon-button"]').should("be.visible").click();
+    cy.get('[data-cy="create-pokemon-modal"]').should("be.visible");
+    cy.get("#name").type("Pikachu");
+    cy.get("#type").type("Electric");
+    cy.get("#imageUrl").type("https://oyster.ignimgs.com/mediawiki/apis.ign.com/pokemon-blue-version/8/89/Pikachu.jpg");
+
+    cy.get("button").contains("OK").click();
+  });
+
+  /// optional definition string used for TDD
+  const definition = `
+  type: Test
+  spec:
+    id: aW1wb3J0cyBhIHBva2Vtb24=
+    name: imports a pokemon
+    trigger:
+      type: cypress
+    specs:
+    - selector: span[tracetest.span.type="http"] span[tracetest.span.type="http"]
+      name: "All HTTP Spans: Status  code is 200"
+      assertions:
+      - attr:http.status_code   =   200
+    - selector: span[tracetest.span.type="database"]
+      name: "All Database Spans: Processing time is less than 100ms"
+      assertions:
+      - attr:tracetest.span.duration < 100ms
+    outputs:
+    - name: MY_OUTPUT
+      selector: span[tracetest.span.type="general" name="Tracetest trigger"]
+      value: attr:name
+    `;
+
+  it("imports a pokemon", { env: { definition } }, () => {
+    cy.get('[data-cy="import-pokemon-button"]').click();
+    cy.get('[data-cy="import-pokemon-form"]').should("be.visible");
+
+    cy.get('[id="id"]')
+      .last()
+      .type(Math.floor(Math.random() * 101).toString());
+    cy.get("button").contains("OK").click({ force: true });
+  });
+
+  it("deletes a pokemon (2)", () => {
+    cy.get('[data-cy="pokemon-list"]').should("be.visible");
+    cy.get('[data-cy="pokemon-card"]').first().click().get('[data-cy="delete-pokemon-button"]').first().click();
+  });
+});
+```
+
+### Setting the Environment Variables
+
+Copy the `.env.example` file to `.env` and add the Tracetest API token and agent tokens to the `TRACETEST_API_TOKEN` and `TRACETEST_AGENT_API_KEY` variables.
+
+### Starting the Pokeshop Demo App
+
+To start the Pokeshop Demo App, run the following command from the root directory:
+
+```bash
+docker compose up -f docker-compose.yml -f docker-compose.e2e.yml
+```
+
+### Running the Tests
+
+Next, you can run the tests by using both the Cypress CLI and the Cypress UI.
+
+#### Using the Cypress CLI
+
+To run the tests using the Cypress CLI, run the following command from the root directory:
+
+```bash
+npm run cy:run
+```
+
+You should see the following output:
+
+```bash
+> pokeshop@1.0.0 cy:run
+> cypress run
+
+
+DevTools listening on ws://127.0.0.1:54740/devtools/browser/8c9d2503-cbaa-4ef2-b716-5bc1ad6cf6c8
+Missing baseUrl in compilerOptions. tsconfig-paths will be skipped
+
+====================================================================================================
+
+  (Run Starting)
+
+  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
+  │ Cypress:        13.6.1                                                                         │
+  │ Browser:        Electron 114 (headless)                                                        │
+  │ Node Version:   v16.15.1 (/Users/oscarr.reyesgaucin/.nvm/versions/node/v16.15.1/bin/node)      │
+  │                                                                                                │
+  │ Specs:          1 found (home.cy.ts)                                                           │
+  │ Searched:       cypress/e2e/**/*.cy.{js,jsx,ts,tsx}                                            │
+  └────────────────────────────────────────────────────────────────────────────────────────────────┘
+
+
+────────────────────────────────────────────────────────────────────────────────────────────────────
+
+  Running:  home.cy.ts                                                                      (1 of 1)
+
+
+  Home
+    ✓ create a pokemon (4108ms)
+    ✓ imports a pokemon (756ms)
+    ✓ deletes a pokemon (2) (653ms)
+
+
+  3 passing (42s)
+
+
+  (Results)
+
+  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
+  │ Tests:        3                                                                                │
+  │ Passing:      3                                                                                │
+  │ Failing:      0                                                                                │
+  │ Pending:      0                                                                                │
+  │ Skipped:      0                                                                                │
+  │ Screenshots:  0                                                                                │
+  │ Video:        false                                                                            │
+  │ Duration:     41 seconds                                                                       │
+  │ Spec Ran:     home.cy.ts                                                                       │
+  └────────────────────────────────────────────────────────────────────────────────────────────────┘
+
+
+====================================================================================================
+
+  (Run Finished)
+
+
+       Spec                                              Tests  Passing  Failing  Pending  Skipped
+  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
+  │ ✔  home.cy.ts                               00:41        3        3        -        -        - │
+  └────────────────────────────────────────────────────────────────────────────────────────────────┘
+    ✔  All specs passed!                        00:41        3        3        -        -        -
+
+```
+
+#### Using the Cypress UI
+
+To run the tests using the Cypress UI, run the following command from the root directory:
+
+```bash
+npm run cy:open
+```
+
+Then, navigate your way to the `e2e` section and select the `home.cy` test script.
+
+![Cypress UI](./img/cypress-scripts.png)
+
+You should see the three tests running and passing.
+And by looking at the browser console log you can find the statuses and results of the tests.
+
+![Cypress Results](./img/cypress-open-results.png)
+
+Then, you can follow any of the result links in the console log to the Tracetest App and see the results of the tests.
+
+The Tracetest library uses the spec name for the trace-based tests. That way you can identify them more easily and it also fills some of the metadata directly from the Cypress execution. 
+
+![Tracetest App](./img/cypress-tracetest-tests.png)
+
+Lastly, you can now create assertions based on the trace data that was captured from the browser to the backend services.
+
+Starting with the click events and the fetch request from the client side, to the HTTP requests and database queries from the backend services, including async processes like the one showcased during the import pokemon test.
+
+![Tracetest App](./img/cypress-tracetest-import-pokemon-test.png)
+
diff --git a/docs/docs/tools-and-integrations/img/cypress-open-results.png b/docs/docs/tools-and-integrations/img/cypress-open-results.png
diff --git a/docs/docs/tools-and-integrations/img/cypress-scripts.png b/docs/docs/tools-and-integrations/img/cypress-scripts.png
diff --git a/docs/docs/tools-and-integrations/img/cypress-tracetest-import-pokemon-test.png b/docs/docs/tools-and-integrations/img/cypress-tracetest-import-pokemon-test.png
diff --git a/docs/docs/tools-and-integrations/img/cypress-tracetest-tests.png b/docs/docs/tools-and-integrations/img/cypress-tracetest-tests.png
diff --git a/docs/docs/tools-and-integrations/overview.mdx b/docs/docs/tools-and-integrations/overview.mdx
@@ -13,8 +13,11 @@ image: https://res.cloudinary.com/djwdcmwdz/image/upload/v1698686403/docs/Blog_T
 
 Tracetest can be integrated and used with other tools. See below which integrations we are working on:
 
+
 ## Integrations
 
+- [Cypress](/tools-and-integrations/cypress) is a JavaScript end-to-end testing framework. It is used for testing web applications by simulating user interactions within the browser. Cypress provides a fast, reliable, and easy-to-use testing environment for developers.
+
 - [Keptn](/tools-and-integrations/keptn) is a powerful tool to automate the lifecycle of your application running on Kubernetes. With this integration, you can run Tracetest alongside Keptn test tasks to validate systems managed by Keptn.
 
 - [K6](/tools-and-integrations/k6) is a powerful tool to run load tests against any type of services (REST, GRPC, GraphQL, etc). It is widely used by Developers, Site Reliability Engineers and Software Engineers in Test/QA teams to find potential issues when testing real life scenarios in both controlled environments and production.

diff --git a/docs/sidebars.js b/docs/sidebars.js
@@ -337,6 +337,11 @@ const sidebars = {
         id: "tools-and-integrations/overview",
       },
       items: [
+        {
+          type: "doc",
+          id: "tools-and-integrations/cypress",
+          label: "Cypress",
+        },
         {
           type: "doc",
           id: "tools-and-integrations/keptn",