Skip to content

Commit

Permalink
Update Run Tests guides
Browse files Browse the repository at this point in the history
  • Loading branch information
flaurida committed Mar 10, 2020
1 parent 4a648a1 commit c8182c1
Show file tree
Hide file tree
Showing 3 changed files with 34 additions and 26 deletions.
2 changes: 2 additions & 0 deletions docs/docs/components/CiArtifacts.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ After your tests run in GitHub Actons, your debug artifacts will be available fo

The video below shows how to download artifacts in GitHub Actions:

TODOUPDATEVIDEO

<video controls title="download artifacts in GitHub" width="100%">
<source
src="https://storage.googleapis.com/docs.qawolf.com/guides/github_artifacts.mp4"
Expand Down
52 changes: 28 additions & 24 deletions docs/docs/run_tests_in_ci.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,14 @@ id: run_tests_in_ci
title: ☁️ Run Tests in CI
---

import CiArtifacts from "./components/CiArtifacts.mdx";
import CiCodeBlock from "./components/CiCodeBlock.mdx";
import CiCreateWorkflow from "./components/CiCreateWorkflow.mdx";
import CiEnvironmentVariables from "./components/CiEnvironmentVariables.mdx";
import CiLocalServer from "./components/CiLocalServer.mdx";
import CiRunOnSchedule from "./components/CiRunOnSchedule.mdx";
import CiArtifacts from './components/CiArtifacts.mdx';
import CiCodeBlock from './components/CiCodeBlock.mdx';
import CiCreateWorkflow from './components/CiCreateWorkflow.mdx';
import CiEnvironmentVariables from './components/CiEnvironmentVariables.mdx';
import CiLocalServer from './components/CiLocalServer.mdx';
import CiRunOnSchedule from './components/CiRunOnSchedule.mdx';

QA Wolf set up your CI with one command. Each test run includes [debug artifacts](#debug) like a video recording and detailed logs.
QA Wolf sets up your CI with one command. Each test run includes [debug artifacts](#debug) like a video recording and logs.

This guide assumes you have already [created a browser test](create_a_test).

Expand All @@ -20,7 +20,7 @@ This guide assumes you have already [created a browser test](create_a_test).

<CiCodeBlock />

- Each test run includes [debug artifacts](#debug) like a video recording and detailed logs
- Each test run includes [debug artifacts](#debug) like a video recording and logs
- You can also [run your tests against your local server](#run-tests-against-your-local-server), [use environment variables](#use-environment-variables), and [run tests on a schedule](#run-tests-on-a-schedule) in CI

## Create workflow file
Expand All @@ -40,36 +40,34 @@ The rest of this guide covers common use cases of running your tests in CI:

## Debug

By default, every test run includes debugging artifacts like a video recording and detailed logs. These help you better understand your tests and debug failures.
By default, every test run includes debug artifacts like a video recording and detailed logs. These help you better understand your tests and debug failures.

Your debugging artifacts are uploaded in a folder called `qawolf`, which has the following structure. Artifacts for each test are stored in separate folders, with the folder named after the test like `myTestName.test.js`. Within each test folder there is a separate folder for each browser that you ran your tests on: [Chromium](https://www.chromium.org/Home), [Firefox](https://www.mozilla.org/en-US/firefox/new), and [WebKit](https://webkit.org). In the example below, the tests were only run on Chromium.
Your debug artifacts are uploaded in a folder called `qawolf`, which has the following structure. Artifacts for each test are stored in separate folders, with the folder named after the test like `myTestName.test.js`. Within each test folder there is a separate folder for each browser that you ran your tests on: [Chromium](https://www.chromium.org/Home), [Firefox](https://www.mozilla.org/en-US/firefox/new), and [WebKit](https://webkit.org). In the example below, the tests were only run on Chromium.

TODOCONFIRM

```bash
qawolf
├── myFirstTest.test.js
│ └── Chromium
│ └── video_{timestamp}.mp4
│ └── video_{timestamp}.gif
│ └── browser_{timestamp}.log
│ └── qawolf_{timestamp}.log
│ └── video_0.mp4
│ └── logs_0.txt
│ └── video_1.mp4
│ └── logs_1.txt
├── mySecondTest.test.js
│ └── Chromium
│ └── video_{timestamp}.mp4
│ └── video_{timestamp}.gif
│ └── browser_{timestamp}.log
│ └── qawolf_{timestamp}.log
│ └── video_0.mp4
│ └── logs_0.txt
# etc
```

The file called `video_{timestamp}.mp4` (where `{timestamp}` is replaced with a real timestamp, for example `video_1580757606885.mp4`) is a video recording of your test. This allows you to watch your test run in the browser. The corresponding `.gif` file `video_{timestamp}.gif` is a [GIF](https://en.wikipedia.org/wiki/GIF) version of the same video.

The two log files are called `browser_{timestamp}.log` and `qawolf_{timestamp}.log`.
The file called `video_0.mp4` is a video recording of your test. This allows you to watch your test run in the browser. The `0` in the file name refers to the index of the page that was recorded. If your tests involve multiple pages (for example, sign in with Twitter), a video of each page is saved separately. Page index starts at `0`, so the video for the first page in your test will be saved at `video_0.mp4`.

The `browser_{timestamp}.log` file contains all logs from the browser. This includes logs from QA Wolf as it uses your applicaton and looks for each element. It also includes console logs, warnings, and errors from your application. This is generally the log file you should look at when debugging as it contains logs relevant to your application.
Video is currently only supported on Chromium. We are [waiting for Playwright](https://github.com/microsoft/playwright/issues/1158) to add support for Firefox and WebKit.

The `qawolf_{timestamp}.log` file contains detailed logs from the QA Wolf Node.js server. You probably won't need to look at it (unless you are submitting it to us for help debugging an issue 😊).
Console logs are saved in the `logs_0.txt` file. As with the video, the `0` refers to the index of the page the logs are from.

To view your debug artifacts, follow the instructions below for your CI provider.
To view your debug artifacts in CI, follow the instructions below for your provider.

<CiArtifacts />

Expand All @@ -81,6 +79,8 @@ In this guide, we assume you have a script that you can run to start your local

To run your tests against your local server, follow the instructions below for your CI provider.

TODOUPDATE

<CiLocalServer />

## Use environment variables
Expand All @@ -91,6 +91,8 @@ Depending on whether the environment variable contains sensitive information, yo

To use environment variables, follow the instructions below for your CI provider.

TODOUPDATE

<CiEnvironmentVariables />

## Run tests on a schedule
Expand All @@ -99,6 +101,8 @@ By default, the [generated workflow file](#create-workflow-file) is configured t

To run your tests on a schedule, follow the instructions below for your CI provider.

TODOUPDATE

<CiRunOnSchedule />

## Do something else
Expand Down
6 changes: 4 additions & 2 deletions docs/docs/run_tests_locally.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ id: run_tests_locally
title: 🏃 Run Tests Locally
---

import BrowserCodeBlock from "./components/BrowserCodeBlock.mdx";
import BrowserCombinedCodeBlock from "./components/BrowserCombinedCodeBlock.mdx";
import BrowserCodeBlock from './components/BrowserCodeBlock.mdx';
import BrowserCombinedCodeBlock from './components/BrowserCombinedCodeBlock.mdx';

In the [previous guide](create_a_test) we created our first browser test for [TodoMVC](http://todomvc.com/examples/react). We will now run our test locally to verify that it works.

Expand All @@ -22,6 +22,8 @@ npx qawolf test myFirstTest

A Chromium browser will open and the test will run using the [Jest](https://jestjs.io) test runner. See the video below for an example:

TODOUPDATEVIDEO

<video controls title="run a test locally" width="100%">
<source
src="https://storage.googleapis.com/docs.qawolf.com/guides/run_test_locally.mp4"
Expand Down

0 comments on commit c8182c1

Please sign in to comment.