Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
71222a9
Improve Playwright Check Suites docs
stefanjudis Oct 2, 2025
a0e0228
Update detect/synthetic-monitoring/playwright-checks/test-organizatio…
stefanjudis Oct 2, 2025
8a0a9b5
Update detect/synthetic-monitoring/playwright-checks/test-organizatio…
stefanjudis Oct 2, 2025
6cac8ef
Update detect/synthetic-monitoring/playwright-checks/test-organizatio…
stefanjudis Oct 2, 2025
0dae313
Update detect/synthetic-monitoring/playwright-checks/test-organizatio…
stefanjudis Oct 2, 2025
2c65c23
Update detect/synthetic-monitoring/playwright-checks/test-organizatio…
stefanjudis Oct 2, 2025
57afccf
Update detect/synthetic-monitoring/playwright-checks/configuration.mdx
stefanjudis Oct 2, 2025
3ecabad
Update detect/synthetic-monitoring/playwright-checks/test-organizatio…
stefanjudis Oct 2, 2025
65c652c
Update detect/synthetic-monitoring/playwright-checks/test-organizatio…
stefanjudis Oct 2, 2025
9acb7a8
Fix typos
stefanjudis Oct 2, 2025
6e7a038
Update detect/synthetic-monitoring/playwright-checks/configuration.mdx
stefanjudis Oct 6, 2025
6684e6c
Update detect/synthetic-monitoring/playwright-checks/configuration.mdx
stefanjudis Oct 6, 2025
a42cdb7
Update detect/synthetic-monitoring/playwright-checks/configuration.mdx
stefanjudis Oct 6, 2025
dfcfbe0
Add note about PWN check execution times
stefanjudis Oct 6, 2025
f559132
Update detect/synthetic-monitoring/playwright-checks/configuration.mdx
stefanjudis Oct 6, 2025
b70229c
Update detect/synthetic-monitoring/playwright-checks/custom-dependenc…
stefanjudis Oct 6, 2025
41e53bc
Update detect/synthetic-monitoring/playwright-checks/custom-dependenc…
stefanjudis Oct 6, 2025
b715853
Update detect/synthetic-monitoring/playwright-checks/custom-dependenc…
stefanjudis Oct 6, 2025
e5560b1
Update detect/synthetic-monitoring/playwright-checks/overview.mdx
stefanjudis Oct 6, 2025
86f8960
Update detect/synthetic-monitoring/playwright-checks/overview.mdx
stefanjudis Oct 6, 2025
ff8533a
Adjust pwTags and pwProjects wording
stefanjudis Oct 6, 2025
54944c8
Adjust PWN code snippet
stefanjudis Oct 6, 2025
6cf6db9
Update detect/synthetic-monitoring/playwright-checks/configuration.mdx
stefanjudis Oct 6, 2025
c838320
Update detect/synthetic-monitoring/playwright-checks/configuration.mdx
stefanjudis Oct 6, 2025
efba843
Update screenshot
stefanjudis Oct 6, 2025
0eb700d
Merge branch 'pwn-ga-launch-overhaul' of github.com:checkly/docs into…
stefanjudis Oct 6, 2025
76eda1d
Update detect/synthetic-monitoring/playwright-checks/test-organizatio…
stefanjudis Oct 6, 2025
9cd1c0f
Update detect/synthetic-monitoring/playwright-checks/test-organizatio…
stefanjudis Oct 6, 2025
ddfa10f
Update detect/synthetic-monitoring/playwright-checks/test-organizatio…
stefanjudis Oct 6, 2025
f157094
Update detect/synthetic-monitoring/playwright-checks/test-organizatio…
stefanjudis Oct 6, 2025
f0c2a82
Update detect/synthetic-monitoring/playwright-checks/test-organizatio…
stefanjudis Oct 6, 2025
de7f981
Update detect/synthetic-monitoring/playwright-checks/test-organizatio…
stefanjudis Oct 6, 2025
f3acd44
Update detect/synthetic-monitoring/playwright-checks/test-organizatio…
stefanjudis Oct 6, 2025
48f2143
Update PWN docs
stefanjudis Oct 6, 2025
ee8dcb6
Merge branch 'pwn-ga-launch-overhaul' of github.com:checkly/docs into…
stefanjudis Oct 6, 2025
138e422
Update wording in PWN docs
stefanjudis Oct 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 6 additions & 11 deletions detect/synthetic-monitoring/playwright-checks/add-to-group.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@ You can define a new group, or associate a Playwright Check Suite to an existing

### 1. Create a group for your checks

To define a new group, create a group file, for example `website-group.check.ts`.
Create a new file to create and configure your new group.

```typescript
```typescript website-group.check.ts
import { CheckGroup } from 'checkly/constructs'

export const myGroup = new CheckGroup('production-group', {
Expand All @@ -39,17 +39,15 @@ export const myGroup = new CheckGroup('production-group', {
```

<Tip>
Learn more about [using groups](/cli/constructs-reference/#checkgroup) to unify checks in Checkly.
Learn more about [using groups](/constructs/check-group-v2) to unify checks in Checkly.
</Tip>

### 2. Associate the group to the check

When specifying your Playwright Check Suite, you can reference the new or existing group, using its name:

```typescript
import { defineConfig } from 'checkly'

const config = defineConfig({
```typescript checkly.config.ts highlight={13}
export default defineConfig({
logicalId: 'checkly-website',
projectName: 'checkly-website',
checks: {
Expand All @@ -65,9 +63,6 @@ const config = defineConfig({
},
],
},
cli: {
runLocation: 'us-east-1',
},
})

export default config
Expand All @@ -80,4 +75,4 @@ npx checkly deploy --preview # confirm what will be deployed
npx checkly deploy # deploy
```

You can now see your Playwright Check Suite in your group.
You can now see your Playwright Check Suite assigned to your group.
125 changes: 68 additions & 57 deletions detect/synthetic-monitoring/playwright-checks/configuration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,17 @@ import PlaywrightCheckSuiteBetaNote from '/snippets/playwright-check-suite-beta-

<PlaywrightCheckSuiteBetaNote />

Use the `checkly.config.ts/js` file to define your Playwright Check Suite.
Use the `checkly.config.ts/js` file to define your Playwright Check Suite. Each Playwright Check Suite can be connected to references in your `playwright.config.ts/js` file.

Each Playwright Check Suite can be connected to references in your `playwright.config.ts/js` file.

During the Beta, **a Playwright Check Suite can run up to 20 minutes**. This limit can be adjusted based on feedback.
<Info>
**A Playwright Check Suite can run up to 15 minutes**. [Please contact us in the Checkly Web App](https://app.checklyhq.com/?support=true) or get in touch with your account executive if you're interested in longer runs.
</Info>

## Playwright Check Suite definition

To add Playwright Check Suites to your Checkly monitoring setup, specify the path to your `playwright.config.ts/js` and add a new `playwrightChecks` property to the existing `checks` configuration in your `checkly.config.ts/js`.

```typescript
```typescript checkly.config.ts highlight={8-14}
import { defineConfig } from 'checkly'
import { Frequency } from 'checkly/constructs'

Expand All @@ -29,11 +29,8 @@ export default defineConfig({
playwrightConfigPath: './playwright.config.ts',
playwrightChecks: [
{
name: 'critical-tagged',
pwTags: 'critical',
pwProjects: 'chromium',
frequency: Frequency.EVERY_1M,
locations: ['us-east-1', 'eu-west-1','eu-central-1', 'ap-south-1'],
name: 'all-pwt-tests',
logicalId: 'all-tests',
}
],
},
Expand All @@ -49,41 +46,48 @@ Other available properties like `frequency`, `alertChannels` or `locations` are

## Playwright references

The Checkly infrastructure will run and deploy all your existing Playwright tests (similar to what `npx playwright test` runs) as monitors if you don't reference Playwright projects or tags.
Without limiting and selecting specific tests or Playwright projects, the Checkly infrastructure will run and deploy **all your existing Playwright tests** (similar to what `npx playwright test` runs) as monitors.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯 for clarity


Specify which tests should become part of global end-to-end monitoring by defining these properties:

* `pwProjects` --- select existing project names from your Playwright configuration to create a Playwright Check Suite.
* `pwProjects`: select an existing project by name from your Playwright configuration to create a Playwright Check Suite.

* `pwTags` --- select test tags that will be grouped into a Playwright Check Suite.
* `pwTags`: select tagged tests that will be grouped into a Playwright Check Suite.

You can combine `pwTags` and `pwProjects` to generate your check suite, too.

For example:

```typescript
```typescript checkly.config.ts highlight={11, 20}
export default defineConfig({
// ...
checks: {
playwrightConfigPath: './playwright.config.ts',
playwrightChecks: [
{
/**
* Run critical tagged tests in Chromium
* every minute from 4 locations.
*/
name: 'critical-tagged',
pwTags: '@critical',
pwProjects: 'chromium',
frequency: Frequency.EVERY_1M,
locations: ['us-east-1', 'eu-west-1','eu-central-1', 'ap-south-1'],
name: "Marketing Environment",
logicalId: "environment-marketing-suite",
// Run the `environment-marketing` Playwright project
// in two locations every hour
pwProjects: ["environment-marketing"],
frequency: Frequency.EVERY_1H,
locations: ["us-west-1", "eu-west-2"],
},
{
name: "Critical production tests",
logicalId: "critical-tests",
// Run `@critical` tagged Playwright tests
// in three locations every ten minutes
pwTags: ["@critical"],
frequency: Frequency.EVERY_10M,
locations: ["us-west-1", "eu-west-2", "af-south-1"],
},
],
},
});
```

> We recommend using `pwTags` and `pwProjects` to group your Playwright tests into different monitoring levels with different monitoring settings. For example, critical application functionality may be monitored best with short monitoring frequencies to lower the mean time to detect (MTTD).
Learn more about [best practices to organize and structure your Playwright tests](/detect/synthetic-monitoring/playwright-checks/test-organization/) for synthetic monitoring.

## Monitoring customizations

Expand All @@ -100,7 +104,7 @@ A Playwright Check Suite inherits multiple properties from [the abstract `Check`
- `retryStrategy`
- `alertEscalationPolicy`

Check [the reference documentation](/cli/constructs-reference/#check) for more information on these settings.
Check [the reference documentation](/constructs/project#param-checks-playwright-checks) for more information on these settings.

Additionally, Playwright Check Suites provide specific configuration for your Playwright monitoring.

Expand All @@ -110,36 +114,43 @@ Additionally, Playwright Check Suites provide specific configuration for your Pl

* `groupName:` The group this check belongs to.

```typescript
checks: {
playwrightConfigPath: './playwright.config.ts', // specify your config file
playwrightChecks: [
{
/**
* Run `@e2e` tagged tests across browsers
* every 5 minute from 4 locations.
*/
// Human readable name
name: 'E2E test suite',
// Reference id
logicalId: 'e2e-test-suite',
// Playwright project references defined in `playwright.config`
pwProjects: ['chromium', 'firefox', 'webkit'],
// Playwright tags defined in `spec` files
pwTags: '@e2e',
// Override default dependencies install command
installCommand: 'npm install --dev',
// Override the default test command
testCommand: 'npx playwright test --grep@checkly --config=playwright.foo.config.ts',
// Activate the check so that it runs on a schedule, true by default
activated: true,
// Mute the check so that it doesn't send alerts
muted: false,
// Add a check to a group
groupName: 'production-group',
frequency: Frequency.EVERY_5M,
locations: ['us-east-1', 'eu-west-1','eu-central-1', 'ap-south-1'],
}
]
},
```typescript checkly.config.ts
export default defineConfig({
// Shared Checkly configuration (scheduling, locations, etc.)
// ...
checks: {
// Specify the relative path to your config file
playwrightConfigPath: './playwright.config.ts',
playwrightChecks: [
{
/**
* Run `@e2e` tagged tests across browsers
* every 5 minute from 4 locations.
*/
// Human readable name
name: 'E2E test suite',
// Reference id
logicalId: 'e2e-test-suite',
// Playwright project references defined in `playwright.config`
pwProjects: ['chromium', 'firefox', 'webkit'],
// Playwright tags defined in `spec` files
pwTags: '@e2e',
// Override default dependencies install command
installCommand: 'npm install --dev',
// Append to the resulting test command, respects your pwTags and pwProjects filters.
testCommand: 'npx playwright test --trace=on',
// Activate the check so that it runs on a schedule, true by default
activated: true,
// Mute the check so that it doesn't send alerts
muted: false,
// Add a check to a group
groupName: 'production-group',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
groupName: 'production-group',
group: 'websiteGroup',

We've updated this one to be just group, like it works everywhere else, the one you specify in your construct to create the group:

import { websiteGroup } from './src/__checks__/website-group.check

It'll be released in the next CLI version. groupName is deprecated but still supported, so nothing will break for people with this setup. See: checkly/checkly-cli#1149

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MariadeAnton Good stuff. Is there a docs PR in the making too?

I'd rather go with a single PR changing this guide and the constructs docs in the same go than having this guide and the constructs docs tell different things?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No docs PR in the making yet, just CLI WIP

// Specify a monitoring frequency
frequency: Frequency.EVERY_5M,
// Define locations to monitor from
locations: ['us-east-1', 'eu-west-1','eu-central-1', 'ap-south-1'],
}
]
},
}
```
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,9 @@ Whether you use [npm](https://www.npmjs.com/), [yarn](https://yarnpkg.com/), or
}
```

**By default, the Checkly infrastructure installs dependencies via npm.**
<Note>
By default, the Checkly infrastructure installs dependencies via npm.
</Note>

However, if an alternative lock file (`pnpm-lock.json` or `yarn.lock`) is discovered, Checkly uses the matching package manager to install the project dependencies.

Expand All @@ -32,7 +34,9 @@ However, if an alternative lock file (`pnpm-lock.json` or `yarn.lock`) is discov
| `pnpm-lock.json` | pnpm |
| `yarn.lock` | yarn |

> **For `pnpm` users:** if you define the same dependency in `dependencies` and `devDependencies`, please use a custom `installCommand` in your `checkly.config.ts|js` to register your package as a development dependency: `pnpm install --dev additional-package`
<Warning>
**For `pnpm` users:** if you define the same dependency in `dependencies` and `devDependencies`, please use a custom `installCommand` in your `checkly.config.ts|js` to ensure all required dependenies are installed: `pnpm install`
</Warning>

## Using private dependencies

Expand Down Expand Up @@ -87,11 +91,11 @@ registry=https://yourcompany.jfrog.io/artifactory/api/npm/yourcompany.jfrog.io/a

### Include custom configuration files

To make the Checkly CLI and infrastructure aware of a custom package manager configuration for private packages, specify the additional files in the `checks.include` property in your `checkly.config.ts|js`.
To make the Checkly CLI and infrastructure aware of your configuration for private packages, specify the additional files in the `checks.include` property in your `checkly.config.ts`.

Checkly CLI commands like `test` or `deploy` will then bundle and upload these files so they are available in the Checkly infrastructure when running essential commands like `install`.
Checkly CLI commands like `test` or `deploy` will bundle and upload these files so they are available in the Checkly infrastructure when running essential commands like `install`; making sure your private dependencies are installed.

```typescript
```typescript checkly.config.ts
export default defineConfig({
checks: {
playwrightConfigPath: './playwright.config.ts',
Expand Down
53 changes: 36 additions & 17 deletions detect/synthetic-monitoring/playwright-checks/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,17 @@ import PlaywrightCheckSuiteBetaNote from '/snippets/playwright-check-suite-beta-

## What are Playwright Checks?

Playwright Checks enable you to run entire Playwright Suites or Projects as both pre-production tests and global production monitors, without rewriting your tests and configuration.
Playwright Check Suites enable you to run entire Playwright end-to-end test suites, Playwright projects or tagged tests as pre-production tests and global production monitors, without rewriting your tests and configuration.

Unlike [Browser Checks](detect/synthetic-monitoring/browser-checks/overview), Playwright Checks are not limited to a single test spec running in an async browser session and our preinstalled runtime dependencies. Playwright Checks enable you to run Playwright projects complete with all of your required dependencies, private packages, storage, data, and more. It's a complete native way to run Playwright in production, enabling you to detect issues with your applications or services even in the most complex scenarios.
Unlike [Browser Checks](detect/synthetic-monitoring/browser-checks/overview), Playwright Check Suites are not limited to a single test spec and our preinstalled runtime dependencies. Playwright Check Suites enable you to run Playwright projects complete with all of your required dependencies, private packages, storage, data, and more, and enable you to detect issues with your applications or services even in the most complex scenarios.

Playwright Checks support for the full Playwright API and ecosystem, meaning you can use your existing tests and `playwright.config.ts` files as is.
<Frame>
![Playwright Check Suites test result](/images/pwn-test-result.png)
</Frame>

**Playwright Checks are perfect for:**
**Playwright Check Suites are the native way to run Playwright in production** and support the full Playwright API and ecosystem, meaning you can use your existing tests and `playwright.config.ts` files as is.

**Playwright Check Suites are perfect for:**
- Converting existing E2E tests into monitoring
- Testing complex user workflows across multiple browsers
- Monitoring critical business processes
Expand All @@ -31,7 +35,7 @@ Playwright Checks support for the full Playwright API and ecosystem, meaning you

<Steps>
<Step title="1. Test Discovery">
Playwright Checks analyze your Playwright configuration and test files to understand your existing test structure.
Playwright Check Suites analyze your Playwright configuration and test files to understand your existing test structure.
</Step>

<Step title="2. Selective Monitoring">
Expand Down Expand Up @@ -59,21 +63,36 @@ Receive notifications when tests fail or performance degrades, keeping your team

Control which tests become monitoring checks using:

- **Tags**: Select tests marked with specific tags (e.g., `@critical`, `@smoke`)
- **Projects**: Choose browser configurations from your Playwright config
- **Tags**: Select tests marked with specific tags (e.g., `@critical`, `@smoke`)
- **Combinations**: Mix tags and projects for precise test selection

```typescript
// Example: Monitor critical tests in Chromium
{
name: 'Critical User Flows',
pwTags: '@critical',
pwProjects: 'chromium',
frequency: Frequency.EVERY_1M,
locations: ['us-east-1', 'eu-west-1']
}
```typescript checkly.config.ts
export default defineConfig({
checks: {
playwrightChecks: [
{
name: "Marketing Environment",
logicalId: "environment-marketing-suite",
// Select the tests defined in
// the `environment-marketing` Playwright project
pwProjects: ["environment-marketing"],
},
{
name: "@critical tests",
logicalId: "critical tests",
// Select tests tagged as `@critical`
pwTags: ["@critical"],
},
],
},
})
```

<Info>
The Playwright Check Suite runner will use the `pwProjects` and `pwTags` values to configure [the native Playwright Test CLI run](https://playwright.dev/docs/test-cli). `pwProjects` reflects the `--project` and `pwTags` the `--grep` CLI options.
</Info>

## Monitoring Configuration

Customize monitoring behavior with:
Expand All @@ -89,11 +108,11 @@ Customize monitoring behavior with:
Ready to turn your Playwright tests into monitoring? Start with our quickstart guide to get running in 5 minutes.

<CardGroup cols={2}>
<Card title="Quickstart Guide" href="/detect/synthetic-monitoring/playwright-checks/quickstart">
<Card title="Quickstart Guide" href="/quickstarts/playwright-check">
Get from zero to deployed monitoring in 5 minutes
</Card>

<Card title="CLI Reference" href="/detect/synthetic-monitoring/playwright-checks/reference">
<Card title="Construct Reference" href="/constructs/playwright-check/">
Complete configuration options and examples
</Card>

Expand Down
Loading