Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,47 +1,55 @@
---
title: Automated tests
description: Learn how to automate ongoing testing and make sure your Actors perform over time. See code examples for configuring the Actor Testing Actor.
slug: /actors/development/deployment/automated-tests
sidebar_position: 3
slug: /actors/development/automated-tests
sidebar_position: 9
---

**Learn how to automate ongoing testing and make sure your Actors perform over time. See code examples for configuring the Actor Testing Actor.**

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

---

You should make sure your [Actors](../../index.mdx) are well-maintained. You might not always get feedback from your users. Therefore, it is crucial that you periodically check if your Actors work as expected. You can do this using our [monitoring suite](https://apify.com/apify/monitoring) or by setting up daily runs of the **Actor Testing** ([pocesar/actor-testing](https://apify.com/pocesar/actor-testing)) tool.
# Automated tests for Actors

The monitoring suite is sufficient for most scenarios and includes automated alerts. [See more information](https://apify.com/apify/monitoring) on the suite's page or [check out our tutorials](../../../monitoring/index.md).
Automated testing is crucial for maintaining the reliability and performance of your Actors over time. This guide will help you set up automated tests using the [Actor Testing Actor](https://apify.com/pocesar/actor-testing).

We recommend using the Actor Testing Actor for specific and advanced use cases. This guide will help you set it up.
## Set up automated tests

## Step-by-step guide
1. Prepare test tasks - Create 1–5 separate testing tasks for your Actor.
1. Configure Actor testing - Set up a task using the Actor Testing Actor.
1. Validate tests - Run the test task multiple times until all tests pass.
1. Schedule tests - Set up a recurring schedule for your tests.
1. Monitor results - Review and address any issues on a weekly basis.

1. Prepare 1–5 separate testing tasks for your Actor. ([See below](#set-up-tasks-you-will-test)).
2. Set up a task from the Actor Testing Actor. ([See below](#set-up-a-task-from-the-actor-testing-actor)).
3. Run the test task until all tests succeed (a few times).
4. Schedule the test to run at the frequency of your choice (recommended daily) and choose a communication channel receiving info about it ([Slack](https://apify.com/katerinahronik/slack-message) or [email](https://apify.com/apify/send-mail)).
5. Ensure you review and fix any issues on a weekly basis.
## Create test tasks

## Set up tasks you will test
Example of Actor testing tasks

![Tasks that test an Actor's configurations](./images/testing-tasks.png)
When creating test tasks:

We also advise you to test your Actor's default run—one that uses the pre-filled inputs. It is often the first task your users run, and they may be put off if it doesn't work.
* Include a test for your Actor's default configuration
* Set a low `maxItem` value to conserve credits
* For large data tests, reduce test frequency to conserve credits

Set a low `maxItem` value for your testing tasks, so that you don't burn your credit. If you need to test your Actor with a large amount of data, set the scheduler to run less frequently.
## Configure the Actor Testing Actor

## Set up a task from the Actor Testing Actor
Follow the [setup guide](https://apify.com/pocesar/actor-testing) in the Actor's README.

You can [find the setup guide](https://apify.com/pocesar/actor-testing) in the Actor's README. We recommend testing for the following scenarios.
Here are some recommended test scenarios:

Run status:
<Tabs groupId="main">
<TabItem value="Run status" label="Run status">

```js
await expectAsync(runResult).toHaveStatus('SUCCEEDED');
```

Crash information from the log:
</TabItem>
<TabItem value="Crash information from the log" label="Crash information from the log">


```js
await expectAsync(runResult).withLog((log) => {
Expand All @@ -58,7 +66,8 @@ await expectAsync(runResult).withLog((log) => {
});
```

Information from statistics (runtime, retries):
</TabItem>
<TabItem value="Information from statistics (runtime, retries)" label="Information from statistics (runtime, retries)">

```js
await expectAsync(runResult).withStatistics((stats) => {
Expand All @@ -74,7 +83,8 @@ await expectAsync(runResult).withStatistics((stats) => {
});
```

Information about and from within the [dataset](../../../storage/dataset.md):
</TabItem>
<TabItem value="Information about and from within the dataset" label="Information about and from within the dataset">

```js
await expectAsync(runResult).withDataset(({ dataset, info }) => {
Expand Down Expand Up @@ -103,7 +113,8 @@ await expectAsync(runResult).withDataset(({ dataset, info }) => {
});
```

Information about the [key-value store](../../../storage/key_value_store.md):
</TabItem>
<TabItem value="Information about the key-value store" label="Information about the key-value store">

```js
await expectAsync(runResult).withKeyValueStore(({ contentType }) => {
Expand All @@ -117,3 +128,6 @@ await expectAsync(runResult).withKeyValueStore(({ contentType }) => {
{ keyName: 'apify.com-scroll_losless-comp' },
);
```

</TabItem>
</Tabs>
6 changes: 3 additions & 3 deletions sources/platform/actors/publishing/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,14 @@ To build a public Actor, you'll go through the following four main stages:

1. [Development](../development/index.md).
2. [Publication](./publishing/publish) and set up of [monetization](./publishing/monetize).
3. [Testing](../development/deployment/automated_tests.md).
3. [Testing](../development/automated_tests.md).
4. [Promotion](../../academy/get-most-of-actors/seo-and-promotion).

While you don't necessarily have to maintain your private Actors, public Actors require a higher degree of responsibility.

As the name implies, Public Actors are available to the public on [Apify Store](https://apify.com/store), which means that an unmaintained public Actor could negatively affect all the other users that depend on it for their own activities on the platform.

Public Actors are regularly submitted to [automated tests](./testing.mdx) to ensure they are functioning properly. Before making an Actor public, we recommend you reserve enough time to maintain the project (~2 hours weekly). This will ensure that your Actor maintains its long-term quality, improving your chances of successfully [monetizing your Actors](./monetize.mdx).
Public Actors are regularly submitted to [automated tests](/platform/actors/publishing/test) to ensure they are functioning properly. Before making an Actor public, we recommend you reserve enough time to maintain the project (~2 hours weekly). This will ensure that your Actor maintains its long-term quality, improving your chances of successfully [monetizing your Actors](./monetize.mdx).

<img src={require("./images/apify-store.png").default} title="Apify Store" width="60%" />

Expand All @@ -34,7 +34,7 @@ If you plan on making a breaking change, please get in touch with us ahead of ti

Also, pay special attention to your Actor's documentation ([README](../../academy/get-most-of-actors/actor-readme)). It should be clear, detailed, and readable. Think of the users, who might not be developers, so try to use simple, easy-to-understand language and avoid technical jargon.

Ensure periodic testing. You can either do it yourself or [set up automatic testing](../development/deployment/automated_tests.md) / [monitoring](https://apify.com/apify/monitoring). Try to prevent your users from coming to you with the issues.
Ensure periodic testing. You can either do it yourself or [set up automatic testing](../development/automated_tests.md) / [monitoring](https://apify.com/apify/monitoring). Try to prevent your users from coming to you with the issues.

## Inspiration

Expand Down
2 changes: 1 addition & 1 deletion sources/platform/actors/publishing/testing.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -39,4 +39,4 @@ If that's the case with your Actor, please contact support at [support@apify.com

You can easily implement your own tests and customize them to fit your Actor's particularities
by using our public [Actor Testing](https://apify.com/pocesar/actor-testing) tool available in Apify Store.
For more information, see the [automated testing](../development/deployment/automated_tests.md) section.
For more information, see the [automated testing](../development/automated_tests.md) section.