From cb4c224fa7a7a652abfe5a36aabd88877391d029 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Olender?= <92638966+TC-MO@users.noreply.github.com> Date: Fri, 23 Aug 2024 02:23:57 +0200 Subject: [PATCH 1/5] docs: rewrite automated tests streamline prose add code tabs to tests scenarios move doc out of deployment section into development section --- .../{deployment => }/automated_tests.md | 58 ++++++++++++------- 1 file changed, 36 insertions(+), 22 deletions(-) rename sources/platform/actors/development/{deployment => }/automated_tests.md (53%) diff --git a/sources/platform/actors/development/deployment/automated_tests.md b/sources/platform/actors/development/automated_tests.md similarity index 53% rename from sources/platform/actors/development/deployment/automated_tests.md rename to sources/platform/actors/development/automated_tests.md index b18dc515a9..2779fd04d8 100644 --- a/sources/platform/actors/development/deployment/automated_tests.md +++ b/sources/platform/actors/development/automated_tests.md @@ -1,47 +1,55 @@ --- title: Automated tests description: Learn how to automate ongoing testing and make sure your Actors perform over time. See code examples for configuring the Actor Testing Actor. -slug: /actors/development/deployment/automated-tests -sidebar_position: 3 +slug: /actors/development/automated-tests +sidebar_position: 9 --- **Learn how to automate ongoing testing and make sure your Actors perform over time. See code examples for configuring the Actor Testing Actor.** +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + --- -You should make sure your [Actors](../../index.mdx) are well-maintained. You might not always get feedback from your users. Therefore, it is crucial that you periodically check if your Actors work as expected. You can do this using our [monitoring suite](https://apify.com/apify/monitoring) or by setting up daily runs of the **Actor Testing** ([pocesar/actor-testing](https://apify.com/pocesar/actor-testing)) tool. +# Automated tests for Actors -The monitoring suite is sufficient for most scenarios and includes automated alerts. [See more information](https://apify.com/apify/monitoring) on the suite's page or [check out our tutorials](../../../monitoring/index.md). +Automated testing is crucial for maintaining the reliaability and performance of your Actors over time. This guide will help you set up automated tests using the [Actor Testing Actor](https://apify.com/pocesar/actor-testing). -We recommend using the Actor Testing Actor for specific and advanced use cases. This guide will help you set it up. +## Set up automated tests -## Step-by-step guide +1. Prepare test tasks - Create 1–5 separate testing tasks for your Actor. +1. Configure Actor testing - Set up a task using the Actor Testing Actor. +1. Validate tests - Run the test task multiple times until all tests pass. +1. Schedule tests - Set up a recurring schedule for your tests. +1. Monitor results - Review and address any issues on a weekly basis. -1. Prepare 1–5 separate testing tasks for your Actor. ([See below](#set-up-tasks-you-will-test)). -2. Set up a task from the Actor Testing Actor. ([See below](#set-up-a-task-from-the-actor-testing-actor)). -3. Run the test task until all tests succeed (a few times). -4. Schedule the test to run at the frequency of your choice (recommended daily) and choose a communication channel receiving info about it ([Slack](https://apify.com/katerinahronik/slack-message) or [email](https://apify.com/apify/send-mail)). -5. Ensure you review and fix any issues on a weekly basis. +## Create test tasks -## Set up tasks you will test +Example of Actor testing tasks -![Tasks that test an Actor's configurations](./images/testing-tasks.png) +When creating test tasks: -We also advise you to test your Actor's default run—one that uses the pre-filled inputs. It is often the first task your users run, and they may be put off if it doesn't work. +* Include a test for your Actor's default configuration +* Set a low `maxItem` value to conserve credits +* For large data tests, reduce test frequency to conserve credits -Set a low `maxItem` value for your testing tasks, so that you don't burn your credit. If you need to test your Actor with a large amount of data, set the scheduler to run less frequently. +## Configure the Actor Testing Actor -## Set up a task from the Actor Testing Actor +Follow the [setup guide](https://apify.com/pocesar/actor-testing) in the Actor's README. -You can [find the setup guide](https://apify.com/pocesar/actor-testing) in the Actor's README. We recommend testing for the following scenarios. +Here are some recommended test scenarios: -Run status: + + ```js await expectAsync(runResult).toHaveStatus('SUCCEEDED'); ``` -Crash information from the log: + + + ```js await expectAsync(runResult).withLog((log) => { @@ -58,7 +66,8 @@ await expectAsync(runResult).withLog((log) => { }); ``` -Information from statistics (runtime, retries): + + ```js await expectAsync(runResult).withStatistics((stats) => { @@ -74,7 +83,8 @@ await expectAsync(runResult).withStatistics((stats) => { }); ``` -Information about and from within the [dataset](../../../storage/dataset.md): + + ```js await expectAsync(runResult).withDataset(({ dataset, info }) => { @@ -103,7 +113,8 @@ await expectAsync(runResult).withDataset(({ dataset, info }) => { }); ``` -Information about the [key-value store](../../../storage/key_value_store.md): + + ```js await expectAsync(runResult).withKeyValueStore(({ contentType }) => { @@ -117,3 +128,6 @@ await expectAsync(runResult).withKeyValueStore(({ contentType }) => { { keyName: 'apify.com-scroll_losless-comp' }, ); ``` + + + From c649912c874fc4fb435939f3d42ab3e1b2500283 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Olender?= <92638966+TC-MO@users.noreply.github.com> Date: Fri, 23 Aug 2024 11:35:20 +0200 Subject: [PATCH 2/5] fix typo --- sources/platform/actors/development/automated_tests.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sources/platform/actors/development/automated_tests.md b/sources/platform/actors/development/automated_tests.md index 2779fd04d8..33a5510a40 100644 --- a/sources/platform/actors/development/automated_tests.md +++ b/sources/platform/actors/development/automated_tests.md @@ -14,7 +14,7 @@ import TabItem from '@theme/TabItem'; # Automated tests for Actors -Automated testing is crucial for maintaining the reliaability and performance of your Actors over time. This guide will help you set up automated tests using the [Actor Testing Actor](https://apify.com/pocesar/actor-testing). +Automated testing is crucial for maintaining the reliability and performance of your Actors over time. This guide will help you set up automated tests using the [Actor Testing Actor](https://apify.com/pocesar/actor-testing). ## Set up automated tests From 70afbe2af3220bec5588732a8a51a33eb3cdf1b1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Olender?= Date: Thu, 5 Sep 2024 13:13:40 +0200 Subject: [PATCH 3/5] fix dead link --- sources/platform/actors/publishing/index.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sources/platform/actors/publishing/index.mdx b/sources/platform/actors/publishing/index.mdx index 9c07639bd0..69abc71dc7 100644 --- a/sources/platform/actors/publishing/index.mdx +++ b/sources/platform/actors/publishing/index.mdx @@ -15,7 +15,7 @@ To build a public Actor, you'll go through the following four main stages: 1. [Development](../development/index.md). 2. [Publication](./publishing/publish) and set up of [monetization](./publishing/monetize). -3. [Testing](../development/deployment/automated_tests.md). +3. [Testing](../development/automated_tests.md). 4. [Promotion](../../academy/get-most-of-actors/seo-and-promotion). While you don't necessarily have to maintain your private Actors, public Actors require a higher degree of responsibility. From af7383b37f4b19c14f6d9f9506738f7fb289d6e5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Olender?= Date: Thu, 5 Sep 2024 13:19:02 +0200 Subject: [PATCH 4/5] fix more dead links --- sources/platform/actors/publishing/index.mdx | 2 +- sources/platform/actors/publishing/testing.mdx | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/sources/platform/actors/publishing/index.mdx b/sources/platform/actors/publishing/index.mdx index 69abc71dc7..34955a77f8 100644 --- a/sources/platform/actors/publishing/index.mdx +++ b/sources/platform/actors/publishing/index.mdx @@ -22,7 +22,7 @@ While you don't necessarily have to maintain your private Actors, public Actors As the name implies, Public Actors are available to the public on [Apify Store](https://apify.com/store), which means that an unmaintained public Actor could negatively affect all the other users that depend on it for their own activities on the platform. -Public Actors are regularly submitted to [automated tests](./testing.mdx) to ensure they are functioning properly. Before making an Actor public, we recommend you reserve enough time to maintain the project (~2 hours weekly). This will ensure that your Actor maintains its long-term quality, improving your chances of successfully [monetizing your Actors](./monetize.mdx). +Public Actors are regularly submitted to [automated tests](/platform/actors/publishing/test) to ensure they are functioning properly. Before making an Actor public, we recommend you reserve enough time to maintain the project (~2 hours weekly). This will ensure that your Actor maintains its long-term quality, improving your chances of successfully [monetizing your Actors](./monetize.mdx). diff --git a/sources/platform/actors/publishing/testing.mdx b/sources/platform/actors/publishing/testing.mdx index d6e02cf16d..18e6d3994e 100644 --- a/sources/platform/actors/publishing/testing.mdx +++ b/sources/platform/actors/publishing/testing.mdx @@ -39,4 +39,4 @@ If that's the case with your Actor, please contact support at [support@apify.com You can easily implement your own tests and customize them to fit your Actor's particularities by using our public [Actor Testing](https://apify.com/pocesar/actor-testing) tool available in Apify Store. -For more information, see the [automated testing](../development/deployment/automated_tests.md) section. +For more information, see the [automated testing](../development/automated_tests.md) section. From f73bafe4c074afe16dc8760d1633b2b7358a36d3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Olender?= Date: Thu, 5 Sep 2024 13:23:27 +0200 Subject: [PATCH 5/5] fix failing tests --- sources/platform/actors/publishing/index.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sources/platform/actors/publishing/index.mdx b/sources/platform/actors/publishing/index.mdx index 34955a77f8..bc7a37b99a 100644 --- a/sources/platform/actors/publishing/index.mdx +++ b/sources/platform/actors/publishing/index.mdx @@ -34,7 +34,7 @@ If you plan on making a breaking change, please get in touch with us ahead of ti Also, pay special attention to your Actor's documentation ([README](../../academy/get-most-of-actors/actor-readme)). It should be clear, detailed, and readable. Think of the users, who might not be developers, so try to use simple, easy-to-understand language and avoid technical jargon. -Ensure periodic testing. You can either do it yourself or [set up automatic testing](../development/deployment/automated_tests.md) / [monitoring](https://apify.com/apify/monitoring). Try to prevent your users from coming to you with the issues. +Ensure periodic testing. You can either do it yourself or [set up automatic testing](../development/automated_tests.md) / [monitoring](https://apify.com/apify/monitoring). Try to prevent your users from coming to you with the issues. ## Inspiration