diff --git a/docs/actor/build.md b/docs/actor/build.md index 5da138a59b..3c970f9b85 100644 --- a/docs/actor/build.md +++ b/docs/actor/build.md @@ -6,9 +6,9 @@ title: Build Before the actor can be run, it first needs to be built. The build effectively creates a snapshot of a specific version of the actor's settings such as the [Source code]({{@link actor/source_code.md}}) and [Environment variables]({{@link actor/run.md}}#run-env-vars), and creates a Docker image that contains everything the actor needs for its run, including necessary NPM packages, web browsers, etc. -Each build is assigned a unique build number of the form `MAJOR.MINOR.BUILD` (e.g. `1.2.345`), where `MAJOR.MINOR` corresponds to the actor version number (see [Versions](#versions)) and `BUILD` is an automatically-incremented number starting at `1`. +Each build is assigned a unique build number of the form `MAJOR.MINOR.BUILD` (e.g. `1.2.345`), where `MAJOR.MINOR` corresponds to the actor version number (see [Versioning]({{@link actor/source_code.md#versioning}})) and `BUILD` is an automatically-incremented number starting at `1`. -By default, the build has a timeout of 300 seconds and consumes 1024 MB of memory from the user's memory limit. See the [Resource limits](#resource-limits) section for more details. +By default, the build has a timeout of 300 seconds and consumes 1024 MB of memory from the user's memory limit. See the [Resource limits]({{@link actor/run.md#resource-limits}}) section for more details. ### [](#build-tags)Tags diff --git a/docs/actor/examples.md b/docs/actor/examples.md index a45e244250..b08ef99a67 100644 --- a/docs/actor/examples.md +++ b/docs/actor/examples.md @@ -4,7 +4,7 @@ title: Examples ## [](#examples)Examples -This section provides examples of actors using various features of the Apify platform. All these examples and many more are also available in the [store](/store?type=acts&search=user%3Aapify%20example). +This section provides examples of actors using various features of the Apify platform. All these examples and many more are also available in the [store](https://apify.com/store?type=acts&search=user%3Aapify%20example). ### [](#examples-puppeteer)Puppeteer @@ -52,11 +52,11 @@ This example demonstrates how to use headless Chrome with Puppeteer to open a we console.log(`- https://api.apify.com/v2/key-value-stores/${storeId}/records/page.pdf`); }); -The code above uses the [`launchPuppeteer()`](https://sdk.apify.com/docs/api/apify#module_Apify.launchPuppeteer) function provided by the [`apify`](https://sdk.apify.com/) NPM package. The function launches Puppeteer with several settings that enable it to run in an actor. Note that the actor needs to have **Base image** set to [Node.js 10 + Puppeteer on Debian](#base-images) in order to run Puppeteer. +The code above uses the [`launchPuppeteer()`](https://sdk.apify.com/docs/api/apify#module_Apify.launchPuppeteer) function provided by the [`apify`](https://sdk.apify.com/) NPM package. The function launches Puppeteer with several settings that enable it to run in an actor. Note that the actor needs to have **Base image** set to [Node.js 10 + Puppeteer on Debian]({{@link actor/build.md#base-images}}) in order to run Puppeteer. ### [](#examples-dockerfile)Custom Dockerfile -This example demonstrates how to create an actor written in PHP using a custom Dockerfile. For more information, see the [Custom Dockerfile](#custom-dockerfile) section. The Dockerfile is based on the [`php:7.0-cli` _open_in_new_](https://hub.docker.com/_/php/) Docker image that contains everything needed to run PHP in a terminal. +This example demonstrates how to create an actor written in PHP using a custom Dockerfile. For more information, see the [Custom Dockerfile]({{@link actor/source_code.md#custom-dockerfile}}) section. The Dockerfile is based on the [`php:7.0-cli` _open_in_new_](https://hub.docker.com/_/php/) Docker image that contains everything needed to run PHP in a terminal. `Dockerfile` contains only two commands. The first copies source code into the container and the second executes `main.php`. @@ -83,7 +83,7 @@ The actor can be found in the Apify store as [apify/example-php](https://apify.c ### [](#examples-state-persistence)State persistence -This actor demonstrates how to persist a state, so that on restart the actor can continue where it left off. For more information, see the [State persistence](#state-persistence) section. The actor simply counts from one up. In each run it prints one number. Its state (counter position) is stored in a named [key-value store](/docs/storage#key-value-store) called `example-counter`. You will find it in the [Storage](https://my.apify.com/key-value-stores) section of the app after you run the actor. +This actor demonstrates how to persist a state, so that on restart the actor can continue where it left off. For more information, see the [State persistence]({{@link actor/run.md#state-persistence}}) section. The actor simply counts from one up. In each run it prints one number. Its state (counter position) is stored in a named [key-value store]({{@link storage/key_value_store.md}}) called `example-counter`. You will find it in the [Storage](https://my.apify.com/key-value-stores) section of the app after you run the actor. The actor can be found in the Apify store as [apify/example-counter](https://apify.com/apify/example-counter). diff --git a/docs/actor/limits.md b/docs/actor/limits.md index 9b8f859b7b..c522260a72 100644 --- a/docs/actor/limits.md +++ b/docs/actor/limits.md @@ -4,7 +4,7 @@ title: Limits ## [](#limits)Limits -This section describes various resource limits of the Apify platform. Do you need to increase any of them? Please [contact us](/contact). +This section describes various resource limits of the Apify platform. Do you need to increase any of them? Please [contact us](https://apify.com/contact). |Description|Value| |--- |--- | diff --git a/docs/actor/publishing.md b/docs/actor/publishing.md index faaa410f15..a9a85d280a 100644 --- a/docs/actor/publishing.md +++ b/docs/actor/publishing.md @@ -4,10 +4,10 @@ title: Publishing ## [](#publishing)Publishing -Actors can be private or public. Private actors can only be accessed and started by their owner, while public actor are shown in the [store](/store) and can be run by anyone. Each public actor has a globally unique identifier that consists of the owner's username and the actor name, e.g. [apify/hello-world](https://apify.com/apify/hello-world). +Actors can be private or public. Private actors can only be accessed and started by their owner, while public actor are shown in the [store](https://apify.com/store) and can be run by anyone. Each public actor has a globally unique identifier that consists of the owner's username and the actor name, e.g. [apify/hello-world](https://apify.com/apify/hello-world). To publish your actor, go to **Settings → Permissions** on the actor detail page and click the **Publish** button. You'll need to have a username set. This can be done on the [Profile](https://my.apify.com/account#/profile) page. -The short actor description shown in the store is taken from **Settings → Description**. Additionally, if the actor's source code is hosted in a [Git repository](#source-git-repo), [Zip file](#source-tarball) or [GitHub Gist](#source-github-gist), you can add a long description in [Markdown _open_in_new_](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet) language to the `README.md` or `README` files in the root of the source code directory. To see an example of how this looks, go to [apify/web-scraper](https://apify.com/apify/web-scraper). +The short actor description shown in the store is taken from **Settings → Description**. Additionally, if the actor's source code is hosted in a [Git repository]({{@link actor/source_code.md#source-git-repo}}), [Zip file]({{@link actor/source_code.md#source-tarball}}) or [GitHub Gist]({{@link actor/source_code.md#source-github-gist}}), you can add a long description in [Markdown _open_in_new_](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet) language to the `README.md` or `README` files in the root of the source code directory. To see an example of how this looks, go to [apify/web-scraper](https://apify.com/apify/web-scraper). **IMPORTANT:** Note that if your actor is public and used by other people, its usage is not charged towards your account. The user running the actor is always the one who pays for the computational resources consumed by an actor's execution. diff --git a/docs/actor/quick_start.md b/docs/actor/quick_start.md index b7d8719b25..a08cb966a3 100644 --- a/docs/actor/quick_start.md +++ b/docs/actor/quick_start.md @@ -19,7 +19,7 @@ Click **Quick run** to build and run your actor. After the run is finished you s Congratulations, you have successfully created and run your first actor! -Let's try something little more complicated. We will change the actor to accept input and generate output (see [Input and output](#input-output) for more details): +Let's try something little more complicated. We will change the actor to accept input and generate output (see [Input and output]({{@link actor/run.md#input-output}}) for more details): const Apify = require('apify'); @@ -54,4 +54,4 @@ For more complicated actors, you'll probably prefer to host the source code on G 4. Paste the Git repo link to **Git URL**, save changes and build your actor. 5. That's it, now you can develop your actor locally on your computer and run it in the Apify cloud! -For more information, go to the [Git repository](#source-git-repo) section. +For more information, go to the [Git repository]({{@link actor/source_code.md#source-git-repo}}) section. diff --git a/docs/actor/run.md b/docs/actor/run.md index 246413fc16..d4cce94cc7 100644 --- a/docs/actor/run.md +++ b/docs/actor/run.md @@ -21,11 +21,11 @@ The following table describes the run settings: The owner of the actor can specify default values for all the above settings in the **Default run configuration** section in the app. If the actor caller does not specify a particular setting, the default value is used. -The actor can also be invoked using the Apify API by sending a HTTP POST request to the [Run actor](/docs/api/v2/#/reference/actors/run-collection/run-actor) API endpoint, such as: +The actor can also be invoked using the Apify API by sending a HTTP POST request to the [Run actor](https://apify.com/docs/api/v2/#/reference/actors/run-collection/run-actor) API endpoint, such as: https://api.apify.com/v2/acts/apify~hello-world/runs?token= -The actor's input and its content type can be passed as a payload of the POST request and additional options can be specified using URL query parameters. For more details, see the [Run actor](/docs/api/v2/#/reference/actors/run-collection/run-actor) section in the API reference. +The actor's input and its content type can be passed as a payload of the POST request and additional options can be specified using URL query parameters. For more details, see the [Run actor](https://apify.com/docs/api/v2/#/reference/actors/run-collection/run-actor) section in the API reference. Actors can also be invoked programmatically from other actors using the [`call()`](https://sdk.apify.com/docs/api/apify#module_Apify.call) function provided by the [`apify`](https://sdk.apify.com/) NPM package. For example: @@ -34,19 +34,19 @@ Actors can also be invoked programmatically from other actors using the [`call() The newly started actor runs under the same user account as the initial actor and therefore all resources consumed are charged to the same user account. This allows more complex actors to be built using simpler actors built and owned by other users. -Internally, the `call()` function takes the user's API token from the `APIFY_TOKEN` environment variable, then it invokes the [Run actor](/docs/api/v2/#/reference/actors/run-collection/run-actor) API endpoint, waits for the actor to finish and reads its output using the [Get record](/docs/api/v2/#/reference/key-value-stores/record/get-record) API endpoint. +Internally, the `call()` function takes the user's API token from the `APIFY_TOKEN` environment variable, then it invokes the [Run actor](https://apify.com/docs/api/v2/#/reference/actors/run-collection/run-actor) API endpoint, waits for the actor to finish and reads its output using the [Get record](https://apify.com/docs/api/v2/#/reference/key-value-stores/record/get-record) API endpoint. ### [](#input-output)Input and output As demonstrated in the hello world example above, actors can accept input and generate output. Both input and output are stored in a key-value store that is created when the actor is started, under the `INPUT` and `OUTPUT` keys, respectively. Note that the actor can store other values under arbitrary keys, for example crawling results or screenshots of web pages. -The key-value store associated with the actor run can be conveniently accessed using the [`getValue()`](https://sdk.apify.com/docs/api/apify#module_Apify.getValue) and [`setValue()`](https://sdk.apify.com/docs/api/apify#module_Apify.setValue) functions provided by the `apify` NPM package. Internally, these functions read the ID of the key-value store from the `APIFY_DEFAULT_KEY_VALUE_STORE_ID` environment variable and then access the store using the Apify API. For more details about the key-value stores, go to the [Storage](#storage) section. +The key-value store associated with the actor run can be conveniently accessed using the [`getValue()`](https://sdk.apify.com/docs/api/apify#module_Apify.getValue) and [`setValue()`](https://sdk.apify.com/docs/api/apify#module_Apify.setValue) functions provided by the `apify` NPM package. Internally, these functions read the ID of the key-value store from the `APIFY_DEFAULT_KEY_VALUE_STORE_ID` environment variable and then access the store using the Apify API. For more details about the key-value stores, go to the [Storage]({{@link storage/index.md}}) section. -The input can be passed to the actor either manually in the Console or using a POST payload when running the actor using API. See [Run](#run) section for details. +The input can be passed to the actor either manually in the Console or using a POST payload when running the actor using API. See [Run]({{@link actor/run.md}}) section for details. ### [](#run-env-vars)Environment variables -Aside from [custom environment variables](#source-env-vars), the actor's process has several environment variables set to provide it with context: +Aside from [custom environment variables]({{@link actor/source_code.md#source-env-vars}}), the actor's process has several environment variables set to provide it with context: ||| |--- |--- | @@ -66,8 +66,8 @@ Aside from [custom environment variables](#source-env-vars), the actor's process |`APIFY_TIMEOUT_AT`|Date when the actor will time out.| |`APIFY_TOKEN`|The API token of the user who started the actor.| |`APIFY_USER_ID`|ID of the user who started the actor. Note that it might be different than the owner of the actor.| -|`APIFY_CONTAINER_PORT`|TCP port on which the actor can start a HTTP server to receive messages from the outside world. See [Container web server](#container-web-server) section for more details.| -|`APIFY_CONTAINER_URL`|A unique public URL under which the actor run web server is accessible from the outside world. See [Container web server](#container-web-server) section for more details.| +|`APIFY_CONTAINER_PORT`|TCP port on which the actor can start a HTTP server to receive messages from the outside world. See [Container web server]({{@link actor/run.md#container-web-server}}) section for more details.| +|`APIFY_CONTAINER_URL`|A unique public URL under which the actor run web server is accessible from the outside world. See [Container web server]({{@link actor/run.md#container-web-server}}) section for more details.| Dates are always in the UTC timezone and are represented in simplified extended ISO format ([ISO 8601 _open_in_new_](https://en.wikipedia.org/wiki/ISO_8601)), e.g. `2017-10-13T14:23:37.281Z` @@ -78,7 +78,7 @@ To access environment variables in Node.js, use the `process.env` object, for ex ### [](#resource-limits)Resource limits -Actors run inside a Docker container whose resources are limited. When invoking the actor, the caller has to specify the amount of memory allocated for the actor. Additionally, each user has a certain total limit of memory for running actors. The sum of memory allocated for all running actors and builds needs to fit into this limit, otherwise the user cannot start a new actor. For more details, see [Limits](#limits). +Actors run inside a Docker container whose resources are limited. When invoking the actor, the caller has to specify the amount of memory allocated for the actor. Additionally, each user has a certain total limit of memory for running actors. The sum of memory allocated for all running actors and builds needs to fit into this limit, otherwise the user cannot start a new actor. For more details, see [Limits]({{@link actor/limits.md}}). The share of CPU is computed automatically from the memory as follows: for each 4096 MB of memory, the actor gets 1 full CPU core. For other amounts of memory the number of CPU cores is computed fractionally. For example, an actor with 1024 MB of memory will have a 1/4 share of a CPU core. @@ -114,17 +114,17 @@ The whole process of resurrection looks as follows: * Updated duration will include the time when actor was not running. This does not affect compute units consumption. * Timeout will be counted from the point when this actor run was resurrected. -Resurrection can be peformed in Apify app using the **resurrect** button or via API using the [resurrect run](/docs/api/v2#/reference/actors/resurrect-run) API endpoint. +Resurrection can be peformed in Apify app using the **resurrect** button or via API using the [resurrect run](https://apify.com/docs/api/v2#/reference/actors/resurrect-run) API endpoint. ### [](#container-web-server)Container web server Each actor run is assigned a unique hard-to-guess URL (e.g. `http://kmdo7wpzlshygi.runs.apify.net`), which enables HTTP access to an optional web server running inside the actor run's Docker container. The URL is available in the following places: * In the web application, on the actor run details page as the **Container URL** field. -* In the API as the `containerUrl` property of the [Run object](./api/v2#/reference/actor/run-object/get-run). +* In the API as the `containerUrl` property of the [Run object](https://apify.com/api/v2#/reference/actor/run-object/get-run). * In the actor run's container as the `APIFY_CONTAINER_URL` environment variable. -The web server running inside the container must listen at the port defined by the `APIFY_CONTAINER_PORT` environment variable (typically 4321). If you want to use another port, simply define the `APIFY_CONTAINER_PORT` environment variable with the desired port number in your actor version configuration - see [Custom environment variable](#source-env-vars) for details. +The web server running inside the container must listen at the port defined by the `APIFY_CONTAINER_PORT` environment variable (typically 4321). If you want to use another port, simply define the `APIFY_CONTAINER_PORT` environment variable with the desired port number in your actor version configuration - see [Custom environment variable]({{@link actor/source_code.md#source-env-vars}}) for details. The following example demonstrates how to start a simple web server in your actor: @@ -147,4 +147,4 @@ The following example demonstrates how to start a simple web server in your acto ### [](#data-retention)Data retention -Actor run gets deleted along with its default storages (key-value store, dataset, request queue) after a data retention period which is based on [subscription plan](/pricing) of a user. +Actor run gets deleted along with its default storages (key-value store, dataset, request queue) after a data retention period which is based on [subscription plan](https://apify.com/pricing) of a user. diff --git a/docs/actor/source_code.md b/docs/actor/source_code.md index 8eeeef10f3..b167ac73ad 100644 --- a/docs/actor/source_code.md +++ b/docs/actor/source_code.md @@ -8,7 +8,7 @@ The **Source type** setting determines the location of the source code for the a ### [](#hosted-source-file)Single JavaScript file -The source code of the actor can be hosted directly on Apify. All the code needs to be in a single file and written in JavaScript / Node.js. The version of Node.js is determined by the **Base image** setting - see [Base images](#base-images) for the description of possible options. +The source code of the actor can be hosted directly on Apify. All the code needs to be in a single file and written in JavaScript / Node.js. The version of Node.js is determined by the **Base image** setting - see [Base images]({{@link actor/build.md#base-images}}) for the description of possible options. The hosted source is especially useful for simple actors. The source code can require arbitrary NPM packages. For example: @@ -23,17 +23,17 @@ Note that certain NPM packages need additional tools for their installation, suc ### [](#hosted-source-files)Multiple source files -If the source code of the actor requires the use of multiple files/directories, then it can be hosted on Apify platform with this option. This is particulary useful when you need to add [`INPUT_SCHEMA.json`](#input-schema) or `README.md` to your actor, or if you want to create your actor in other language then JavaScript. +If the source code of the actor requires the use of multiple files/directories, then it can be hosted on Apify platform with this option. This is particulary useful when you need to add [`INPUT_SCHEMA.json`]({{@link actor/source_code.md#input-schema}}) or `README.md` to your actor, or if you want to create your actor in other language then JavaScript. The only required file for multifile is `Dockerfile`, all other files depend on your `Dockerfile` settings. By default Apify's custom NodeJS `Dockerfile` is used, which requires `main.js` file containing your source code and `package.json` file, with package configuration for npm. Unlike the [Single Javascript file](#hosted-source-file) option, `package.json` is not automaticaly generated, and you need to configure it yourself. -See [Custom Dockerfile](#custom-dockerfile) and [Base Images](#base-images) for more information about creating your own Dockerfile and using Apify's prepared base images. +See [Custom Dockerfile]({{@link actor/source_code.md#custom-dockerfile}}) and [Base Images]({{@link actor/build.md#base-images}}) for more information about creating your own Dockerfile and using Apify's prepared base images. ### [](#source-git-repo)Git repository -If the source code of the actor is hosted externally in a Git repository, it can consist of multiple files and directories, use its own `Dockerfile` to control the build process (see [Custom Dockerfile](#custom-dockerfile) for details) and have a user description in store fetched from the `README.md` file. The location of the repository is specified by the **Git URL** setting, which can be an `https`, `git` or `ssh` URL. +If the source code of the actor is hosted externally in a Git repository, it can consist of multiple files and directories, use its own `Dockerfile` to control the build process (see [Custom Dockerfile]({{@link actor/source_code.md#custom-dockerfile}}) for details) and have a user description in store fetched from the `README.md` file. The location of the repository is specified by the **Git URL** setting, which can be an `https`, `git` or `ssh` URL. To help you get started quickly, you can use the [apify/quick-start](https://apify.com/apify/quick-start) actor which contains all the boilerplate necessary when creating a new actor hosted on Git. The source code is available on [GitHub](https://github.com/apifytech/actor-quick-start). @@ -41,7 +41,7 @@ To specify a Git branch or tag to check out, add a URL fragment to the URL. For Optionally, the second part of the fragment in the Git URL (separated by a colon) specifies the context directory for the Docker build. For example, `https://github.com/jancurn/act-analyse-pages.git#develop:some/dir` will check out the `develop` branch and set `some/dir` as a context directory for the Docker build. -Note that you can easily set up an integration where the actor is automatically rebuilt on every commit to the Git repository. For more details, see [GitHub integration](#github-integration). +Note that you can easily set up an integration where the actor is automatically rebuilt on every commit to the Git repository. For more details, see [GitHub integration]({{@link actor/source_code.md#github-integration}}). #### [](#source-git-repo-private)Private repositories @@ -51,7 +51,7 @@ To obtain the key click at the **deployment key** link under the **Git URL** tex ### [](#source-tarball)Zip file -The source code for the actor can also be located in a Zip archive hosted on an external URL. This option enables integration with arbitrary source code or continuous integration systems. Similarly as with the [Git repository](#source-git-repo), the source code can consist of multiple files and directories, can contain a custom `Dockerfile` and the actor description is taken from `README.md`. +The source code for the actor can also be located in a Zip archive hosted on an external URL. This option enables integration with arbitrary source code or continuous integration systems. Similarly as with the [Git repository]({{@link actor/source_code.md#source-git-repo}}), the source code can consist of multiple files and directories, can contain a custom `Dockerfile` and the actor description is taken from `README.md`. ### [](#source-github-gist)GitHub Gist @@ -65,11 +65,11 @@ Then set the **Source Type** to **GitHub Gist** and paste the Gist URL as follow Note that the example actor is available in the Apify Store as [apify/example-github-gist](https://apify.com/apify/example-github-gist). -Similarly as with the [Git repository](#source-git-repo), the source code can consist of multiple files and directories, it can contain a custom `Dockerfile` and the actor description is taken from `README.md`. +Similarly as with the [Git repository]({{@link actor/source_code.md#source-git-repo}}), the source code can consist of multiple files and directories, it can contain a custom `Dockerfile` and the actor description is taken from `README.md`. ### [](#custom-dockerfile)Custom Dockerfile -Internally, Apify uses Docker to build and run actors. To control the build of the actor, you can create a custom `Dockerfile` in the root of the Git repository or Zip directory. Note that this option is not available for the [Single JavaScript file](#hosted-source-file) option. If the `Dockerfile` is missing, the system uses the following default: +Internally, Apify uses Docker to build and run actors. To control the build of the actor, you can create a custom `Dockerfile` in the root of the Git repository or Zip directory. Note that this option is not available for the [Single JavaScript file]({{@link actor/source_code.md#hosted-source-file}}) option. If the `Dockerfile` is missing, the system uses the following default: FROM apify/actor-node-basic @@ -83,7 +83,7 @@ Internally, Apify uses Docker to build and run actors. To control the build of t For more information about Dockerfile syntax and commands, see the [Dockerfile reference _open_in_new_](https://docs.docker.com/engine/reference/builder/). -Note that `apify/actor-node-basic` is a base Docker image provided by Apify. There are other base images with other features available. However, you can use arbitrary Docker images as the base for your actors, although using the Apify images has some performance advantages. See [Base images](#base-images) for details. +Note that `apify/actor-node-basic` is a base Docker image provided by Apify. There are other base images with other features available. However, you can use arbitrary Docker images as the base for your actors, although using the Apify images has some performance advantages. See [Base images]({{@link actor/build.md#base-images}}) for details. By default, all Apify base Docker images start your Node.js application same way as `npm start` does, i.e. by running the command specified in the `package.json` file under the `scripts` - `start` key. The default `package.json` file looks similarly to this one: @@ -125,13 +125,13 @@ The actor owner can specify custom environment variables that are set to the act ![Custom environment variables](/img/docs/actor/source-env-vars.png) -Note that the custom environment variables are fixed during the build of the actor and cannot be changed later. See the [Build](#build) section for details. +Note that the custom environment variables are fixed during the build of the actor and cannot be changed later. See the [Build]({{@link actor/build.md#build}}) section for details. To access environment variables in Node.js, use the `process.env` object, for example: console.log(process.env.SMTP_HOST); -The actor runtime sets additional environment variables for the actor process during the run. See [Environment variables](#run-env-vars) for details. +The actor runtime sets additional environment variables for the actor process during the run. See [Environment variables]({{@link actor/run.md#run-env-vars}}) for details. ### [](#versioning)Versioning @@ -139,21 +139,21 @@ In order to enable active development, the actor can have multiple versions of t For example, the actor can have a production version `1.1`, a beta version `1.2` that contains new features but is still backwards compatible, and a development version `2.0` that contains breaking changes. -The versions of the actors are built and run separately. For details, see [Build](#build) and [Run](#run). +The versions of the actors are built and run separately. For details, see [Build]({{@link actor/build.md}}) and [Run]({{@link actor/run.md). ### [](#local-development)Local development -It is possible to develop actors locally on your computer and then only deploy them to the Apify cloud when they are ready. This is especially useful if you're using Git integration. See [Git repository](#source-git-repo) for more details. The boilerplate for creating an actor in a Git repository is available on [GitHub](https://github.com/apifytech/actor-quick-start). +It is possible to develop actors locally on your computer and then only deploy them to the Apify cloud when they are ready. This is especially useful if you're using Git integration. See [Git repository]({{@link actor/source_code.md#source-git-repo}}) for more details. The boilerplate for creating an actor in a Git repository is available on [GitHub](https://github.com/apifytech/actor-quick-start). In order to test the input and output of your actors on your local machine, you might define the `APIFY_DEV_KEY_VALUE_STORE_DIR` environment variable, which will cause the `apify` NPM package to emulate the key-value store locally using files in a directory. For more details, please see the [`apify` package documentation](https://sdk.apify.com/). -Unfortunately, not all features of the Apify platform can be emulated locally, therefore you might still need to let the [`apify`](/docs/api/apify-client-js/latest) NPM package use your API token in order to interact with the Apify platform. The simplest way to achieve that is by setting the `APIFY_TOKEN` environment variable on your local development machine. +Unfortunately, not all features of the Apify platform can be emulated locally, therefore you might still need to let the [`apify`](https://apify.com/docs/api/apify-client-js/latest) NPM package use your API token in order to interact with the Apify platform. The simplest way to achieve that is by setting the `APIFY_TOKEN` environment variable on your local development machine. ### [](#input-schema)Input schema Actor source files may contain an input schema defining the input that actor accepts and the UI components used for input at Apify platform. Using input schema you can provide UI to actor users that is easy to use and also ensure that input of your actor is valid. -For more information on this topic see [input schema documentation](/docs/actor/input-schema) on a separate page. +For more information on this topic see [input schema documentation](https://apify.com/docs/actor/input-schema) on a separate page. ### [](#metamorph)Metamorph diff --git a/docs/api/index.md b/docs/api/index.md index 98ec2f9907..c1823dafbf 100644 --- a/docs/api/index.md +++ b/docs/api/index.md @@ -10,19 +10,19 @@ Note that you can find API endpoints related to specific sections of the app in **IMPORTANT: All requests with JSON payloads need to specify the `Content-Type: application/json` HTTP header or they will not work!** -For JavaScript / Node.js applications, the easiest way to access the Apify API is using the [`apify-client`](/docs/api/apify-client-js/latest) NPM package. +For JavaScript / Node.js applications, the easiest way to access the Apify API is using the [`apify-client`](https://apify.com/docs/api/apify-client-js/latest) NPM package. ### [](#js-client)JavaScript API client The `apify-client` NPM package provides a JavaScript client for Apify API. This package greatly simplifies the development of apps that depend on the Apify platform. -* [Documentation for latest version](/docs/api/apify-client-js/latest) +* [Documentation for latest version](https://apify.com/docs/api/apify-client-js/latest) * [Package page on NPM _open_in_new_](https://www.npmjs.com/package/apify-client) * [Source code on GitHub _open_in_new_](https://github.com/apifytech/apify-client-js) ### [](#reference)API reference -The latest version of the API used to manage [Apify actors](./actor) and associated resources. The API enables you to manage: +The latest version of the API used to manage [Apify actors]({{@link actor/index.md}}) and associated resources. The API enables you to manage: * List actors and actor tasks, view and update their settings * Run and stop actors and actor tasks @@ -31,4 +31,4 @@ The latest version of the API used to manage [Apify actors](./actor) and associa * View user information * Download logs -[View API reference](/docs/api/v2) +[View API reference](https://apify.com/docs/api/v2) diff --git a/docs/proxy/connection_settings.md b/docs/proxy/connection_settings.md index 29898576a6..c9029b99c8 100644 --- a/docs/proxy/connection_settings.md +++ b/docs/proxy/connection_settings.md @@ -11,8 +11,8 @@ The following table shows HTTP proxy connection settings for the Apify Proxy. |`HTTP`| |`proxy.apify.com`| |`8000`| -|Specifies proxy settings. See [username parameters](#datacenter-proxy--username-params) below for details. **Beware that this is not your Apify username!**| -|Proxy password. Your password is displayed on the [Proxy page](https://my.apify.com/proxy) in the app. Also, in Apify actors, it is passed as the `APIFY_PROXY_PASSWORD` environment variable. See [actor documentation](./actor#run-env-vars) for more details.| +|Specifies proxy settings. See [username parameters]({{@link proxy/datacenter_proxy.md#datacenter-proxy--username-params}}) below for details. **Beware that this is not your Apify username!**| +|Proxy password. Your password is displayed on the [Proxy page](https://my.apify.com/proxy) in the app. Also, in Apify actors, it is passed as the `APIFY_PROXY_PASSWORD` environment variable. See [actor documentation]({{@link actor/run.md#run-env-vars}}) for more details.| |`http://:@proxy.apify.com:8000`| diff --git a/docs/proxy/datacenter_proxy.md b/docs/proxy/datacenter_proxy.md index cc9bacbb9a..123c346a6c 100644 --- a/docs/proxy/datacenter_proxy.md +++ b/docs/proxy/datacenter_proxy.md @@ -70,7 +70,7 @@ If you do not want to specify both `groups` and `session` parameters and therefo ### [](#datacenter-proxy--session-persistence)Session persistence -When using Apify Proxy with `session` parameter set in the username (see [Username parameters](#datacenter-proxy--username-params)) a single IP is assigned to the session ID provided after the first request is made. This IP/session_id combination is persited, and its expiration is set to 24 hours later. Each additional request extends the expiration back to 24 hours, so if you use the session at least once a day it will never expire, with two possible exceptions: +When using Apify Proxy with `session` parameter set in the username (see [Username parameters]({{@link proxy/datacenter_proxy.md#datacenter-proxy--username-params}})) a single IP is assigned to the session ID provided after the first request is made. This IP/session_id combination is persited, and its expiration is set to 24 hours later. Each additional request extends the expiration back to 24 hours, so if you use the session at least once a day it will never expire, with two possible exceptions: * Proxy server stops responding and is marked as dead during a health check * If the Proxy Server is part of a Proxy Group that is refreshed monthly and is rotated out. @@ -79,7 +79,7 @@ If the session is discarded due to the reasons above, then a new IP is assigned ### [](#datacenter-proxy--nodejs-examples)NodeJS Examples -The following sections contain several examples of how to use Apify Proxy in NodeJS (used as the default language in [actors](./actor)). +The following sections contain several examples of how to use Apify Proxy in NodeJS (used as the default language in [actors]({{@link actor/index.md}})). #### Usage in [PuppeteerCrawler](/docs/sdk/apify-runtime-js/latest#PuppeteerCrawler) diff --git a/docs/proxy/google_serp_proxy.md b/docs/proxy/google_serp_proxy.md index d0b134c264..dc01d88b3f 100644 --- a/docs/proxy/google_serp_proxy.md +++ b/docs/proxy/google_serp_proxy.md @@ -15,7 +15,7 @@ Currently supported google search services are: **Important:** Only HTTP requests are allowed, and the Google hostname needs to start with `www.` subdomain. -**Pricing is based on the number of requests made**. Please [contact us](/contact) if you want to use Google SERP Proxy or if you need more information. +**Pricing is based on the number of requests made**. Please [contact us](https://apify.com/contact) if you want to use Google SERP Proxy or if you need more information. ### [](#google-serp--username-params)Username parameters @@ -42,9 +42,9 @@ See [full list of available domain names](https://ipfs.io/ipfs/QmXoypizjW3WknFiJ ### [](#google-serp--nodejs-examples)NodeJS Examples -The following sections contain several examples of how to use Google SERP proxy in [actors](./actor). +The following sections contain several examples of how to use Google SERP proxy in [actors]({{@link actor/index.md}}). -#### Usage in [PuppeteerCrawler](/docs/sdk/apify-runtime-js/latest#PuppeteerCrawler) +#### Usage in [PuppeteerCrawler](https://apify.com/docs/sdk/apify-runtime-js/latest#PuppeteerCrawler) Get a list of results from the US for keyword `wikipedia` and parse them through cheerio @@ -145,7 +145,7 @@ Get a list of shopping results from the Czech Republic for keyword `Apple iPhone await crawler.run(); }); -#### Usage in [Apify.launchPuppeteer()](/docs/sdk/apify-runtime-js/latest#module-Apify-launchPuppeteer) +#### Usage in [Apify.launchPuppeteer()](https://apify.com/docs/sdk/apify-runtime-js/latest#module-Apify-launchPuppeteer) Get a list of results from the US for keyword `wikipedia` and parse them through cheerio diff --git a/docs/proxy/index.md b/docs/proxy/index.md index ebfcf622b5..3811131374 100644 --- a/docs/proxy/index.md +++ b/docs/proxy/index.md @@ -4,7 +4,7 @@ title: Proxy # [](./proxy)Proxy -Apify Proxy provides access to Apify's proxy services that can be used in [actors](./actor) or any other application that support HTTP proxies. Currently, Apify Proxy provides access to [datacenter proxy servers](#datacenter-proxy), [residential proxy](#residential-proxy), and [Google SERP proxy](#google-serp). It supports HTTP as well as other protocols like HTTPS and FTP. +Apify Proxy provides access to Apify's proxy services that can be used in [actors]({{@link actor/index.md}}) or any other application that support HTTP proxies. Currently, Apify Proxy provides access to [datacenter proxy servers]({{@link proxy/datacenter_proxy.md}}), [residential proxy]({{@link proxy/residential_proxy.md}}), and [Google SERP proxy]({{@link proxy/google_serp_proxy.md}}). It supports HTTP as well as other protocols like HTTPS and FTP. You can view your Apify Proxy settings on the [Proxy page](https://my.apify.com/proxy) in the app. diff --git a/docs/proxy/residential_proxy.md b/docs/proxy/residential_proxy.md index 54abe23fc7..389989a13f 100644 --- a/docs/proxy/residential_proxy.md +++ b/docs/proxy/residential_proxy.md @@ -6,9 +6,9 @@ title: Residential proxy Sometimes datacenter proxy servers are not a viable option for certain solutions and for these cases Apify Proxy includes an option to use Residential Proxy. This proxy solution allows the user access to a much larger pool of proxy servers than with datacenter proxy servers and therefore it is usually a better option for situations where a large number of proxy servers is required. -On the Apify platform, users can use Residential Proxy after they are given access to it by the [](/contact)Apify Support Team. **Pricing is based on data traffic**, which is measured for each connection made through the proxy and displayed on the platform's dashboard. +On the Apify platform, users can use Residential Proxy after they are given access to it by the Apify Support Team. **Pricing is based on data traffic**, which is measured for each connection made through the proxy and displayed on the platform's dashboard. -Please [contact us](/contact) if you want to use Apify Residential Proxy or if you need more information. +Please [contact us](https://apify.com/contact) if you want to use Apify Residential Proxy or if you need more information. ### [](#residential-proxy--username-params)Username parameters @@ -39,14 +39,14 @@ And here is how it would look if you need a random proxy from the US ### [](#residential-proxy--session-persistence)Session persistence -When using Apify Proxy with `session` parameter set in the username (see [Username parameters](#residential-proxy--username-params)) a single IP is assigned to the session ID provided after the first request is made. This IP is persisted for one minute and its expiration is refreshed with each request. If the proxy server becomes unresponsive or the session expires a new IP is selected for the next request. +When using Apify Proxy with `session` parameter set in the username (see [Username parameters]({{@link proxy/residential_proxy.md##residential-proxy--username-params}})) a single IP is assigned to the session ID provided after the first request is made. This IP is persisted for one minute and its expiration is refreshed with each request. If the proxy server becomes unresponsive or the session expires a new IP is selected for the next request. ### [](#residential-proxy--nodejs-examples)NodeJS Examples -The following sections contain several examples of how to use Apify Proxy in NodeJS (used as the default language in [actors](./actor)). +The following sections contain several examples of how to use Apify Proxy in NodeJS (used as the default language in [actors]({{@link actor/index.md}})). -#### Usage in [PuppeteerCrawler](/docs/sdk/apify-runtime-js/latest#PuppeteerCrawler) +#### Usage in [PuppeteerCrawler](https://apify.com/docs/sdk/apify-runtime-js/latest#PuppeteerCrawler) Use a single session with IP from the US for the whole PuppeteerCrawler run (for as long as the session lasts) @@ -106,7 +106,7 @@ Create a new session with IP from GB for each browser launched during the Crawle await crawler.run(); }); -#### Usage in [Apify.launchPuppeteer()](/docs/sdk/apify-runtime-js/latest#module-Apify-launchPuppeteer) +#### Usage in [Apify.launchPuppeteer()](https://apify.com/docs/sdk/apify-runtime-js/latest#module-Apify-launchPuppeteer) Use a single IP from Germany for all requests done in the launched browser diff --git a/docs/storage/dataset.md b/docs/storage/dataset.md index ab090eeb5a..4c93e27a08 100644 --- a/docs/storage/dataset.md +++ b/docs/storage/dataset.md @@ -51,4 +51,4 @@ Data without hidden fields are called "clean" and can be downloaded at Apify app #### [](#dataset-api-client)API and JavaScript client -The dataset provides a [HTTP API](/docs/api/v2#/reference/datasets) to manage datasets and to add/retrieve their data. If you are developing a Node.js application, then you can also use the [Apify JavaScript client](/docs/api/apify-client-js/latest#ApifyClient-datasets). +The dataset provides a [HTTP API](https://apify.com/docs/api/v2#/reference/datasets) to manage datasets and to add/retrieve their data. If you are developing a Node.js application, then you can also use the [Apify JavaScript client](https://apify.com/docs/api/apify-client-js/latest#ApifyClient-datasets). diff --git a/docs/storage/index.md b/docs/storage/index.md index f6e80b11bc..3a3299430e 100644 --- a/docs/storage/index.md +++ b/docs/storage/index.md @@ -4,8 +4,8 @@ title: Storage # [](./storage)Storage -The Apify platform contains three storage types that you can use in your [actors](/docs/actor) and also outside the Apify platform using [HTTP API](/docs/api/v2) and [JavaScript client](/docs/api/apify-client-js/latest). +The Apify platform contains three storage types that you can use in your [actors]({{@link actor/index.md}}) and also outside the Apify platform using [HTTP API](https://apify.com/docs/api/v2) and [JavaScript client](https://apify.com/docs/api/apify-client-js/latest). -* [Key-value store](#key-value-store) - simple storage for strings and files -* [Dataset](#dataset) - storage for sequential data objects (results) -* [Request queue](#queue) - dynamic queue of URLs to be processed +* [Key-value store]({{@link storage/key_value_store.md}}) - simple storage for strings and files +* [Dataset]({{@link storage/dataset.md}}) - storage for sequential data objects (results) +* [Request queue]({{@link storage/request_queue.md}}) - dynamic queue of URLs to be processed diff --git a/docs/storage/key_value_store.md b/docs/storage/key_value_store.md index b1152676e8..7dc8960d46 100644 --- a/docs/storage/key_value_store.md +++ b/docs/storage/key_value_store.md @@ -34,4 +34,4 @@ If you want to use something other than the default key-value store, e.g. some s #### [](#key-value-store-api-client)API and JavaScript client -The key-value store also provides a [HTTP API](/docs/api/v2#/reference/key-value-stores) to manage key-value stores and their records. If you are developing a Node.js application then you can also use the [Apify JavaScript client](/docs/api/apify-client-js/latest#ApifyClient-keyValueStores). +The key-value store also provides a [HTTP API](https://apify.com/docs/api/v2#/reference/key-value-stores) to manage key-value stores and their records. If you are developing a Node.js application then you can also use the [Apify JavaScript client](https://apify.com/docs/api/apify-client-js/latest#ApifyClient-keyValueStores). diff --git a/docs/storage/request_queue.md b/docs/storage/request_queue.md index ac90b434af..eb47c43f14 100644 --- a/docs/storage/request_queue.md +++ b/docs/storage/request_queue.md @@ -42,4 +42,4 @@ If queue is opened then you can use it: #### [](#queue-api-client)API and Javascript client -The request queue provides a [HTTP API](/docs/api/v2#/reference/request-queues) to manage queues and to add/retrieve requests. If you are developing a Node.js application, then you can also use the [Apify JavaScript client](/docs/api/apify-client-js/latest#ApifyClient-requestQueues). +The request queue provides a [HTTP API](https://apify.com/docs/api/v2#/reference/request-queues) to manage queues and to add/retrieve requests. If you are developing a Node.js application, then you can also use the [Apify JavaScript client](https://apify.com/docs/api/apify-client-js/latest#ApifyClient-requestQueues). diff --git a/docs/tasks/create.md b/docs/tasks/create.md index a6e8ce7e15..e11a69b1ae 100644 --- a/docs/tasks/create.md +++ b/docs/tasks/create.md @@ -12,4 +12,4 @@ When you click the "Create Task" button, a new task will be created with the inp ### Create a Task for a public actor -Tasks can also be set up for actors made by someone else. Just [search the store](/store) to find an actor you want to use, try it out, and if you like it, you can create a task from it with a single click of a button - the same as for your actor. +Tasks can also be set up for actors made by someone else. Just [search the store](https://apify.com/store) to find an actor you want to use, try it out, and if you like it, you can create a task from it with a single click of a button - the same as for your actor. diff --git a/docs/tasks/index.md b/docs/tasks/index.md index 7cc37703f6..73a2e45b7c 100644 --- a/docs/tasks/index.md +++ b/docs/tasks/index.md @@ -6,4 +6,4 @@ title: Tasks Actor Tasks help you prepare the configuration of an actor to perform a specific job (think of it as giving your actor a screenplay). Tasks enable you to create multiple configurations for a single Actor and then run the selected configuration directly from the Apify platform, scheduler or API. -Tasks can also be set up for actors made by someone else. Just [search the store](/store) to find an actor you want to use, try it out, and if you like it, you can create a task from it with a single click of a button. +Tasks can also be set up for actors made by someone else. Just [search the store](https://apify.com/store) to find an actor you want to use, try it out, and if you like it, you can create a task from it with a single click of a button. diff --git a/docs/tasks/run.md b/docs/tasks/run.md index f588478e3d..7557789c6a 100644 --- a/docs/tasks/run.md +++ b/docs/tasks/run.md @@ -4,4 +4,4 @@ title: Run ## [](#run-task)Running the task -When you are happy with the configuration of your Task. You can run it directly from the UI with a click of a button. After you start it, you will be directed to the run page. Tasks can also be started using the [API](./API) or [scheduler](./scheduler). +When you are happy with the configuration of your Task. You can run it directly from the UI with a click of a button. After you start it, you will be directed to the run page. Tasks can also be started using the [API]({{@link api/index.md}}) or [scheduler]({{@link scheduler/index.md}}). diff --git a/docs/webhooks/actions.md b/docs/webhooks/actions.md index 94af9f4513..f1515c30ca 100644 --- a/docs/webhooks/actions.md +++ b/docs/webhooks/actions.md @@ -18,7 +18,7 @@ For safety reasons, the webhook URL should contain a secret token to ensure only The payload template is a JSON-like string, whose syntax is extended with the use of variables. This is useful when a custom payload structure is needed, but at the same time dynamic data, that is only known at the time of the webhook's invocation, need to be injected into the payload. Aside from the variables, the string must be a valid JSON. -The variables need to be enclosed in double curly braces and cannot be chosen arbitrarily. A pre-defined list, [that can be found below](./webhooks#actions-http-request-available-variables), shows all the currently available variables. Using any other variable than one of the pre-defined will result in a validation error. +The variables need to be enclosed in double curly braces and cannot be chosen arbitrarily. A pre-defined list, [that can be found below]({{@link webhooks/actions.md#actions-http-request-available-variables}}), shows all the currently available variables. Using any other variable than one of the pre-defined will result in a validation error. The syntax of a variable therefore is: `{{oneOfAvailableVariables}}`. @@ -55,17 +55,17 @@ The syntax of a variable therefore is: `{{oneOfAvailableVariables}}`. You may have noticed that the `eventData` and `resource` properties contain redundant data. This is for backwards compatibility. Feel free to only use `eventData` or `resource` in your templates, depending on your use case. -#### [](./webhooks#actions-http-request-available-variables)Available variables +#### []({{@link webhooks/actions.md#actions-http-request-available-variables}})Available variables |Variable|Type|Description| |--- |--- |--- | |`userId`|string|ID of the user who owns the webhook.| |`createdAt`|string|ISO string date of the webhook's trigger event.| -|`eventType`|string|Type of the trigger event, [see Events](./webhooks#events).| -|`eventData`|Object|Data associated with the trigger event, [see Events](./webhooks#events).| -|`resource`|Object|The resource that caused the trigger event, [see below](./webhooks#actions-http-request-available-variables-resource).| +|`eventType`|string|Type of the trigger event, [see Events]({{@link webhooks/events.md}}).| +|`eventData`|Object|Data associated with the trigger event, [see Events]({{@link webhooks/events.md}}).| +|`resource`|Object|The resource that caused the trigger event, [see below]({{@link webhooks/actions.md#actions-http-request-available-variables-resource}}).| ##### Resource -The `resource`variable represents the triggering system resource. For example when using the `ACTOR.RUN.SUCCEEDED` event, the resource is the actor run. The variable will be replaced by an `Object` that one would receive as response from the relevant API at the moment when the webhook is triggered. So for the actor run resource, it would be the response of the [`GET` actor run](/docs/api/v2#/reference/actors/run-object/get-run) API endpoint. +The `resource`variable represents the triggering system resource. For example when using the `ACTOR.RUN.SUCCEEDED` event, the resource is the actor run. The variable will be replaced by an `Object` that one would receive as response from the relevant API at the moment when the webhook is triggered. So for the actor run resource, it would be the response of the [`GET` actor run](https://apify.com/docs/api/v2#/reference/actors/run-object/get-run) API endpoint. diff --git a/docs/webhooks/ad_hoc_webhooks.md b/docs/webhooks/ad_hoc_webhooks.md index 6b553f68a2..e241c5ac30 100644 --- a/docs/webhooks/ad_hoc_webhooks.md +++ b/docs/webhooks/ad_hoc_webhooks.md @@ -4,7 +4,7 @@ title: Ad hoc webhooks3 ## [](./webhooks#adhoc)Ad hoc webhooks -An ad hoc webhook is a one-time webhook created for a certain actor run when starting the run using Apify [API](/docs/api/v2). It's triggered at most once when the given run transitions into the selected state. Ad hoc webhooks can be defined using a URL parameter `webhooks` added to the API endpoint that starts an actor or actor task: +An ad hoc webhook is a one-time webhook created for a certain actor run when starting the run using Apify [API](https://apify.com/docs/api/v2). It's triggered at most once when the given run transitions into the selected state. Ad hoc webhooks can be defined using a URL parameter `webhooks` added to the API endpoint that starts an actor or actor task: https://api.apify.com/v2/acts/[ACTOR_ID]/runs?token=[YOUR_API_TOKEN]&webhooks=[AD_HOC_WEBHOOKS] diff --git a/docs/webhooks/events.md b/docs/webhooks/events.md index 15b38a4b29..02ca7ef326 100644 --- a/docs/webhooks/events.md +++ b/docs/webhooks/events.md @@ -10,7 +10,7 @@ An event is represented by a **type** and related **data**. The type is a `strin ### [](./webhooks#events-actor-run)Actor run -Actor run events are triggered when an actor run gets created or transitions into a new state. Webhook can be defined for all runs of an [actor](/docs/actor) at its detail page or for a specific [actor task](/docs/tasks) at its detail page. In that case, the webhook is invoked only for runs started for that task. +Actor run events are triggered when an actor run gets created or transitions into a new state. Webhook can be defined for all runs of an [actor]({{@link actor/index.md}}) at its detail page or for a specific [actor task]({{@link tasks/index.md}}) at its detail page. In that case, the webhook is invoked only for runs started for that task. #### Event types: