Apify command-line interface (Apify CLI) helps you create, develop, build and run Apify actors, and manage the Apify cloud platform from any computer.
Apify actors are cloud programs that can perform arbitrary web scraping, automation or data processing job. They accept input, perform their job and generate output. While you can develop actors in an online IDE directly in the Apify web application, for complex projects it is more convenient to develop actors locally on your computer using Apify SDK and only push the actors to the Apify cloud during deployment. This is where the Apify CLI comes in.
Note that actors running on the Apify platform are executed in Docker containers, so with an appropriate Dockerfile
you can build your actors in any programming language.
However, we recommend using JavaScript / Node.js, for which we provide most libraries and support.
First, make sure you have Node.js version 10 or higher with NPM installed on your computer:
node --version
npm --versionInstall or upgrade Apify CLI by running:
npm -g install apify-cliIf you receive an EACCES error, you might need to run the commnad as root:
sudo npm -g install apify-cliAlternativaly, you can use Node Version Manager (nvm) and install Apify CLI only into a selected user-level Node version without requiring root privileges:
nvm install 12
nvm use 12
npm -g install apify-cli
Finally, verify that Apify CLI was installed correctly by running:
apify --versionwhich should print something like:
apify-cli/0.5.3 darwin-x64 node-v12.16.1
The following examples demonstrate the basic usage of Apify CLI.
apify create my-hello-worldFirst, you will be prompted to select a template with the boilerplate for the actor, to help you get started quickly.
The command will create a directory called my-hello-world that contains a Node.js project
for the actor and a few configuration files.
cd ./my/awesome/project
apify initThis command will only set up local actor development environment in an existing directory,
i.e. it will create the apify.json file and apify_storage directory.
Before you can run your project using apify run, you have to set up the right start command in package.json under scripts.start. For example:
{
...
"scripts": {
"start": "node your_main_file.js",
},
...
}
You can find more information about by running apify help run.
cd my-hello-world
apify runThis command runs the actor on your local machine. Now's your chance to develop the logic - or magic 😏
apify loginBefore you can interact with the Apify cloud, you need to create an Apify account
and log in to it using the above command. You will be prompted for
your Apify API token.
Note that the command will store the API token and other sensitive information to ~/.apify.
apify pushThis command uploads your project to the Apify cloud and builds an actor from it.
apify callRuns the actor corresponding to the current directory on the Apify platform.
This command can also be used to run other actors, for example:
apify call apify/hello-worldThis file associates your local development project with an actor on the Apify platform. It contains information such as actor name, version, build tag and environment variables. Make sure you commit this file to the Git repository.
For example, apify.json file can look as follows:
{
"name": "dataset-to-mysql",
"version": "0.1",
"buildTag": "latest",
"env": {
"MYSQL_USER": "my_username",
"MYSQL_PASSWORD": "@mySecretPassword"
},
"template": "basic"
}There are two options how you can set up environment variables for actors.
All keys from env will be set as environment variables into Apify platform after you push actor to Apify. Current values on Apify will be overridden.
{
"name": "dataset-to-mysql",
"version": "0.1",
"buildTag": "latest",
"env": {
"MYSQL_USER": "my_username",
"MYSQL_PASSWORD": "@mySecretPassword"
},
"template": "basic"
}In Apify app select your actor, you can set up variables into Source tab.
After setting up variables in the app, set up env to null apify.json. Otherwise, variables from apify.json will override variables in the app.
{
"name": "dataset-to-mysql",
"version": "0.1",
"buildTag": "latest",
"env": null,
"template": "basic"
}CLI provides commands to manage secrets environment variables. Secrets are stored to the ~/.apify directory. Adds a new secret using command:
apify secrets:add mySecretPassword pwd1234After adding a new secret you can use the secret in apify.json
{
"name": "dataset-to-mysql",
...
"env": {
"MYSQL_PASSWORD": "@mySecretPassword"
},
...
}
To see all CLI commands simply run:
apify helpTo get information about a specific command run:
apify help COMMANDStill haven't found what you were looking for? Please go to Apify Help center or contact us.
This section contains printouts of apify help for all commands.
apify call [ACTID]apify create [ACTORNAME]apify infoapify init [ACTORNAME]apify loginapify logoutapify push [ACTORID]apify runapify secretsapify secrets:add NAME VALUEapify secrets:rm NAMEapify vis [PATH]
Runs a specific actor remotely on the Apify cloud platform.
USAGE
$ apify call [ACTID]
ARGUMENTS
ACTID Name or ID of the actor to run (e.g. "apify/hello-world" or "E2jjCZBezvAZnX8Rb"). If not provided, the command
runs the remote actor specified in the "apify.json" file.
OPTIONS
-b, --build=build Tag or number of the build to run (e.g. "latest" or "1.2.34").
-m, --memory=memory Amount of memory allocated for the actor run, in megabytes.
-t, --timeout=timeout Timeout for the actor run in seconds. Zero value means there is no timeout.
-w, --wait-for-finish=wait-for-finish Seconds for waiting to run to finish, if no value passed, it waits forever.
DESCRIPTION
The actor is run under your current Apify account, therefore you need to be logged in by calling "apify login". It
takes input for the actor from the default local key-value store by default.See code: src/commands/call.js
Creates a new actor project directory from a selected boilerplate template.
USAGE
$ apify create [ACTORNAME]
ARGUMENTS
ACTORNAME Name of the actor and its directory
OPTIONS
-t, --template=hello_world|puppeteer_crawler|puppeteer_single_page|cheerio_crawler|basic_crawler|apify_project
Boilerplate template for the actor. If not provided, the command will prompt for it.
See code: src/commands/create.js
Displays information about the currently active Apify account.
USAGE
$ apify info
DESCRIPTION
The information is printed to the console.See code: src/commands/info.js
Initializes a new actor project in an existing directory.
USAGE
$ apify init [ACTORNAME]
ARGUMENTS
ACTORNAME Name of the actor. If not provided, you will be prompted for it.
DESCRIPTION
The command only creates the "apify.json" file and the "apify_storage" directory in the current directory, but will
not touch anything else.
WARNING: The directory at "apify_storage" will be overwritten if it already exists.See code: src/commands/init.js
Logs in to your Apify account using a provided API token.
USAGE
$ apify login
OPTIONS
-t, --token=token [Optional] Apify API token
DESCRIPTION
The API token and other account information is stored in the ~/.apify directory, from where it is read by all other
"apify" commands. To log out, call "apify logout".See code: src/commands/login.js
Logs out of your Apify account.
USAGE
$ apify logout
DESCRIPTION
The command deletes the API token and all other account information stored in the ~/.apify directory. To log in again,
call "apify login".See code: src/commands/logout.js
Uploads the actor to the Apify platform and builds it there.
USAGE
$ apify push [ACTORID]
ARGUMENTS
ACTORID ID of an existing actor on the Apify platform where the files will be pushed. If not provided, the command
will create or modify the actor with the name specified in "apify.json" file.
OPTIONS
-b, --build-tag=build-tag Build tag to be applied to the successful actor build. By default, it is taken
from the "apify.json" file
-v, --version=version Actor version number to which the files should be pushed. By default, it is
taken from the "apify.json" file.
-w, --wait-for-finish=wait-for-finish Seconds for waiting to build to finish, if no value passed, it waits forever.
--version-number=version-number DEPRECATED: Use flag version instead. Actor version number to which the files
should be pushed. By default, it is taken from the "apify.json" file.
DESCRIPTION
The actor settings are read from the "apify.json" file in the current directory, but they can be overridden using
command-line options.
NOTE: If the source files are smaller than 3 MB then they are uploaded as
"Multiple source files", otherwise they are uploaded as "Zip file".
WARNING: If the target actor already exists in your Apify account, it will be overwritten!See code: src/commands/push.js
Runs the actor locally in the current directory by executing "npm start".
USAGE
$ apify run
OPTIONS
-p, --purge Shortcut that combines the --purge-queue, --purge-dataset and --purge-key-value-store
options.
--purge-dataset Deletes the local directory containing the default dataset before the run starts.
--purge-key-value-store Deletes all records from the default key-value store in the local directory before the run
starts, except for the "INPUT" key.
--purge-queue Deletes the local directory containing the default request queue before the run starts.
DESCRIPTION
It sets various APIFY_XYZ environment variables in order to provide a working execution environment for the actor. For
example, this causes the actor input, as well as all other data in key-value stores, datasets or request queues to be
stored in the "apify_storage" directory, rather than on the Apify platform.
NOTE: You can override the default behaviour of command overriding npm start script value in a package.json file. You
can set up your own main file or environment variables by changing it.See code: src/commands/run.js
Manages secret values for actor environment variables.
USAGE
$ apify secrets
DESCRIPTION
Example:
$ apify secrets:add mySecret TopSecretValue123
Now the "mySecret" value can be used in an environment variable defined in "apify.json" file by adding the "@" prefix:
{
"name": "my_actor",
"env": { "SECRET_ENV_VAR": "@mySecret" },
"version": "0.1
}
When the actor is pushed to Apify cloud, the "SECRET_ENV_VAR" and its value is stored as a secret environment variable
of the actor.See code: src/commands/secrets/index.js
Adds a new secret value.
USAGE
$ apify secrets:add NAME VALUE
ARGUMENTS
NAME Name of the secret
VALUE Value of the secret
DESCRIPTION
The secrets are stored to a file at ~/.apifySee code: src/commands/secrets/add.js
Removes the secret.
USAGE
$ apify secrets:rm NAME
ARGUMENTS
NAME Name of the secretSee code: src/commands/secrets/rm.js
Validates INPUT_SCHEMA.json file and prints errors found.
USAGE
$ apify vis [PATH]
ARGUMENTS
PATH Optional path to your INPUT_SCHEMA.json file. If not provided ./INPUT_SCHEMA.json is used.See code: src/commands/vis.js