Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@ Actors have the following elements:

The documentation and the input/output schemas make it possible for people to easily understand what the Actor does, enter the required inputs both in the user interface or API, and integrate the Actor's results with their other workflows. Actors can easily call and interact with each other, enabling building more complex systems on top of simple ones.

The Apify platform provides an open [API](/api/v2), cron-style [scheduler](../../schedules), [webhooks](../../../integrations/webhooks/index.md), and [integrations](../../integrations) to services such as Zapier or Make, which make it easy for users to integrate Actors with their existing workflows. Anyone is welcome to [publish Actors](/platform/actors/publishing) in [Apify Store](https://apify.com/store), and you can even [monetize your Actors](/platform/actors/publishing/monetize).
The Apify platform provides an open [API](/api/v2), cron-style [scheduler](../../schedules), [webhooks](../../../integrations/programming/webhooks/index.md), and [integrations](../../integrations) to services such as Zapier or Make, which make it easy for users to integrate Actors with their existing workflows. Anyone is welcome to [publish Actors](/platform/actors/publishing) in [Apify Store](https://apify.com/store), and you can even [monetize your Actors](/platform/actors/publishing/monetize).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The [webhooks](../../../integrations/programming/webhooks/index.md) link works? wow, that's cool


Actors can be developed and run locally and then easily deployed to the Apify platform using the [Apify CLI](/cli) or a [GitHub integration](../../../integrations/github.md). For more details, see the [Deployment](../deployment/index.md) section.
Actors can be developed and run locally and then easily deployed to the Apify platform using the [Apify CLI](/cli) or a [GitHub integration](../../../integrations/programming/github.md). For more details, see the [Deployment](../deployment/index.md) section.

> **To get a better idea of what Apify Actors are, visit [Apify Store](https://apify.com/store), and try out some of them!**
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ To specify a Git branch or tag to check out, add a URL fragment to the URL. For

Optionally, the second part of the fragment in the Git URL (separated by a colon) specifies the directory from which the Actor will be built (and where the `.actor`) folder is located. For example, `https://github.com/jancurn/some-actor.git#develop:some/dir` will check out the **develop** branch and set **some/dir** as the root directory of the Actor.

Note that you can easily set up an integration where the Actor is automatically rebuilt on every commit to the Git repository. For more details, see [GitHub integration](../../../integrations/github.md).
Note that you can easily set up an integration where the Actor is automatically rebuilt on every commit to the Git repository. For more details, see [GitHub integration](../../../integrations/programming/github.md).

### [](#private-repositories)Private repositories

Expand Down
1 change: 1 addition & 0 deletions sources/platform/integrations/actors/_category_.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
position: 3
4 changes: 2 additions & 2 deletions sources/platform/integrations/actors/index.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Actors
title: Actor-to-Actor
description: Learn how to integrate with other Actors and tasks.
sidebar_position: 11.10
sidebar_label: Actor-to-Actor
slug: /integrations/actors
---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Integration-ready Actors
title: Creating integration Actors
description: Learn how to create Actors that are ready to be integrated with other Actors and tasks.
sidebar_position: 1
slug: /integrations/actors/integration-ready-actors
Expand Down
2 changes: 2 additions & 0 deletions sources/platform/integrations/ai/_category_.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
label: 'AI'
position: 6
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
---
title: Flowise
title: Flowise integration
description: Learn how to integrate Apify with Flowise.
sidebar_position: 11.20
sidebar_label: Flowise
sidebar_position: 3
slug: /integrations/flowise
---

Expand Down Expand Up @@ -43,23 +44,23 @@ The first step is to create a new flow in the web UI.

In the left menu, you need to find Apify Website Content Crawler under Document Loaders.

![Flowise add Apify Crawler](./images/flowise-apify.png)
![Flowise add Apify Crawler](../images/flowise-apify.png)

Now you need to configure the crawler. You can find more information about the crawler [here](https://apify.com/apify/website-content-crawler).

![Flowise and Apify](images/flowise.png)
![Flowise and Apify](../images/flowise.png)

In the configuration, provide your Apify API token, which you can find in your [Apify account](https://my.apify.com/account#/integrations).

![Apify API token screen](./images/flowise-apify-api.png)
![Apify API token screen].(./images/flowise-apify-api.png)

You can add more loaders, or you can add some processors to process the data.
In our case, we create the flow that loads data from the Apify docs using Website Content Crawler and save them into the in-memory vector database.
Connect the ChatOpenAI and the OpenAI embeddings and QA retrieval into the chatbot.

The final flow can answer questions about Apify docs.

![Flowise and Apify](images/flowise-2.png)
![Flowise and Apify](../images/flowise-2.png)

For more information visit the Flowise [documentation](https://flowiseai.com/).

Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: 🦜🔗 LangChain
title: 🦜🔗 LangChain integration
sidebar_label: LangChain
description: Learn how to integrate Apify with 🦜🔗 LangChain, in order to feed vector databases and LLMs with data crawled from the web.
sidebar_position: 11.19
sidebar_position: 1
slug: /integrations/langchain
---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
---
title: LlamaIndex
title: LlamaIndex integration
description: Learn how to integrate Apify with LlamaIndex in order to feed vector databases and LLMs with data crawled from the web.
sidebar_position: 11.21
sidebar_label: Llamaindex
sidebar_position: 2
slug: /integrations/llama
---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
---
title: Pinecone
title: Pinecone integration
description: Learn how to integrate Apify with Pinecone to feed data crawled from the web into the Pinecone vector database.
sidebar_position: 11.22
sidebar_label: Pinecone
sidebar_position: 4
slug: /integrations/pinecone
toc_min_heading_level: 2
toc_max_heading_level: 4
Expand Down Expand Up @@ -41,11 +42,11 @@ The examples utilize the Website Content Crawler Actor, which deeply crawls webs

1. Once you have the crawler ready, navigate to the integration section and add Apify’s Pinecone integration.

![Website Content Crawler with Pinecone integration](./images/pinecone-wcc-integration.png)
![Website Content Crawler with Pinecone integration](../images/pinecone-wcc-integration.png)

1. Select when to trigger this integration (typically when a run succeeds) and fill in all the required fields for the Pinecone integration. You can learn more about the input parameters at the [Pinecone integration input schema](https://apify.com/jan.turon/pinecone-integration/input-schema).

![Pinecone integration configuration](./images/pinecone-integration-setup.png)
![Pinecone integration configuration](../images/pinecone-integration-setup.png)

1. For a comprehensive explanation on how to combine Actors to accomplish more complex tasks, refer to the guide on [Actor-to-Actor](https://blog.apify.com/connecting-scrapers-apify-integration/) integrations.

Expand Down
2 changes: 2 additions & 0 deletions sources/platform/integrations/data-storage/_category_.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
label: Data Storage
position: 5
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: Airbyte
sidebar_lbel: Airbyte
title: Airbyte integration
description: Learn how to integrate your Apify datasets with Airbyte.
sidebar_position: 11.15
sidebar_label: Airbyte
sidebar_position: 1
slug: /integrations/airbyte
---

Expand All @@ -23,19 +23,19 @@ To use Airbyte's Apify connector you need to:
Once you have all the necessary accounts set up, you need to set up the Apify connector.
To do so, you will need to navigate to **Sources** tab in Airbyte and select **Apify Dataset**

![Airbyte sources tab](./images/airbyte-sources.png)
![Airbyte sources tab](../images/airbyte-sources.png)

You will need to provide a **dataset ID** and your Apify API Token. You can find both of these in [Apify Console](https://console.apify.com).

![Airbyte source setup](./images/airbyte-source-setup.png)
![Airbyte source setup](../images/airbyte-source-setup.png)

To find your **dataset ID**, you need to navigate to the **Storage** tab in Apify Console. Copy it and paste it in Airbyte.

![Datasets in app](./images/datasets-app.png)
![Datasets in app](../images/datasets-app.png)

To find your Apify API token, you need to navigate to the **Settings** tab and select **Integrations**. Copy it and paste it in the relevant field in Airbyte.

![Integrations token](./images/apify-integrations-token.png)
![Integrations token](../images/apify-integrations-token.png)

And that's it! You now have Apify datasets set up as a Source, and you can use Airbyte to transfer your datasets to one of the available destinations.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: Google Drive
sidebar_label: Google Drive
title: Google Drive integration
description: Learn how to integrate Apify with Google Drive
sidebar_position: 11.18
sidebar_label: Google Drive
sidebar_position: 3
slug: /integrations/drive
---

Expand All @@ -26,19 +26,19 @@ To use the Apify integration for Google Drive, you will need:

1. Head over to **Integrations** tab in your saved task and click on the Google Drive integration.

![Google Drive integration](./images/gdrive/google-maps-task-integrations.png)
![Google Drive integration](../images/gdrive/google-maps-task-integrations.png)

1. Click on **Sign in with Google** button and select the account with which you want to use the integration.

![Google Drive integration signup](./images/gdrive/google-maps-task-integration-setup.png)
![Google Drive integration signup](../images/gdrive/google-maps-task-integration-setup.png)

1. Select the desired format of the dataset file.

![Google Drive integration format](./images/gdrive/google-maps-task-integration-format.png)
![Google Drive integration format](../images/gdrive/google-maps-task-integration-format.png)

1. Click on **Save** & enable the integration.

![Google Drive integration save](./images/gdrive/google-maps-task-integration-save.png)
![Google Drive integration save](../images/gdrive/google-maps-task-integration-save.png)

Once this is done, run your Actor to test whether the integration is working.

Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: Keboola
sidebar_lbel: Keboola
title: Keboola integration
description: Learn how to integrate your Apify datasets with Airbyte.
sidebar_position: 11.16
sidebar_label: Keboola
sidebar_position: 2
slug: /integrations/keboola
---

Expand All @@ -25,25 +25,25 @@ To use the Apify integration on Keboola, you will need to:

Once your Keboola account is ready and you are logged in, navigate to the **Components** section in the top menu and click the **Add Component** button.

![Keboola dashboard](./images/keboola/keboola-dashboard.png)
![Keboola dashboard](../images/keboola/keboola-dashboard.png)

In the list of available Components, find and select the **Apify** from Data Sources and click on the **Add Component** button.

![Keboola component search](./images/keboola/keboola-components.png)
![Keboola component search](../images/keboola/keboola-components.png)

On the new window, click **Connect** to my data**

![Keboola Connector](./images/keboola/keboola-connector.png)
![Keboola Connector](../images/keboola/keboola-connector.png)

Provide a name and description for your configuration, then click the **Create** Configuration** button.

![Keboola configuration setup](./images/keboola/keboola-create-configuration.png)
![Keboola configuration setup](../images/keboola/keboola-create-configuration.png)

### Step 2: Configure the Apify Data Source

With the new configuration created, you can now configure the data source to retrieve the needed data. Click on the **Configure Component** button to begin the setup process.

![Keboola component configuration](./images/keboola/keboola-configure-component.png)
![Keboola component configuration](../images/keboola/keboola-configure-component.png)


#### Choose an action
Expand All @@ -56,13 +56,13 @@ In the next step, you can choose the action you want to perform:
- **Retrieve items from the last task run** - This action takes the dataset of a specific task last run.
- **Retrieve items from Dataset**: This action takes the dataset ID or dataset name and retrieves all items from that dataset.

![Keboola component action setup ](./images/keboola/keboola-component-setup.png)
![Keboola component action setup ](../images/keboola/keboola-component-setup.png)

#### Authentication

After selecting the action, you will need to provide your Apify API credentials. You can find these credentials on your Apify account page by navigating to **Settings > Integrations** and copying them into the provided form.

![Keboola API authentication](./images/keboola/keboola-setup-api-token.png)
![Keboola API authentication](../images/keboola/keboola-setup-api-token.png)

#### Specifications

Expand All @@ -77,13 +77,13 @@ In the specifications step, you can set up various options for your Actor run:

Once you have filled in all the necessary options, click the **Save** button to save your configuration.

![Keboola component specification setup](./images/keboola/keboola-setup-specification.png)
![Keboola component specification setup](../images/keboola/keboola-setup-specification.png)

### Step 3: Run the configured Data Source

After your data source has been configured, you can run it by clicking the **Run** button in the upper-right corner of your configuration.

![Keboola run configured component](./images/keboola/keboola-run-component.png)
![Keboola run configured component](../images/keboola/keboola-run-component.png)

You can monitor the progress of your run in the job detail section on the right-hand side of the page.

Expand Down
16 changes: 1 addition & 15 deletions sources/platform/integrations/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -116,12 +116,6 @@ If you use one of the main automation platforms, Apify support is here for you.
imageUrl="/img/platform/integrations/zapier.svg"
smallImage
/>
<Card
title="Transposit"
to="https://www.transposit.com/docs/guides/apify/"
imageUrl="/img/platform/integrations/transposit.svg"
smallImage
/>
<Card
title="Telegram (via Zapier)"
to="./integrations/telegram"
Expand All @@ -130,7 +124,6 @@ If you use one of the main automation platforms, Apify support is here for you.
/>
</CardGrid>


## Data pipelines, ETLs, and AI/LLM tools

The Apify platform integrates with popular ETL and data pipeline services, enabling you to integrate Apify Actors directly into your data integration processes.
Expand All @@ -148,12 +141,6 @@ The Apify platform integrates with popular ETL and data pipeline services, enabl
imageUrl="/img/platform/integrations/airbyte.svg"
smallImage
/>
<Card
title="Hevo"
to="https://docs.hevodata.com/sources/engg-analytics/apify/"
imageUrl="/img/platform/integrations/hevo.svg"
smallImage
/>
</CardGrid><br />


Expand Down Expand Up @@ -181,10 +168,9 @@ If you are working on an AI/LLM-related project, we recommend you look into the
<Card
title="Pinecone"
to="./integrations/pinecone"
imageUrl="/img/platform/integrations/pinecone.png"
imageUrl="/img/platform/integrations/pinecone.svg"
smallImage
/>

</CardGrid>

## Other Actors
Expand Down
10 changes: 5 additions & 5 deletions sources/platform/integrations/integrate_with_apify.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
---
title: Integrate with Apify 🤝
sidebar_label: Integrate with Apify
sidebar_label: Create new integration
description: Learn about how to integrate your service with Apify to benefit from a mutual integration.
sidebar_position: 90.00
slug: /integrations/integrate
---

If you are building a service and your users could benefit from integrating with Apify or vice versa, we would love to hear from you! Contact us at [partners@apify.com](mailto:partners@apify.com) to discuss potential collaboration. We are always looking for ways to make our platform more useful and powerful for our users. Here are some examples of services and tools integrating with Apify:

- [Langchain](./langchain.md) integration enables developers to build their projects, such as AI chatbots, with Langchain to feed them with data from the web easily.
- [Zapier](./zapier.md) integration allows Zapier users to enrich their automation workflows with data from the web or to add additional Actions performed by [Apify Actors](https://apify.com/store).
- [Keboola](./keboola.md) integration enables Keboola users to easily pull data crawled from the web into their data pipelines.
- [Langchain](./ai/langchain.md) integration enables developers to build their projects, such as AI chatbots, with Langchain to feed them with data from the web easily.
- [Zapier](./workflows-and-notifications/zapier.md) integration allows Zapier users to enrich their automation workflows with data from the web or to add additional Actions performed by [Apify Actors](https://apify.com/store).
- [Keboola](/platform/integrations/keboola) integration enables Keboola users to easily pull data crawled from the web into their data pipelines.

and many more that you can see at [integrations](./index.mdx).

Expand All @@ -19,7 +19,7 @@ and many more that you can see at [integrations](./index.mdx).
To integrate your service with Apify, you have two options. You can either:

- build an [Apify actor](https://apify.com/docs/actor) that will be used as integration within the [Apify Console](https://console.apify.com)
- build an external integration, such as [Zapier](./zapier.md).
- build an external integration, such as [Zapier](./workflows-and-notifications/zapier.md).

![Integration-ready Actors](./images/integration-ready-actors.png)

Expand Down
2 changes: 2 additions & 0 deletions sources/platform/integrations/programming/_category_.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
label: 'Programming'
position: 2
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
---
title: API
title: API integration
description: Learn how to integrate with Apify via API.
sidebar_position: 11.01
sidebar_label: API
sidebar_position: 1
slug: /integrations/api
---

Expand All @@ -21,7 +22,7 @@ You are not required to those packages—the REST API works with any HTTP client

To access the Apify API in your integrations, you need to authenticate using your secret API token. You can find it on the [Integrations](https://console.apify.com/account?tab=integrations) page in Apify Console. Give your token a reasonable description, and never use one token for several services, much like you shouldn't use the same password for different accounts.

![Integrations page in Apify Console](./images/api-token.png)
![Integrations page in Apify Console](../images/api-token.png)

## Authentication

Expand All @@ -39,7 +40,7 @@ do not require an authentication token because they contain a hard-to-guess iden

When working under an organization account, you will see two types of API tokens on the Integrations page.

![Integrations page in the Apify Console in organizatoin mode](./images/api-token-organization.png)
![Integrations page in the Apify Console in organizatoin mode](../images/api-token-organization.png)

The Personal API tokens are different from your own Personal API tokens mentioned above. If you use this token in an integration, it will have the same permissions that you have within the organization, and all the operations you use it for will be ascribed to you.

Expand Down
Loading