CrewAI integration tutorial

docs.json

@@ -142,6 +142,21 @@
             "group": "Storage",
             "pages": ["storage/network-volumes", "storage/s3-api"]
           },
+          {
+          "group": "Integrations",
+          "pages": [
+            "integrations/overview",
+            {
+              "group": "Guides",
+              "pages":[
+                "integrations/n8n-integration",
+                "integrations/dstack",
+                "integrations/mods",
+                "integrations/skypilot"
+                ]
+            }
+          ]
+        },
           {
             "group": "Hub",
             "pages": [
@@ -191,14 +206,6 @@
                   "references/troubleshooting/manage-payment-cards"
                 ]
               },
-              {
-                "group": "Integrations",
-                "pages": [
-                  "integrations/dstack",
-                  "integrations/mods",
-                  "integrations/skypilot"
-                ]
-              },
               {
                 "group": "Migrations",
                 "pages": [
@@ -472,6 +479,10 @@
   },
 
   "redirects": [        
+    {
+      "source": "/serverless/development/integrations",
+      "destination": "/integrations/overview"
+    },
     {
       "source": "/references/faq",
       "destination": "/references/troubleshooting/zero-gpus"

integrations/crewai-integration.mdx

@@ -0,0 +1,130 @@
+---
+title: "Integrate Runpod with CrewAI"
+sidebarTitle: CrewAI
+description: "Learn how to deploy a vLLM worker on Runpod and connect it to CrewAI for orchestrating autonomous AI agents."
+tag: "BETA"
+---
+
+Learn how to integrate Runpod Serverless with CrewAI, a framework for orchestrating role-playing autonomous AI agents. By the end of this tutorial, you'll have a vLLM endpoint running on Runpod that you can use to power your CrewAI agents.
+
+## What you'll learn
+
+In this tutorial, you'll learn how to:
+
+* Deploy a vLLM worker on Runpod Serverless.
+* Configure your vLLM endpoint for OpenAI compatibility.
+* Connect CrewAI to your Runpod endpoint.
+* Test your integration with a simple agent.
+
+## Requirements
+
+Before you begin, you'll need:
+
+* A [Runpod account](/get-started/manage-accounts) (with available credits).
+* A [Runpod API key](/get-started/api-keys).
+* A [CrewAI](https://crewai.com/) account.
+* (Optional) For gated models, you've created a [Hugging Face access token](https://huggingface.co/docs/hub/en/security-tokens).
+
+## Step 1: Deploy a vLLM worker on Runpod
+
+First, you'll deploy a vLLM worker to serve your language model.
+
+<Steps>
+  <Step title="Create a new vLLM endpoint">
+    Open the [Runpod console](https://www.console.runpod.io/serverless) and navigate to the Serverless page.
+
+    Click **New Endpoint** and select **vLLM** under **Ready-to-Deploy Repos**.
+
+  </Step>
+
+  <Step title="Configure your endpoint">
+    <Tip>
+    For more details on vLLM deployment options, see [Deploy a vLLM worker](/serverless/vllm/get-started).
+    </Tip>
+
+    In the deployment modal:
+
+    * Enter the model name or Hugging Face model URL (e.g., `openchat/openchat-3.5-0106`).
+    * Expand the **Advanced** section:
+      * Set **Max Model Length** to `8192` (or an appropriate context length for your model).
+      * You may need to enable tool calling and set an appropriate reasoning parser depending on your model.
+    * Click **Next**.
+    * Click **Create Endpoint**.
+
+    Your endpoint will now begin initializing. This may take several minutes while Runpod provisions resources and downloads your model. Wait until the status shows as **Running**.
+  </Step>
+
+  <Step title="Copy your endpoint ID">
+    Once deployed, navigate to your endpoint in the Runpod console. You can find your endpoint ID in the **Overview** tab:
+
+    <Frame>
+      <img src="/images/serverless-endpoint-id.png" />
+    </Frame>
+
+    You can also find your endpoint ID in the URL of the endpoint detail page. For example, if the URL for your endpoint is `https://console.runpod.io/serverless/user/endpoint/isapbl1e254mbj`, the endpoint ID is `isapbl1e254mbj`.
+
+     Copy your endpoint ID to the clipboard. You'll need this to connect your endpoint to CrewAI.
+  </Step>
+</Steps>
+
+## Step 2: Connect CrewAI to your Runpod endpoint
+
+Now you'll configure CrewAI to use your Runpod endpoint as an OpenAI-compatible API.
+
+<Steps>
+  <Step title="Open LLM connections settings">
+    Open the CrewAI dashboard and open the **LLM connections** section.
+  </Step>
+
+  <Step title="Select custom OpenAI provider">
+    Under **Add New Conncection**, enter a name for your connection. Then under **Provider**, select **custom-openai-compatible** from the dropdown menu.
+  </Step>
+
+  <Step title="Configure API credentials">
+    Configure the connection with your Runpod credentials:
+
+    * For `OPENAI_API_KEY`, use your Runpod API Key. You can find or create API keys in the settings page of the [Runpod console](https://console.runpod.io/user/settings).
+
+    For `OPENAI_API_BASE`, enter the base URL for your vLLM's OpenAI-compatible endpoint:
+
+    ```
+    https://api.runpod.ai/v2/ENDPOINT_ID/openai/v1
+    ```
+
+    Replace `ENDPOINT_ID` with your actual endpoint ID from Step 1.
+  </Step>
+
+  <Step title="Test the connection">
+    Click **Fetch Available Models** to test the connection. If successful, CrewAI will retrieve the list of models available on your endpoint.
+  </Step>
+</Steps>
+
+## Step 3: Test your integration
+
+To verify that your CrewAI agents can use your Runpod endpoint, you can try using it in an automation:
+
+<Steps>
+  <Step title="Create a test agent">
+    Create a blank automation and add an Agent node. Click the edit button to configure it. the Agent node. Under **Model**, select your Runpod endpoint from the dropdown menu (if you have trouble finding it, try filtering for **Custom OpenAI Compatible** models).
+  </Step>
+
+  <Step title="Run a test task">
+    Assign a simple task to your agent and run it to verify that it can communicate with your Runpod endpoint.
+  </Step>
+
+  <Step title="Monitor requests">
+    Monitor requests from your CrewAI agents in the endpoint details page of the Runpod console.
+  </Step>
+
+  <Step title="Verify responses">
+    Confirm that your agent is receiving appropriate responses from your model running on Runpod.
+  </Step>
+</Steps>
+
+## Next steps
+
+Now that you've integrated Runpod with CrewAI, you can 
+
+* Build complex multi-agent systems using your Runpod endpoint to serve the necessary models.
+* Explore other [integration options](/integrations/overview).
+* Learn more about [OpenAI compatibility](/serverless/vllm/openai-compatibility) features in vLLM.