Bedrock Antropic

docs/how-to/deploy-on-gcp/deploy-on-google-cloud-platform.mdx

@@ -0,0 +1,160 @@
+Learn how to deploy LLMstudio as a containerized application on Google Kubernetes Engine and make calls from a local repository.
+
+
+## Prerequisites
+To follow this guide you need to have the following set-up:
+
+- A **project** on google cloud platform.
+- **Kubernetes Engine** API enabled on your project.
+- **Kubernetes Engine Admin** role for the user performing the guide.
+
+## Deploy LLMstudio
+
+This example demonstrates a public deployment. For a private service accessible only within your enterprise infrastructure, deploy it within your own Virtual Private Cloud (VPC).
+<Steps>
+  <Step title="Navigate to Kubernetes Engine">
+    Begin by navigating to the Kubernetes Engine page.
+  </Step>
+  <Step title="Select Deploy">
+    Go to **Workloads** and **Create a new Deployment**.
+    <Frame>
+      <img src="how-to/deploy-on-gcp/step-2.png" />
+    </Frame>
+  </Step>
+  <Step title="Name Your Deployment">
+    Rename your project. We will call the one in this guide **llmstudio-on-gcp**.
+    <Frame>
+      <img src="how-to/deploy-on-gcp/step-3.png" />
+    </Frame>
+  </Step>
+  <Step title="Select Your Cluster">
+    Choose between **creating a new cluster** or **using an existing cluster**. 
+    For this guide, we will create a new cluster and use the default region.
+    <Frame>
+      <img src="how-to/deploy-on-gcp/step-4.png" />
+    </Frame>
+  </Step>
+  <Step title="Proceed to Container Details">
+    Once done done with the **Deployment configuration**, proceed to **Container details**.
+  </Step>
+  <Step title="Set Image Path">
+    In the new container section, select **Existing container image**.
+
+
+    Copy the path to LLMstudio's image available on Docker Hub.
+    ```bash Image Path
+    tensoropsai/llmstudio:latest
+    ```
+    Set it as the **Image path** to your container.
+    <Frame>
+      <img src="how-to/deploy-on-gcp/step-6.png" />
+    </Frame>
+  </Step>
+  <Step title="Set Environment Variables">
+    Configure the following mandatory environment variables:
+| Environment Variable       | Value     |
+|----------------------------|-----------|
+| `LLMSTUDIO_ENGINE_HOST`    | 0.0.0.0   |
+| `LLMSTUDIO_ENGINE_PORT`    | YOUR_DESIRED_PORT     |
+| `LLMSTUDIO_TRACKING_HOST`  | 0.0.0.0   |
+| `LLMSTUDIO_TRACKING_PORT`  | YOUR_DESIRED_PORT     |
+
+Additionally, set the `GOOGLE_API_KEY` environment variable to enable calls to Google's Gemini models.
+<Tip>Refer to **SDK/LLM/Providers** for instructions on setting up other providers.</Tip>
+
+<Frame>
+  <img src="how-to/deploy-on-gcp/step-7.png" />
+</Frame>
+
+  </Step>
+  <Step title="Proceed to Expose (Optional)">
+    After configuring your container, proceed to **Expose (Optional)**.
+  </Step>
+  <Step title="Expose Ports">
+    Select **Expose deployment as a new service** and leave the first item as is.
+
+    <Frame>
+      <img src="how-to/deploy-on-gcp/step-9-1.png" />
+    </Frame>
+
+    Add two other items, and expose the ports defined in the **Set Environment Variables** step.
+
+    <Frame>
+      <img src="how-to/deploy-on-gcp/step-9-2.png" />
+    </Frame>
+  </Step>
+  <Step title="Deploy">
+    After setting up and exposing the ports, press **Deploy**.
+  <Check>You have successfully deployed **LLMstudio on Google Cloud Platform**!</Check>
+  </Step>
+
+</Steps>
+
+## Make a Call
+Now let's make a call to our LLMstudio instance on GCP!
+
+
+
+<Steps>
+  <Step title="Set Up Project">
+    Setup a simple project with this two files:
+    1. `simple-call.ipynb`
+    2. `.env`
+  </Step>
+
+  <Step title="Set Up Files">
+    <Tabs>
+      <Tab title=".env">
+
+        Go to your newly deployed **Workload**, scroll to the **Exposing services** section, and take note of the Host of your endpoint.
+        <Frame>
+          <img src="how-to/deploy-on-gcp/step-env.png" />
+        </Frame>
+
+        Create your `.env` file with the following:
+
+        ```env .env
+        LLMSTUDIO_ENGINE_HOST = "YOUR_HOST"
+        LLMSTUDIO_ENGINE_PORT = "YOUR_ENGINE_PORT"
+        LLMSTUDIO_TRACKING_HOST = "YOUR_TRACKING_PORT"
+        LLMSTUDIO_TRACKING_PORT = "YOUR_HOST"
+        ```
+
+        <Check>You are done seting up you **.env** file!</Check>
+
+
+
+
+      </Tab>
+      <Tab title="simple-call.ipynb">
+        Start by importing llmstudio:
+        ```python 1st cell
+        from llmstudio import LLM
+        ```
+
+        Set up your LLM. We will be using `gemini-1.5-flash` for this guide.
+        ```python 2nd cell
+        llm = LLM('vertexai/gemini-1.5-flash')
+        ```
+
+        Chat with your model.
+        ```python 3rd cell
+        llm.chat('Hello!')
+        print(response.chat_output)
+        ```
+
+        <Frame>
+          <img src="how-to/deploy-on-gcp/step-llmstudio-call.png" />
+        </Frame>
+
+
+        <Check>You are done calling llmstudio on GCP!</Check>
+
+      </Tab>
+
+    </Tabs>
+  </Step>
+
+
+</Steps>
+

docs/mint.json

@@ -87,3 +87,4 @@
     "baseUrl": "http://localhost:8000"
   }
 }
+

docs/sdk/llm/chat.mdx

@@ -0,0 +1,86 @@
+Make chat calls using your LLM.
+
+## Parameters
+The llm.chat method can have the following parameters.
+| Parameter      | Type   | Description                                                                 |
+|-------------------|--------|-----------------------------------------------------------------------------|
+| `input `       | str    | The input message to send to the chat model.                                   |
+| `is_stream`    | bool  | The temperature parameter for the model.                                       |
+| `**kwargs`          | dict  | Additional parameters to pass to the chat model.                           |
+
+<Check>Refer to your provider-specific documentation for additional kwargs you can use.</Check>
+
+## Returns
+| Output            | Type   | Description                                                                 |
+|-------------------|--------|-----------------------------------------------------------------------------|
+| `ChatCompletion`  | object | A chat completion object in the OpenAI format + metrics computed by LLMstudio.|
+
+
+## Usage
+Here's how to use `.chat()` to make calls to your LLM.
+
+<Steps>
+    <Step>
+        Start by importing LLM.
+        ```python
+        from llmstudio import LLM
+        ```
+    </Step>
+    <Step >
+        Set up an LLM from your desired provider.
+        ```python
+        llm = LLM('openai/gpt-4o')
+        ```
+    </Step>
+    <Step>
+        Create your message. Your message can be a simple `string` or a message in the `OpenAI format`.
+
+       <Tabs>
+            <Tab title="String format">
+                ```python
+                message = "Hello, how are you today?"
+                ```
+            </Tab>
+            <Tab title="OpenAI format">
+                ```python
+                message = [
+                {"role": "system", "content": "You are a helpfull assistant."},
+                {"role": "user", "content": "Hello, how are you today?"}
+                ]
+                ```
+            </Tab>
+        </Tabs> 
+
+    </Step>
+    <Step>
+
+        <Tabs>
+            <Tab title="Non-stream response">
+                Get your response.
+                ```python
+                response = llm.chat(message)
+                ```
+
+                Vizualize your response.
+                 ```python
+                print(response)
+                ```
+            </Tab>
+            <Tab title="Stream response">
+                Get your response.
+                ```python
+                response = llm.chat(message, is_stream = True)
+                ```
+
+                Vizualize your response.
+                 ```python
+                for chunk in response:
+                    print(chunk)
+                ```
+            </Tab>
+        </Tabs> 
+
+        <Check>You are done chating with your **LLMstudio LLM**!</Check>
+    </Step>
+</Steps>
+

docs/sdk/llm/providers/anthropic.mdx

@@ -0,0 +1,101 @@
+Interact with your Anthropic models using LLMstudios LLM.
+
+## Supported models
+1. `claude-3-opus-20240229`
+2. `claude-3-sonnet-2024022`
+3. `claude-3-haiku-20240307`
+4. `claude-2.1`
+5. `claude-2`
+6. `claude-instant-1.2`
+
+## Parameters
+An Anthropic LLM interface can have the following parameters:
+| Parameter         | Type   | Description                                                                 |
+|-------------------|--------|-----------------------------------------------------------------------------|
+| `api_key`         | str    | The API key for authentication.                                              |
+| `temperature`     | float  | The temperature parameter for the model.                                     |
+| `top_p`           | float  | The top-p parameter for the model.                                           |
+| `max_tokens`      | int    | The maximum number of tokens for the model's output.                         |
+| `top_k`           | int    | The top-k parameter for the model.                                           |
+
+
+## Usage
+Here is how you setup an interface to interact with your Anthropic models.
+
+<Tabs>
+  <Tab title="w/ .env">
+    <Steps>
+        <Step>
+            Create a `.env` file with you `ANTHROPIC_API_KEY`
+
+            <Warning>Make sure you call your environment variable ANTHROPIC_API_KEY</Warning>
+            ```bash
+            ANTHROPIC_API_KEY="YOUR-KEY"
+            ```
+        </Step>
+        <Step >
+            In your python code, import LLM from llmstudio.
+            ```python
+            from llmstudio import LLM
+            ```
+        </Step>
+        <Step>
+            Create your **llm** instance.
+            ```python
+            llm = LLM('anthropic/{model}')
+            ```
+        </Step>
+        <Step>
+            **Optional:** You can add your parameters as follows:
+            ```python
+            llm = LLM('anthropic/model', 
+            temperature= ...,
+            max_tokens= ...,
+            top_p= ...,
+            frequency_penalty= ...,
+            presence_penalty= ...)
+            ```
+            <Check>You are done setting up your **Anthropic LLM**!</Check>
+        </Step>
+    </Steps>
+  </Tab>
+  <Tab title="w/o .env">
+    <Steps>
+        <Step >
+            In your python code, import LLM from llmstudio.
+            ```python
+            from llmstudio import LLM
+            ```
+        </Step>
+        <Step>
+            Create your **llm** instance.
+            ```python
+            llm = LLM('anthropic/{model}',api_key="YOUR_API_KEY")
+            ```
+        </Step>
+        <Step>
+            **Optional:** You can add your parameters as follows:
+            ```python
+            llm = LLM('anthropic/model', 
+            temperature= ...,
+            max_tokens= ...,
+            top_p= ...,
+            frequency_penalty= ...,
+            presence_penalty= ...)
+            ```
+            <Check>You are done setting up your **Anthropic LLM**!</Check>
+        </Step>
+    </Steps>
+  </Tab>
+</Tabs>
+
+
+## What's next?
+<CardGroup cols={2}>
+    <Card title="LLM.chat()" icon="link" href="../chat">
+    Learn how to send messeges and recieve responses next!
+    </Card>
+    <Card title="Tool calling Agent" icon="link" href="../../../how-to/build-a-tool-agent">
+    Learn how to build a tool calling agent using llmstudio.
+    </Card>
+</CardGroup>