TogetherAI_API_with_LangChain.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Checkout my [Twitter(@rohanpaul_ai)](https://twitter.com/rohanpaul_ai) for daily LLM bits"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import logging\n",
    "import together\n",
    "from langchain.llms.base import LLM\n",
    "from langchain import PromptTemplate, LLMChain\n",
    "from dotenv import load_dotenv\n",
    "load_dotenv()\n",
    "logging.basicConfig(level=logging.INFO)\n",
    "\n",
    "class TogetherLLM(LLM):\n",
    "    model: str = \"togethercomputer/llama-2-7b-chat\"\n",
    "    together_api_key: str = os.environ[\"TOGETHER_API_KEY\"]\n",
    "    temperature: float = 0.7\n",
    "    max_tokens: int = 512\n",
    "\n",
    "    def __init__(self, model=None, max_tokens=None, temperature=None):\n",
    "        if model:\n",
    "            self.model = model\n",
    "        if max_tokens:\n",
    "            self.max_tokens = max_tokens\n",
    "        if temperature:\n",
    "            self.temperature = temperature\n",
    "\n",
    "    @property\n",
    "    def llm_type(self) -> str:\n",
    "        return \"together\"\n",
    "\n",
    "    def __call__(self, prompt: str, **kwargs) -> str:\n",
    "        try:\n",
    "            logging.info(\"Calling Together endpoint.\")\n",
    "            return self.make_api_call(prompt)\n",
    "        except Exception as e:\n",
    "            logging.error(f\"Error in TogetherLLM call: {e}\", exc_info=True)\n",
    "            raise\n",
    "\n",
    "    def make_api_call(self, prompt: str) -> str:\n",
    "        together.api_key = self.together_api_key\n",
    "        output = together.Complete.create(\n",
    "            prompt,\n",
    "            model=self.model,\n",
    "            max_tokens=self.max_tokens,\n",
    "            temperature=self.temperature,\n",
    "        )\n",
    "        logging.info(\"API call successful.\")\n",
    "        return output['output']['choices'][0]['text']\n",
    "\n",
    "# Now let's use the class\n",
    "llm = TogetherLLM(\n",
    "    model=\"togethercomputer/llama-2-7b-chat\",\n",
    "    max_tokens=256,\n",
    "    temperature=0.8\n",
    ")\n",
    "\n",
    "prompt_template = \"You are a friendly AI. Answer the following question: {question}\"\n",
    "prompt = PromptTemplate(\n",
    "    input_variables=[\"question\"], template=prompt_template\n",
    ")\n",
    "chat = LLMChain(llm=llm, prompt=prompt)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Integrating TogetherAI with LangChain 🦙\n",
    "\n",
    "📌 `langchain.llms.base.LLM` is an abstract base class. The purpose of this class is to expose a simpler interface for working with LLMs, rather than expect the user to implement the full _generate method."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "------------\n",
    "\n",
    "### the significance of the this block of code\n",
    "\n",
    "```py\n",
    "  @property\n",
    "    def _llm_type(self) -> str:\n",
    "        \"\"\"Return type of LLM.\"\"\"\n",
    "        return \"together\"\n",
    "```\n",
    "\n",
    "\n",
    "📌 This method, `_llm_type`, serves as a getter for a property of the `TogetherLLM` object. \n",
    "\n",
    "📌 The `@property` decorator transforms the `_llm_type` method to behave like an attribute of the `TogetherLLM` class, rather than a method that needs to be explicitly called. This means you can access the type of the LLM by referring to `instance._llm_type` rather than `instance._llm_type()`.\n",
    "\n",
    "📌 The significance of defining this property in the class is primarily for internal use within the `TogetherLLM` or its parent classes. It could be used for type checking, logging, conditional processing based on the LLM type, or other purposes where identifying the type of the language model is necessary."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---------\n",
    "\n",
    "### 📌 The reason for using `__call__` method above\n",
    "\n",
    "The job of this method is to Check Cache and run the LLM on the given prompt and input.\n",
    "\n",
    "📌 When you define the `__call__` method in a class, it enables you to use instances of that class like this: `instance(parameters)`, where `instance` is an object of the class.\n",
    "\n",
    "📌 So with `__call__` the `TogetherLLM` class instances becomes callable objects. \n",
    "\n",
    "📌 That means, instead of having to explicitly call a method like `instance.call(prompt)`, you can simply use `instance(prompt)`. This makes the code more concise and can improve readability, especially for users who are familiar with functional programming paradigms.\n",
    "\n",
    "📌 Another advantage is that it allows the `TogetherLLM` class to integrate more seamlessly with Python features and libraries that expect callable objects. For instance, if you're using a higher-order function that takes a function as an argument, you could directly pass an instance of `TogetherLLM` instead of having to wrap it in another function or lambda."
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}