cookbook/liteLLM_function_calling.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "## Demo Notebook of Function Calling with liteLLM\n",
        "- Supported Providers for Function Calling\n",
        "  - OpenAI - `gpt-4-0613` and `gpt-3.5-turbo-0613`\n",
        "- In this notebook we use function calling with `litellm.completion()`"
      ],
      "metadata": {
        "id": "vnvlwUDZK7VA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "## Install liteLLM\n",
        "!pip install litellm"
      ],
      "metadata": {
        "id": "KrINCwRfLgZV"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import os, litellm\n",
        "from litellm import completion"
      ],
      "metadata": {
        "id": "nK7zR5OgLlh2"
      },
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "os.environ['OPENAI_API_KEY'] = \"\" #@param"
      ],
      "metadata": {
        "id": "dCQlyBxKLqbA"
      },
      "execution_count": 27,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Define Messages, Functions\n",
        "We create a get_current_weather() function and pass that to GPT 3.5\n",
        "\n",
        "See OpenAI docs for this: https://openai.com/blog/function-calling-and-other-api-updates"
      ],
      "metadata": {
        "id": "gfdGv-FMRCdX"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "messages = [\n",
        "    {\"role\": \"user\", \"content\": \"What is the weather like in Boston?\"}\n",
        "]\n",
        "\n",
        "def get_current_weather(location):\n",
        "  if location == \"Boston, MA\":\n",
        "    return \"The weather is 12F\"\n",
        "\n",
        "functions = [\n",
        "    {\n",
        "      \"name\": \"get_current_weather\",\n",
        "      \"description\": \"Get the current weather in a given location\",\n",
        "      \"parameters\": {\n",
        "        \"type\": \"object\",\n",
        "        \"properties\": {\n",
        "          \"location\": {\n",
        "            \"type\": \"string\",\n",
        "            \"description\": \"The city and state, e.g. San Francisco, CA\"\n",
        "          },\n",
        "          \"unit\": {\n",
        "            \"type\": \"string\",\n",
        "            \"enum\": [\"celsius\", \"fahrenheit\"]\n",
        "          }\n",
        "        },\n",
        "        \"required\": [\"location\"]\n",
        "      }\n",
        "    }\n",
        "  ]"
      ],
      "metadata": {
        "id": "ERzsP1sfM19C"
      },
      "execution_count": 25,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Call gpt-3.5-turbo-0613 to Decide what Function to call"
      ],
      "metadata": {
        "id": "NX6by2VuRPnp"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "response = completion(model=\"gpt-3.5-turbo-0613\", messages=messages, functions=functions)\n",
        "print(response)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QVoJ5PtxMlVx",
        "outputId": "efe7a81f-e04a-4afc-aa60-a2b2648f5fb9"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{\n",
            "  \"id\": \"chatcmpl-7mX4RiqdoislVEqfmfVjFSKp3hyIy\",\n",
            "  \"object\": \"chat.completion\",\n",
            "  \"created\": 1691801223,\n",
            "  \"model\": \"gpt-3.5-turbo-0613\",\n",
            "  \"choices\": [\n",
            "    {\n",
            "      \"index\": 0,\n",
            "      \"message\": {\n",
            "        \"role\": \"assistant\",\n",
            "        \"content\": null,\n",
            "        \"function_call\": {\n",
            "          \"name\": \"get_current_weather\",\n",
            "          \"arguments\": \"{\\n  \\\"location\\\": \\\"Boston, MA\\\"\\n}\"\n",
            "        }\n",
            "      },\n",
            "      \"finish_reason\": \"function_call\"\n",
            "    }\n",
            "  ],\n",
            "  \"usage\": {\n",
            "    \"prompt_tokens\": 82,\n",
            "    \"completion_tokens\": 18,\n",
            "    \"total_tokens\": 100\n",
            "  }\n",
            "}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Parse GPT 3.5 Response\n",
        "Read Information about what Function to Call"
      ],
      "metadata": {
        "id": "Yu0o2saDNLx8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "function_call_data = response[\"choices\"][0][\"message\"][\"function_call\"]\n",
        "function_call_data"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "u1DzXLJsNOR5",
        "outputId": "177e9501-0ce2-4619-9067-3047f18f6c79"
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<OpenAIObject at 0x7922c70ce930> JSON: {\n",
              "  \"name\": \"get_current_weather\",\n",
              "  \"arguments\": \"{\\n  \\\"location\\\": \\\"Boston, MA\\\"\\n}\"\n",
              "}"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import json\n",
        "function_name = function_call_data['name']\n",
        "function_args = function_call_data['arguments']\n",
        "function_args = json.loads(function_args)\n",
        "print(function_name, function_args)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tYb96Mh0NhH9",
        "outputId": "13c4bb89-6f29-4b3b-afa7-302dcf2cdd5f"
      },
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "get_current_weather {'location': 'Boston, MA'}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Call the get_current_weather() function"
      ],
      "metadata": {
        "id": "z3tstH_yN3fX"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "if function_name == \"get_current_weather\":\n",
        "  result = get_current_weather(**function_args)\n",
        "  print(result)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TSb8JHhgN5Zc",
        "outputId": "ef140572-4020-4daf-ac8c-d5161be9aa5c"
      },
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "12F\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Send the response from get_current_weather back to the model to summarize"
      ],
      "metadata": {
        "id": "k4HGJE3NRmMI"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "messages = [\n",
        "    {\"role\": \"user\", \"content\": \"What is the weather like in Boston?\"},\n",
        "    {\"role\": \"assistant\", \"content\": None, \"function_call\": {\"name\": \"get_current_weather\", \"arguments\": \"{ \\\"location\\\": \\\"Boston, MA\\\"}\"}},\n",
        "    {\"role\": \"function\", \"name\": \"get_current_weather\", \"content\": result}\n",
        "]\n",
        "response = completion(model=\"gpt-3.5-turbo-0613\", messages=messages, functions=functions)\n",
        "print(response)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "a23cmEwiPaw7",
        "outputId": "43259b86-0c4c-4fcb-eab7-6e1a788b2f21"
      },
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{\n",
            "  \"id\": \"chatcmpl-7mXGN62u75WXp1Lgen4iSgNvA7hHT\",\n",
            "  \"object\": \"chat.completion\",\n",
            "  \"created\": 1691801963,\n",
            "  \"model\": \"gpt-3.5-turbo-0613\",\n",
            "  \"choices\": [\n",
            "    {\n",
            "      \"index\": 0,\n",
            "      \"message\": {\n",
            "        \"role\": \"assistant\",\n",
            "        \"content\": \"The current weather in Boston is 12 degrees Fahrenheit.\"\n",
            "      },\n",
            "      \"finish_reason\": \"stop\"\n",
            "    }\n",
            "  ],\n",
            "  \"usage\": {\n",
            "    \"prompt_tokens\": 109,\n",
            "    \"completion_tokens\": 12,\n",
            "    \"total_tokens\": 121\n",
            "  }\n",
            "}\n"
          ]
        }
      ]
    }
  ]
}