quickstarts/Function_calling_config.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tce3stUlHN0L"
      },
      "source": [
        "##### Copyright 2024 Google LLC."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "tuOe1ymfHZPu"
      },
      "outputs": [],
      "source": [
        "# @title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IDS9Xcj_8k-T"
      },
      "source": [
        "# Gemini API: Function calling config\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/google-gemini/cookbook/blob/main/quickstarts/Function_calling_config.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1e41a2ce62eb"
      },
      "source": [
        "Specifying a `function_calling_config` allows you to control how the Gemini API acts when `tools` have been specified. For example, you can choose to only allow free-text output (disabling function calling), force it to choose from a subset of the functions provided in `tools`, or let it act automatically.\n",
        "\n",
        "This guide assumes you are already familiar with function calling. For an introduction, check out the [docs](https://ai.google.dev/docs/function_calling)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "m4DhA4907Asz"
      },
      "outputs": [],
      "source": [
        "!pip install -U -q google-generativeai"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aU-mY9hi8pQh"
      },
      "source": [
        "To run the following cell, your API key must be stored it in a Colab Secret named `GOOGLE_API_KEY`. If you don't already have an API key, or you're not sure how to create a Colab Secret, see the [Authentication](https://github.com/google-gemini/gemini-api-cookbook/blob/main/quickstarts/Authentication.ipynb) quickstart for an example."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wp3W4Pdf8rBO"
      },
      "outputs": [],
      "source": [
        "from google.colab import userdata\n",
        "import google.generativeai as genai\n",
        "\n",
        "genai.configure(api_key=userdata.get(\"GOOGLE_API_KEY\"))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iJqil-VL8ug-"
      },
      "source": [
        "## Set up a model with tools\n",
        "\n",
        "This example uses 3 functions that control a simple hypothetical lighting system. Using these functions requires them to be called in a specific order. For example, you must turn the light system on before you can change color.\n",
        "\n",
        "While you can pass these directly to the model and let it try to call them correctly, specifying the `function_calling_config` gives you precise control over the functions that are available to the model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "gLS26n7A9l9B"
      },
      "outputs": [],
      "source": [
        "def enable_lights():\n",
        "    \"\"\"Turn on the lighting system.\"\"\"\n",
        "    print(\"LIGHTBOT: Lights enabled.\")\n",
        "\n",
        "\n",
        "def set_light_color(rgb_hex: str):\n",
        "    \"\"\"Set the light color. Lights must be enabled for this to work.\"\"\"\n",
        "    print(f\"LIGHTBOT: Lights set to {rgb_hex}.\")\n",
        "\n",
        "\n",
        "def stop_lights():\n",
        "    \"\"\"Stop flashing lights.\"\"\"\n",
        "    print(\"LIGHTBOT: Lights turned off.\")\n",
        "\n",
        "\n",
        "light_controls = [enable_lights, set_light_color, stop_lights]\n",
        "instruction = \"You are a helpful lighting system bot. You can turn lights on and off, and you can set the color. Do not perform any other tasks.\"\n",
        "\n",
        "model = genai.GenerativeModel(\n",
        "    \"models/gemini-1.5-pro\", tools=light_controls, system_instruction=instruction\n",
        ")\n",
        "\n",
        "chat = model.start_chat()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JqROCznFCj_Y"
      },
      "source": [
        "Create a helper function for setting `function_calling_config` on `tool_config`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "_QgLFPL4Chon"
      },
      "outputs": [],
      "source": [
        "from google.generativeai.types import content_types\n",
        "from collections.abc import Iterable\n",
        "\n",
        "\n",
        "def tool_config_from_mode(mode: str, fns: Iterable[str] = ()):\n",
        "    \"\"\"Create a tool config with the specified function calling mode.\"\"\"\n",
        "    return content_types.to_tool_config(\n",
        "        {\"function_calling_config\": {\"mode\": mode, \"allowed_function_names\": fns}}\n",
        "    )"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ofMEuh_MFdMf"
      },
      "source": [
        "## Text-only mode: `NONE`\n",
        "\n",
        "If you have provided the model with tools, but do not want to use those tools for the current conversational turn, then specify `NONE` as the mode. `NONE` tells the model not to make any function calls, and will behave as though none have been provided."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "6ZlIFwXqGA09"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "I can turn lights on and off, and I can set the color of the lights.  What can I do for you? \n",
            "\n"
          ]
        }
      ],
      "source": [
        "tool_config = tool_config_from_mode(\"none\")\n",
        "\n",
        "response = chat.send_message(\n",
        "    \"Hello light-bot, what can you do?\", tool_config=tool_config\n",
        ")\n",
        "print(response.text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uux063sjHZ_Z"
      },
      "source": [
        "## Automatic mode: `AUTO`\n",
        "\n",
        "To allow the model to decide whether to respond in text or call specific functions, you can specify `AUTO` as the mode."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "vwO9dUjvHoT8"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "function_call {\n",
            "  name: \"enable_lights\"\n",
            "  args {\n",
            "  }\n",
            "}\n",
            "\n"
          ]
        }
      ],
      "source": [
        "tool_config = tool_config_from_mode(\"auto\")\n",
        "\n",
        "response = chat.send_message(\"Light this place up!\", tool_config=tool_config)\n",
        "print(response.parts[0])\n",
        "chat.rewind();  # You are not actually calling the function, so remove this from the history."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oHhaO-P9CBPb"
      },
      "source": [
        "## Function-calling mode: `ANY`\n",
        "\n",
        "Setting the mode to `ANY` will force the model to make a function call. By setting `allowed_function_names`, the model will only choose from those functions. If it is not set, all of the functions in `tools` are candidates for function calling.\n",
        "\n",
        "In this example system, if the lights are already on, then the user can change color or turn the lights off."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "GQpz94zrCNJF"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "function_call {\n",
            "  name: \"set_light_color\"\n",
            "  args {\n",
            "    fields {\n",
            "      key: \"rgb_hex\"\n",
            "      value {\n",
            "        string_value: \"800080\"\n",
            "      }\n",
            "    }\n",
            "  }\n",
            "}\n",
            "\n"
          ]
        }
      ],
      "source": [
        "available_fns = [\"set_light_color\", \"stop_lights\"]\n",
        "\n",
        "tool_config = tool_config_from_mode(\"any\", available_fns)\n",
        "\n",
        "response = chat.send_message(\"Make this place PURPLE!\", tool_config=tool_config)\n",
        "print(response.parts[0])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8cGrRy-uJ7-J"
      },
      "source": [
        "## Automatic function calling\n",
        "\n",
        "`tool_config` works when enabling automatic function calling too."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "hx7aIX8OXvi6"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "LIGHTBOT: Lights enabled.\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "response:\n",
              "GenerateContentResponse(\n",
              "    done=True,\n",
              "    iterator=None,\n",
              "    result=glm.GenerateContentResponse({\n",
              "      \"candidates\": [\n",
              "        {\n",
              "          \"content\": {\n",
              "            \"parts\": [\n",
              "              {\n",
              "                \"text\": \"Let there be light! \\ud83d\\udca1 \\n\"\n",
              "              }\n",
              "            ],\n",
              "            \"role\": \"model\"\n",
              "          },\n",
              "          \"finish_reason\": 1,\n",
              "          \"index\": 0,\n",
              "          \"safety_ratings\": [\n",
              "            {\n",
              "              \"category\": 9,\n",
              "              \"probability\": 1,\n",
              "              \"blocked\": false\n",
              "            },\n",
              "            {\n",
              "              \"category\": 8,\n",
              "              \"probability\": 1,\n",
              "              \"blocked\": false\n",
              "            },\n",
              "            {\n",
              "              \"category\": 7,\n",
              "              \"probability\": 1,\n",
              "              \"blocked\": false\n",
              "            },\n",
              "            {\n",
              "              \"category\": 10,\n",
              "              \"probability\": 1,\n",
              "              \"blocked\": false\n",
              "            }\n",
              "          ],\n",
              "          \"token_count\": 0,\n",
              "          \"grounding_attributions\": []\n",
              "        }\n",
              "      ]\n",
              "    }),\n",
              ")"
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "available_fns = [\"enable_lights\"]\n",
        "tool_config = tool_config_from_mode(\"any\", available_fns)\n",
        "\n",
        "auto_chat = model.start_chat(enable_automatic_function_calling=True)\n",
        "auto_chat.send_message(\"It's awful dark in here...\", tool_config=tool_config)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kz8McBZfXg0N"
      },
      "source": [
        "## Further reading\n",
        "\n",
        "Check out the function calling [quickstart](https://github.com/google-gemini/cookbook/blob/main/quickstarts/Function_calling.ipynb) for an introduction to function calling. You can find another fun function calling example [here](https://github.com/google-gemini/cookbook/blob/main/quickstarts/rest/Function_calling_REST.ipynb) using curl.\n"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "Function_calling_config.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}