diff --git a/quickstarts/function_calling_python.ipynb b/quickstarts/function_calling_python.ipynb new file mode 100644 index 000000000..b6f961985 --- /dev/null +++ b/quickstarts/function_calling_python.ipynb @@ -0,0 +1,420 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "2edc81e382cf" + }, + "source": [ + "##### Copyright 2024 Google LLC." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "cellView": "form", + "id": "906e07f6e562" + }, + "outputs": [], + "source": [ + "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", + "# you may not use this file except in compliance with the License.\n", + "# You may obtain a copy of the License at\n", + "#\n", + "# https://www.apache.org/licenses/LICENSE-2.0\n", + "#\n", + "# Unless required by applicable law or agreed to in writing, software\n", + "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", + "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", + "# See the License for the specific language governing permissions and\n", + "# limitations under the License." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "yeadDkMiISin" + }, + "source": [ + "# Gemini API: Function calling with Python" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "lEXQ3OwKIa-O" + }, + "source": [ + "\n", + " \n", + " \n", + "
\n", + " Run in Google Colab\n", + " \n", + " View source on GitHub\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "df1767a3d1cc" + }, + "source": [ + "You can provide Gemini models with descriptions of functions. The model may ask you to call a function and send it back the result to help the model handle your query." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "FFPBKLapSCkM" + }, + "source": [ + "## Setup\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "wFNV1e3ASJha" + }, + "source": [ + "### Install the Python SDK\n", + "\n", + "The Python SDK for the Gemini API is contained in the [`google-generativeai`](https://pypi.org/project/google-generativeai/) package. Install the dependency using pip:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "id": "9OEoeosRTv-5" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[33mWARNING: There was an error checking the latest version of pip.\u001b[0m\u001b[33m\n", + "\u001b[0m" + ] + } + ], + "source": [ + "!pip install -U -q google-generativeai" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "KCFF5VSTbcAR" + }, + "source": [ + "### Import packages" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "vRC2HngneEeQ" + }, + "source": [ + "Import the necessary packages." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "id": "TS9l5igubpHO" + }, + "outputs": [], + "source": [ + "import google.generativeai as genai" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "gHYFrFPjSGNq" + }, + "source": [ + "### Set up your API key\n", + "\n", + "Before you can use the Gemini API, you must first obtain an API key. If you don't already have one, create a key with one click in Google AI Studio.\n", + "\n", + "Get an API key\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "tHhsUxDTdw0W" + }, + "source": [ + "In Colab, add the key to the secrets manager under the \"🔑\" in the left panel. Give it the name `API_KEY`." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "VmSlTHXxb5pV" + }, + "source": [ + "Once you have the API key, pass it to the SDK. You can do this in two ways:\n", + "\n", + "* Put the key in the `GOOGLE_API_KEY` environment variable (the SDK will automatically pick it up from there).\n", + "* Pass the key to `genai.configure(api_key=...)`\n" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "id": "ab9ASynfcIZn" + }, + "outputs": [], + "source": [ + "import os\n", + "try:\n", + " # Used to securely store your API key\n", + " from google.colab import userdata\n", + " GOOGLE_API_KEY=userdata.get('GOOGLE_API_KEY')\n", + " genai.configure(api_key=GOOGLE_API_KEY)\n", + "except ImportError:\n", + " pass \n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "3f383614ec30" + }, + "source": [ + "## Function Basics" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "b82c1aecb657" + }, + "source": [ + "You can pass a list of functions to the `tools` argument when creating a `genai.GenerativeModel`.\n", + "\n", + "> Important: The SDK converts the function's argument's type annotations to a format the API understands (`glm.FunctionDeclaration`). The API only supports a limited selection of argument types, and this automatic conversion only supports a subset of that: `AllowedTypes = int | float | bool | str | list['AllowedTypes'] | dict`" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "id": "42b27b02d2f5" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "genai.GenerativeModel(\n", + " model_name='models/gemini-1.0-pro',\n", + " generation_config={},\n", + " safety_settings={},\n", + " tools=,\n", + ")" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "def multiply(a:float, b:float):\n", + " \"\"\"returns a * b.\"\"\"\n", + " return a*b\n", + "\n", + "model = genai.GenerativeModel(model_name='gemini-1.0-pro',\n", + " tools=[multiply])\n", + "\n", + "model" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "d5fd91032a1e" + }, + "source": [ + "The recomended way to use function calling is through the chat interface. The main reason is that `FunctionCalls` fit nicely into chat's multi-turn structure." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "id": "d3b91c855257" + }, + "outputs": [], + "source": [ + "chat = model.start_chat(enable_automatic_function_calling=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "1481a6159399" + }, + "source": [ + "With automatic function calling enabled `chat.send_message` automatically calls your function if the model asks it to.\n", + "\n", + "It appears to simply return a text response, containing the correct answer:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "id": "81d8def3d865" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "'So that would be 2508 mittens in total.'" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "response = chat.send_message('I have 57 cats, each owns 44 mittens, how many mittens is that in total?')\n", + "response.text" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "id": "951c0f83f72e" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "2508" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "57*44" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "7731e35f2383" + }, + "source": [ + "If you look in the `ChatSession.history` you can the sequence of `glm.Content` objects representing the turns of the conversation.\n", + "Each `glm.Content` contains a list of `glm.Part` objects that either contain text, a `glm.FunctionCall` or a `glm.FunctionResponse`.\n", + "\n", + "Here the sequence of turns is:\n", + "\n", + "1. You sent the question.\n", + "2. The model replied with a `glm.FunctionCall`.\n", + "3. The `genai.ChatSession` executed the function locally and sent the model back a `glm.FunctionResponse`.\n", + "4. The model used the function output in its answer." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "id": "9f7eff1e8e60" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "user -> [{'text': 'I have 57 cats, each owns 44 mittens, how many mittens is that in total?'}]\n", + "--------------------------------------------------------------------------------\n", + "model -> [{'function_call': {'name': 'multiply', 'args': {'a': 57.0, 'b': 44.0}}}]\n", + "--------------------------------------------------------------------------------\n", + "user -> [{'function_response': {'name': 'multiply', 'response': {'result': 2508.0}}}]\n", + "--------------------------------------------------------------------------------\n", + "model -> [{'text': 'So that would be 2508 mittens in total.'}]\n", + "--------------------------------------------------------------------------------\n" + ] + } + ], + "source": [ + "for content in chat.history:\n", + " print(content.role, \"->\", [type(part).to_dict(part) for part in content.parts])\n", + " print('-'*80)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "2471fd72f05e" + }, + "source": [ + "In general the state diagram is:\n", + "\n", + "\"The" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "f42d69800cff" + }, + "source": [ + "The model can respond with multiple function calls before returning a text response, and function calls come before the text response." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "9610f3465a69" + }, + "source": [ + "While this was all handled automatically, if you need more control, you can:\n", + "\n", + "- Leave the default `enable_automatic_function_calling=False` and process the `glm.FunctionCall` responses yourself.\n", + "- Or use `GenerativeModel.generate_content`, where you also need to manage the chat history. " + ] + } + ], + "metadata": { + "colab": { + "name": "function_calling_python.ipynb", + "toc_visible": true + }, + "google": { + "image_path": "/site-assets/images/share.png", + "keywords": [ + "examples", + "googleai", + "samplecode", + "python", + "embed", + "function" + ] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/quickstarts/safety_python.ipynb b/quickstarts/safety_python.ipynb new file mode 100644 index 000000000..1c7c4a12c --- /dev/null +++ b/quickstarts/safety_python.ipynb @@ -0,0 +1,570 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "Tce3stUlHN0L" + }, + "source": [ + "##### Copyright 2024 Google LLC." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "cellView": "form", + "id": "tuOe1ymfHZPu" + }, + "outputs": [], + "source": [ + "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", + "# you may not use this file except in compliance with the License.\n", + "# You may obtain a copy of the License at\n", + "#\n", + "# https://www.apache.org/licenses/LICENSE-2.0\n", + "#\n", + "# Unless required by applicable law or agreed to in writing, software\n", + "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", + "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", + "# See the License for the specific language governing permissions and\n", + "# limitations under the License." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "yeadDkMiISin" + }, + "source": [ + "# Gemini API: Safety Quickstart" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "lEXQ3OwKIa-O" + }, + "source": [ + "\n", + " \n", + " \n", + "
\n", + " Run in Google Colab\n", + " \n", + " View source on GitHub\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "uOxMUKTxR-_j" + }, + "source": [ + "The Gemini API is careful not to return inapropriate responses. What's apropriate can change depending on your use case or audience, so the safety settings are configurable.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "H9__zr1nSBpE" + }, + "source": [ + "## Prerequisites\n", + "\n", + "You can run this quickstart in [Google Colab](https://colab.research.google.com/github/google/generative-ai-docs/blob/main/site/en/tutorials/python_quickstart.ipynb), which runs this notebook directly in the browser and does not require additional environment configuration.\n", + "\n", + "Alternatively, to complete this quickstart locally, ensure that your development environment meets the following requirements:\n", + "\n", + "- Python 3.9+\n", + "- An installation of `jupyter` to run the notebook." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "FFPBKLapSCkM" + }, + "source": [ + "## Setup" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "wFNV1e3ASJha" + }, + "source": [ + "### Install the Python SDK\n", + "\n", + "The Python SDK for the Gemini API, is contained in the [`google-generativeai`](https://pypi.org/project/google-generativeai/) package. Install the dependency using pip:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "id": "9OEoeosRTv-5" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[33mWARNING: There was an error checking the latest version of pip.\u001b[0m\u001b[33m\n", + "\u001b[0m" + ] + } + ], + "source": [ + "!pip install -q -U google-generativeai" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "KCFF5VSTbcAR" + }, + "source": [ + "### Import packages" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "vRC2HngneEeQ" + }, + "source": [ + "Import the necessary packages." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "id": "TS9l5igubpHO" + }, + "outputs": [], + "source": [ + "import google.generativeai as genai\n", + "\n", + "import google.ai.generativelanguage as glm" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "gHYFrFPjSGNq" + }, + "source": [ + "### Setup your API key\n", + "\n", + "Before you can use the Gemini API, you must first obtain an API key. If you don't already have one, create a key with one click in Google AI Studio.\n", + "\n", + "Get an API key" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "tHhsUxDTdw0W" + }, + "source": [ + "In Colab, add the key to the secrets manager under the \"🔑\" in the left panel. Give it the name `GOOGLE_API_KEY`." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "VmSlTHXxb5pV" + }, + "source": [ + "Once you have the API key, pass it to the SDK. You can do this in two ways:\n", + "\n", + "* Put the key in the `GOOGLE_API_KEY` environment variable (the SDK will automatically pick it up from there).\n", + "* Pass the key to `genai.configure(api_key=...)`" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "id": "ab9ASynfcIZn" + }, + "outputs": [], + "source": [ + "import os\n", + "try:\n", + " from google.colab import userdata\n", + " GOOGLE_API_KEY=userdata.get('GOOGLE_API_KEY')\n", + " genai.configure(api_key=GOOGLE_API_KEY) \n", + "except ImportError:\n", + " pass\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "LZfoK3I3hu6V" + }, + "source": [ + "## Prompt Feedback\n", + "\n", + "The result returned by the `Model.generate_content` method is a `genai.GenerateContentResponse` (a wrapper around a `glm.GenerateContentResponse` object)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "2bcfnGEviwTI" + }, + "outputs": [], + "source": [ + "model = genai.GenerativeModel('gemini-1.0-pro')\n", + "\n", + "bad_prompt = # Put your bad prompt here\n", + "response = model.generate_content(bad_prompt)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "WR_2A_sxk8sK" + }, + "source": [ + "This response object gives you safety feedback in two ways, first the `prompt_feedback` (a `glm.PromptFeedback`) attribute contains a list of `glm.SafetyRatings` for the input prompt, and an optional `block_reason` (a `glm.PromptFeedback.BlockReason`): " + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": { + "id": "8887de812dc0" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 21, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "bool(response.prompt_feedback.block_reason)" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": { + "id": "he-OfzBbhACQ" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "block_reason: SAFETY\n", + "safety_ratings {\n", + " category: HARM_CATEGORY_SEXUALLY_EXPLICIT\n", + " probability: NEGLIGIBLE\n", + "}\n", + "safety_ratings {\n", + " category: HARM_CATEGORY_HATE_SPEECH\n", + " probability: HIGH\n", + "}\n", + "safety_ratings {\n", + " category: HARM_CATEGORY_HARASSMENT\n", + " probability: HIGH\n", + "}\n", + "safety_ratings {\n", + " category: HARM_CATEGORY_DANGEROUS_CONTENT\n", + " probability: NEGLIGIBLE\n", + "}" + ] + }, + "execution_count": 22, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "response.prompt_feedback" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "72b4a8808bb9" + }, + "source": [ + "If the prompt is blocked because of the safety ratings, you will not get any candidates in the response:" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": { + "id": "f20d9269325d" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "[]" + ] + }, + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "response.candidates" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4672af98ac57" + }, + "source": [ + "### Safety settings" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "2a6229f6d3a1" + }, + "source": [ + "Adjust the safety settings and the prompt is no longer blocked:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "338fb9a6af78" + }, + "outputs": [], + "source": [ + "response = model.generate_content(\n", + " bad_prompt,\n", + " safety_settings={\n", + " 'HATE': 'BLOCK_NONE',\n", + " 'HARASSMENT': 'BLOCK_NONE',\n", + " })" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "0ffb6ec2c3bd" + }, + "source": [ + "While the SDK makes it simple and flexible to specify the safety settings (above), API actually expects a list of `glm.SafetySetting` objects (or their JSON representations). The ocode below is an equivalent to the code above:" + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": { + "id": "914aaf8d224d" + }, + "outputs": [], + "source": [ + "response = model.generate_content(\n", + " bad_prompt,\n", + " safety_settings=[\n", + " glm.SafetySetting(category=glm.HarmCategory.HARM_CATEGORY_HATE_SPEECH,\n", + " threshold=glm.SafetySetting.HarmBlockThreshold.BLOCK_NONE),\n", + " glm.SafetySetting(category=glm.HarmCategory.HARM_CATEGORY_HATE_SPEECH,\n", + " threshold=glm.SafetySetting.HarmBlockThreshold.BLOCK_NONE)])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "86c560e0a641" + }, + "source": [ + "With the new settings, the `prompt_feedback` doesn't changed, except that the `blocked_reason` is no longer set." + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": { + "id": "0c2847c49262" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "False" + ] + }, + "execution_count": 26, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "bool(response.prompt_feedback.block_reason)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "47298a4eef40" + }, + "source": [ + "And a candidate response is returned." + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": { + "id": "028febe8df68" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "1" + ] + }, + "execution_count": 27, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "len(response.candidates)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "3d401c247957" + }, + "source": [ + "### Candidate ratings" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "3d306960dffb" + }, + "source": [ + "For a prompt that is not blocked, the response object contains a list of `glm.Candidate` objects (just 1 for now). Each candidate includes a `finish_reason` (from `glm.Candidate.FinishReason`): " + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": { + "id": "e49b53f69a2c" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 28, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "candidate = response.candidates[0]\n", + "candidate.finish_reason" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "badddf10089b" + }, + "source": [ + "`FinishReason.STOP` means that the model finished it's output normally. If a candidate's finish reason is `SAFETY` it's becauase the candidate's `safety_ratings` exceeded the the request's `safety_settings` threshold. This response was rated `NEGLIGIBLE` harm probability: " + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": { + "id": "2b60d9f96af0" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "[category: HARM_CATEGORY_SEXUALLY_EXPLICIT\n", + "probability: NEGLIGIBLE\n", + ", category: HARM_CATEGORY_HATE_SPEECH\n", + "probability: NEGLIGIBLE\n", + ", category: HARM_CATEGORY_HARASSMENT\n", + "probability: NEGLIGIBLE\n", + ", category: HARM_CATEGORY_DANGEROUS_CONTENT\n", + "probability: NEGLIGIBLE\n", + "]" + ] + }, + "execution_count": 29, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "candidate.safety_ratings" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "2e006804a445" + }, + "source": [ + "The `safety_settings` " + ] + } + ], + "metadata": { + "colab": { + "name": "safety_python.ipynb", + "toc_visible": true + }, + "google": { + "image_path": "/static/site-assets/images/docs/logo-python.svg", + "keywords": [ + "examples", + "gemini", + "beginner", + "googleai", + "quickstart", + "python", + "text", + "chat", + "vision", + "embed" + ] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +}