Gemini function/tool calls not executing automatically

[KaiquanMah]

Hi Gemini team,

• What you're trying to do

1. Created 3 functions to be used for function/tool calls
2. Created function declarations for each function
3. Created Gemini chat config
4. Input a user query into the Gemini API
5. Gemini API plans the function/tool calls needed and extracts the arguments from a user query
6. Gemini calls the required functions/tools in parallel -> This was where the issue happened
7. Gemini generates a response at the end

• What code you've already tried

from google import genai
from google.genai import types

# both models have the same issue
GOOGLE_API_KEY = userdata.get('GOOGLE_API_KEY')
# model = "gemini-2.0-flash"
model = "models/gemini-2.5-flash-preview-04-17-thinking"
client = genai.Client(api_key=GOOGLE_API_KEY)



# Define pizza functions
def get_menu(vegan: bool = False) -> str:
    """Get the pizza menu, with optional filter for vegan options."""
    if vegan:
        return "Vegan menu: Veggie Delight (tomatoes, mushrooms, peppers), Garden Fresh (spinach, olives, onions)"
    else:
        return "Full menu: Pepperoni, Cheese, Meat Lovers, Veggie Delight, Garden Fresh, Hawaiian"

def order_pizza(toppings: Optional[List[str]] = None, 
                size: str = "medium") -> str:
    """Order a pizza with specified toppings and size."""
    toppings_str = ", ".join(toppings) if toppings else "no extra toppings"
    return f"Order confirmed! Your {size} pizza with {toppings_str} is being prepared."

def check_order_status(order_id: int) -> str:
    """Check the status of an existing pizza order."""
    statuses = {
        12345: "Your order is in the oven!",
        12346: "Your order is out for delivery.",
        12347: "Your order has been delivered.",
    }
    return statuses.get(order_id, "Order not found. Please check your order ID.")




# Define function schemas for the LLM
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_menu",
            "description": "Get the pizza menu, with optional filter for vegan options",
            "parameters": {
                "type": "object",
                "properties": {
                    "vegan": {
                        "type": "boolean",
                        "description": "Whether to show only vegan options"
                    }
                }
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "order_pizza",
            "description": "Order a pizza with specified toppings and size",
            "parameters": {
                "type": "object",
                "properties": {
                    "toppings": {
                        "type": "array",
                        "items": {"type": "string", 
                                  "enum": ["extra_cheese", "garlic", "onions"]},
                        "description": "List of toppings to add to the pizza"
                    },
                    "size": {
                        "type": "string", 
                        "enum": ["small", "medium", "large"],
                        "description": "Size of the pizza"
                    }
                },
                "required": ["size"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "check_order_status",
            "description": "Check the status of an existing pizza order",
            "parameters": {
                "type": "object",
                "properties": {
                    "order_id": {
                        "type": "integer",
                        "description": "The ID of the order to check"
                    }
                },
                "required": ["order_id"]
            }
        }
    }
]




def execute_function_call(function_call):
    print(function_call) # each function call from the list of function call(s)
    function_name = function_call.name

    if (len(function_call.args.items()) == 0) and (function_name == "get_menu"):
      arguments = {"vegan": False}
    else:
      # arguments = json.loads(function_call.args)
      # GOOGLE'S .args are already in dict format
      arguments = function_call.args

    
    if function_name == "order_pizza":
        return order_pizza(**arguments)
    elif function_name == "get_menu":
        return get_menu(**arguments)
    elif function_name == "check_order_status":
        return check_order_status(**arguments)
    else:
        return f"Function {function_name} not implemented"



# GOOGLE NEEDS JUST THE 'FUNCTION' part will do
# no need to say "type": "function"
get_menu_declaration = tools[0]["function"]
order_pizza_declaration = tools[1]["function"]
check_order_status_declaration = tools[2]["function"]

tools_google = [
    types.Tool(function_declarations=[get_menu_declaration, order_pizza_declaration, check_order_status_declaration])
]


# https://ai.google.dev/gemini-api/docs/function-calling?example=meeting#parallel_function_calling
tool_fn_names = [tool["function"]["name"] for tool in tools]

tool_config = types.ToolConfig(
    function_calling_config=types.FunctionCallingConfig(
        mode="ANY", allowed_function_names=tool_fn_names
    )
)


config = types.GenerateContentConfig(system_instruction = "You are a helpful assistant for a pizza delivery service.",
                                     temperature = 0.1,
                                     seed=38,
                                     tools = tools_google,
                                     tool_config = tool_config,
                                     max_output_tokens = 2048,
                                     # automatic_function_calling enabled by default
                                     automatic_function_calling=types.AutomaticFunctionCallingConfig(disable=False) # disable=True by Default
                                     )


query = "I'd like to order a large pizza with extra cheese and garlic, and please tell me your menu"
response1 = client.models.generate_content(
    model = model,
    config = config,
    contents = queries[0]
)

# response1 output
GenerateContentResponse(candidates=[Candidate(content=Content(parts=[Part(video_metadata=None, thought=None, inline_data=None, code_execution_result=None, executable_code=None, file_data=None, function_call=FunctionCall(id=None, args={'size': 'large', 'toppings': ['extra_cheese', 'garlic']}, name='order_pizza'), function_response=None, text=None), Part(video_metadata=None, thought=None, inline_data=None, code_execution_result=None, executable_code=None, file_data=None, function_call=FunctionCall(id=None, args={}, name='get_menu'), function_response=None, text=None)], role='model'), citation_metadata=None, finish_message=None, token_count=None, finish_reason=<FinishReason.STOP: 'STOP'>, avg_logprobs=None, grounding_metadata=None, index=0, logprobs_result=None, safety_ratings=None)], create_time=None, response_id=None, model_version='models/gemini-2.5-flash-preview-04-17', prompt_feedback=None, usage_metadata=GenerateContentResponseUsageMetadata(cache_tokens_details=None, cached_content_token_count=None, candidates_token_count=36, candidates_tokens_details=None, prompt_token_count=228, prompt_tokens_details=[ModalityTokenCount(modality=<MediaModality.TEXT: 'TEXT'>, token_count=228)], thoughts_token_count=334, tool_use_prompt_token_count=None, tool_use_prompt_tokens_details=None, total_token_count=598, traffic_type=None), automatic_function_calling_history=[], parsed=None)




# trying using another syntax for interacting with Gemini
chat = client.chats.create(model=model, config=config)
response2 = chat.send_message(queries[0])

# response2 output
GenerateContentResponse(candidates=[Candidate(content=Content(parts=[Part(video_metadata=None, thought=None, inline_data=None, code_execution_result=None, executable_code=None, file_data=None, function_call=FunctionCall(id=None, args={'toppings': ['extra_cheese', 'garlic'], 'size': 'large'}, name='order_pizza'), function_response=None, text=None), Part(video_metadata=None, thought=None, inline_data=None, code_execution_result=None, executable_code=None, file_data=None, function_call=FunctionCall(id=None, args={}, name='get_menu'), function_response=None, text=None)], role='model'), citation_metadata=None, finish_message=None, token_count=None, finish_reason=<FinishReason.STOP: 'STOP'>, avg_logprobs=-0.02231034423623766, grounding_metadata=None, index=None, logprobs_result=None, safety_ratings=None)], create_time=None, response_id=None, model_version='gemini-2.0-flash', prompt_feedback=None, usage_metadata=GenerateContentResponseUsageMetadata(cache_tokens_details=None, cached_content_token_count=None, candidates_token_count=14, candidates_tokens_details=[ModalityTokenCount(modality=<MediaModality.TEXT: 'TEXT'>, token_count=14)], prompt_token_count=123, prompt_tokens_details=[ModalityTokenCount(modality=<MediaModality.TEXT: 'TEXT'>, token_count=123)], thoughts_token_count=None, tool_use_prompt_token_count=None, tool_use_prompt_tokens_details=None, total_token_count=137, traffic_type=None), automatic_function_calling_history=[], parsed=None)

• Any error messages you're getting
  From response1 and response2 above,

1. function_call is captured from user query -> this is correct
2. .function_call.args is captured from user query -> this is correct
3. function_call.name is reasoned by the LLM -> this is correct
4. function_response=None -> why are the functions not executed and generating a response, unlike the Gemini documentation?

• Other readings already checked out

1. Tutorial I was following to understand Gemini's syntax: https://ai.google.dev/gemini-api/docs/function-calling?example=meeting#parallel_function_calling
2. Gemini not returning function_response https://stackoverflow.com/questions/78343977/gemini-api-not-giving-output-when-the-input-is-a-question-and-longer-then-5-word#:~:text=Just%20a%20note%20to%20someone%20who%20are%20facing,modify%20and%20set%20it%20to%202048%20or%20high.

PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.

[hkt74]

Thanks @KaiquanMah for asking this question!

It sounds like you're looking to use the automatic function calling feature You can achieve this by directly passing your callable functions to the tools configuration.

The SDK will then work with the model to get the necessary function call arguments, invoke the associated function for you, and provide the final result back in natural language.

Here's an example demonstrating this approach:

# Define pizza functions
def get_menu(vegan: bool = False) -> str:
    """Get the pizza menu, with optional filter for vegan options."""
    if vegan:
        return "Vegan menu: Veggie Delight (tomatoes, mushrooms, peppers), Garden Fresh (spinach, olives, onions)"
    else:
        return "Full menu: Pepperoni, Cheese, Meat Lovers, Veggie Delight, Garden Fresh, Hawaiian"

def order_pizza(toppings: List[str] = [],
                size: str = "medium") -> str:
    """Order a pizza with specified toppings and size."""
    toppings_str = ", ".join(toppings) if toppings else "no extra toppings"
    return f"Order confirmed! Your {size} pizza with {toppings_str} is being prepared."

def check_order_status(order_id: int) -> str:
    """Check the status of an existing pizza order."""
    statuses = {
        12345: "Your order is in the oven!",
        12346: "Your order is out for delivery.",
        12347: "Your order has been delivered.",
    }
    return statuses.get(order_id, "Order not found. Please check your order ID.")



config = types.GenerateContentConfig(
    system_instruction = "You are a helpful assistant for a pizza delivery service.",
    tools = [get_menu, order_pizza, check_order_status],
)

query = "I'd like to order a large pizza with extra cheese and garlic, and please tell me your menu"
response1 = client.models.generate_content(
    model = 'models/gemini-2.5-flash-preview-04-17',
    config = config,
    contents = query
)

print(response1.model_dump(exclude_none=True))

[mercuryyy]

@hkt74 have you seen - #843 (comment)

We are not able to run function calling from the API to

"gemini-2.5-flash-preview-native-audio-dialog"

where same code works perfectly fine for

"gemini-2.0-flash-live-001"

in "https://aistudio.google.com/" function calling was also not working for "gemini-2.5-flash-preview-native-audio-dialog" but now it seems to be working consistently and stable but the API is not working

[maxpastor]

@mercuryyy try enabling grounding with google search. For us it worked. It probably triggers their python env in the backend while the function calling doesn't (probably a bug on their end).

[td2thinh]

How do you do this with the API? @maxpastor

[maxpastor]

@td2thinh This is more or less what I do:
config: { systemInstruction: "this is my prompt", responseModalities: [Modality.AUDIO], tools: [{ googleSearch: {} }, { functionDeclarations: YOUR_FUNCTION_DEFINITIONS }], tool_choice: 'auto', temperature: 1.5}

Keep in mind the names of the functions have to be python friendly as google spawns them as python functions in their server env. Also the model doesn't seem to use functions as often as it should tbh, seems like a post training bias.
Let me know if it helps. For me I have 100% success rate with that technique

[saisatwik-ontune]

@maxpastor Will this work in livekit as well?

[maxpastor]

@maxpastor Will this work in livekit as well?

No clue, I'm not using Livekit, but I wouldn't see why not, everything is a Websocket request at the end of the day. I don't know how you might have to tweak their SDK though

[td2thinh]

@maxpastor Will this work in livekit as well?

No clue, I'm not using Livekit, but I wouldn't see why not, everything is a Websocket request at the end of the day. I don't know how you might have to tweak their SDK though

Are you using Vertex AI API or are you using Gemini API?

[maxpastor]

@maxpastor Will this work in livekit as well?

No clue, I'm not using Livekit, but I wouldn't see why not, everything is a Websocket request at the end of the day. I don't know how you might have to tweak their SDK though

Are you using Vertex AI API or are you using Gemini API?

I shared more info here: #843 (comment)

But Genai (gemini api) works for me and this workaround is for that. The Vertex API is a no-go. None of my tricks worked there.

[yyyu-google]

Hi thank you for the discussion, as @hkt74 explained in #859 (comment), just need to pass in the callable function as opposed to the function declaration in the tool list, and by default (if you don't explicitly set the values of automatic_function_calling field in the config parameter) automatic function calling will work. @hkt74 also cited the README section in this SDK for more details. Resolving.