Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[gemini-function-calling]: Unexpected response from given prompt #1375

Open
xprilion opened this issue Mar 15, 2024 · 1 comment
Open

[gemini-function-calling]: Unexpected response from given prompt #1375

xprilion opened this issue Mar 15, 2024 · 1 comment

Comments

@xprilion
Copy link

The prompt provided in Step 8. Generate a function call might at times provide responses which ask for when the exchange was made. For instance:

Screenshot 2024-03-16 at 12 25 46 AM

^ Using "gemini-1.0-pro-001"

The better way to frame this prompt would be:

prompt = """What is the latest exchange rate from Australian dollars to Swedish krona?
How much is 500 Australian dollars worth in Swedish krona?"""

this accurately invokes the expected function calling.

@koverholt
Copy link

koverholt commented May 24, 2024

Hi @xprilion, apologies that we missed this feedback. And thank you for reporting this behavior!

  • I intentionally crafted the prompt this way to exercise the following parameter description in the FunctionDeclaration, which should fall back to the value latest if a date is not specified:
            "currency_date": {
                "type": "string",
                "description": "A date that must always be in YYYY-MM-DD format or the value 'latest' if a time period is not specified"
            },
    
  • I've updated the codelab to use gemini-1.5-pro-001 (after testing it and validating that it is behaving as expected) and it's improved reasoning and Function Calling behavior. This should increase the level of determinism and improve the behavior here so that the model follows the instructions in the parameter description as intended.
  • Finally, I've updated the feedback link in the codelab to auto assign folks so we don't miss feedback like this in the future!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants