-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Open
Labels
eval[Component] This issue is related to evaluation[Component] This issue is related to evaluation
Description
Describe the bug
When evaluating text in other languages (e.g., Thai, etc.), the eval logic incorrectly returns mismatches (Match score: 0)— even when the evaluated expression should be valid or identical in meaning.
To Reproduce
root_agent = Agent(
name="sample_agent",
model="gemini-2.5-flash",
description="Greeting agent",
instruction="""
You are an assistant that will reply to the user with only the word 'สวัสดี'
""",
)
Expected behavior
Eval should correctly handle input in other languages.
Screenshots
Desktop (please complete the following information):
- OS: macOS
- Python : 3.12.8
- ADK version: 1.15.1
Model Information:
- Are you using LiteLLM: No
- Which model is being used : gemini-2.5-flash
Additional context
Add any other context about the problem here.
Metadata
Metadata
Assignees
Labels
eval[Component] This issue is related to evaluation[Component] This issue is related to evaluation