chore: add golden dataset for eval

[Yuan325]

Adding golden datasets that will be used in llm system evaluation.

The golden dataset is separated into multiple types of queries:

• queries that uses a specific tool
• airline related queries (no tool calling - answer is within prompt)
• assistant related question (no tool calling - answer is within prompt)
• out of context questions (no tool calling)
• multitool selections (agent selecting multiple tool before returning a final answer to the user)

[jackwotherspoon]

Just putting this here as a friendly suggestion, can take it or leave it 😄

Long-term open source maintainability can improve drastically by PRs having at a minimum a one or two sentence description (especially for feat PRs but even for all types). Just going over what you are adding and why.

The benefits:

• Helps your reviewer quickly understand the PR
• Helps future maintainers follow-along with PRs
• Allows you to more easily search for old PRs in the future
• Encourages open source external contributions as external users better understand the features and history

I recently have found this extremely beneficial in Cloud SQL repos (example)

Github also recommends this themselves

[kurtisvg]

Curious to the structure -- why is this list -> object {key: object{} }?

Wouldn't it be better to just use an object at the higher level with key -> object?

[Yuan325]

Each evaluation data object contains of "query, tool call, output etc.".

And since I am trying to categorize them into different categories (e.g. we'll provide 2 test data for 'Search Airport Tool' so that we can test out 2 different queries that trigger that tool. If we want to add more queries in the future we can just add it to the category's list.), that's why we have list of evaluation data based on the category (a.k.a. key).

[kurtisvg]

+1 to Jack's comment. We could do a better job on PR descriptions.