0.0.7

Features

• Add pytest-adk eval CLI for remote agent evaluation by @ftnext in #9

$ pytest-adk eval http://localhost:8000 tests/evals/

• Always create temporary eval sessions with server-assigned ids by @ftnext in #16
• Register EvalConfig custom metrics before evaluation runs by @ftnext in #13
  • Scope fixture-path custom metrics to their own evalset's evaluation in #14

Full Changelog: 0.0.6...0.0.7

0.0.6

Features

• Add optional Jinja prompt-template engine in #7

Full Changelog: 0.0.5...0.0.6

0.0.5

Features

• Provide AgentEvaluator as a pytest fixture in #5

Improvements

• Support .toml as well as .test.toml (Route evalset discovery by directory vs file path) in #4
• Report eval results dir in terminal summary in #6
• Add docs (README, docstrings, type hints)

Full Changelog: 0.0.4...0.0.5

0.0.4

Features

• Add prompt template expansion for evalset text fields in #2

text = "<prompt:FILENAME [KEY=VALUE ...]>"

Full Changelog: 0.0.3...0.0.4

0.0.3

Features

• Evalset TOML support: *.test.toml
• Generate empty evalset (as TOML or JSON) by pytest-adk-eval-schema

Full Changelog: 0.0.2...0.0.3

0.0.2

Features

• feat: Add patched AgentEvaluator
  • Use it like Google ADK's AgentEvaluator, with an extra results_dir argument
  • Saves eval result JSON files under results_dir/test_app/.adk/eval_history/
  • This is a pytest-adk version of the eval result persistence proposed upstream in google/adk-python#4414

Improvement

• docs: Loaded session example: fields_planner

Full Changelog: 0.0.1...0.0.2

0.0.1

Starting the project!🎉

Features

• Add load_session_from_json and runner_from_exported_session
  • Use with exported session JSON

Full Changelog: https://github.com/ftnext/pytest-adk/commits/0.0.1