Skip to content

Releases: berkayildi/mcp-llm-eval

v0.9.2

Choose a tag to compare

@github-actions github-actions released this 03 May 21:53
3334376

0.9.2 (2026-05-03)

Documentation

  • remove references to private internal consumer (65f7936)

v0.9.1

Choose a tag to compare

@github-actions github-actions released this 29 Apr 08:19
29b2270

0.9.1 (2026-04-29)

Documentation

  • readme: replace static version pins with PyPI badges, update model counts (fe1f716)

v0.9.0

Choose a tag to compare

@github-actions github-actions released this 28 Apr 21:16
b3e5b0e

0.9.0 (2026-04-28)

Features

  • eval: expand reasoning category to 10 questions for statistical separation (ea4d8bb)

v0.8.0

Choose a tag to compare

@github-actions github-actions released this 28 Apr 15:17
3c9bd8c

0.8.0 (2026-04-28)

Features

  • eval: expand benchmark matrix with frontier models (c63d46e)

Bug Fixes

  • providers: use max_completion_tokens for OpenAI to support gpt-5.x reasoning models (c63d46e)

v0.7.1

Choose a tag to compare

@github-actions github-actions released this 26 Apr 21:20
024224f

0.7.1 (2026-04-26)

Documentation

  • readme: bump status to v0.7.0, add role in ecosystem section (ca06097)

v0.7.0

Choose a tag to compare

@github-actions github-actions released this 26 Apr 20:48
fb0f632

0.7.0 (2026-04-26)

Features

  • retrieval: add OpenAI and Google embedding adapters (799ea2a)

Bug Fixes

  • cli: handle RAG result schema in threshold check (799ea2a)
  • rag: split RAG summary and benchmark output files (799ea2a)

v0.6.0

Choose a tag to compare

@github-actions github-actions released this 26 Apr 10:38
9f637e4

0.6.0 (2026-04-26)

Features

  • eval: add retrieval/RAG benchmark with AWS docs corpus (42badb0)

v0.5.1

Choose a tag to compare

@github-actions github-actions released this 25 Apr 15:28
917e337

0.5.1 (2026-04-25)

Documentation

  • real benchmark numbers and status line in README (3816a97)
  • update README for v0.5.0 retrieval and RAG evaluation (401fbca)

v0.5.0

Choose a tag to compare

@github-actions github-actions released this 25 Apr 15:04
30508c8

0.5.0 (2026-04-25)

Features

  • cli: add evaluate-retrieval and evaluate-rag subcommands (f6133ba)
  • config: extend ThresholdConfig with retrieval and RAG thresholds (725c56e)
  • engine: add run_rag_evaluation orchestrator and retrieval drift check (202a741)
  • judge: add context_relevance and citation_faithfulness judges (8b5074b)
  • metrics: add retrieval_metrics module with recall, precision, MRR, nDCG (91ac8d0)
  • retrieval: add RetrievalAdapter protocol and BM25 in-memory adapter (7109118)
  • server: add evaluate_retrieval, evaluate_rag, drift, and poisoned-corpus MCP tools (c8a1ab1)
  • types: add retrieval and RAG result types, extend EvalEntry (650fbc0)

Documentation

  • design: add v0.5.0 retrieval eval design spec (5bf7e04)

v0.4.1

Choose a tag to compare

@github-actions github-actions released this 19 Apr 19:24
fb86ef8

0.4.1 (2026-04-19)

Documentation

  • add benchmark workflow and CI flow diagram (e1e0900)
  • update readme (46f8de5)