Releases: berkayildi/mcp-llm-eval
Releases · berkayildi/mcp-llm-eval
v0.9.2
Compare
Sorry, something went wrong.
No results found
0.9.2 (2026-05-03)
Documentation
remove references to private internal consumer (65f7936 )
v0.9.1
Compare
Sorry, something went wrong.
No results found
0.9.1 (2026-04-29)
Documentation
readme: replace static version pins with PyPI badges, update model counts (fe1f716 )
v0.9.0
Compare
Sorry, something went wrong.
No results found
0.9.0 (2026-04-28)
Features
eval: expand reasoning category to 10 questions for statistical separation (ea4d8bb )
v0.8.0
Compare
Sorry, something went wrong.
No results found
0.8.0 (2026-04-28)
Features
eval: expand benchmark matrix with frontier models (c63d46e )
Bug Fixes
providers: use max_completion_tokens for OpenAI to support gpt-5.x reasoning models (c63d46e )
v0.7.1
Compare
Sorry, something went wrong.
No results found
0.7.1 (2026-04-26)
Documentation
readme: bump status to v0.7.0, add role in ecosystem section (ca06097 )
v0.7.0
Compare
Sorry, something went wrong.
No results found
0.7.0 (2026-04-26)
Features
retrieval: add OpenAI and Google embedding adapters (799ea2a )
Bug Fixes
cli: handle RAG result schema in threshold check (799ea2a )
rag: split RAG summary and benchmark output files (799ea2a )
v0.6.0
Compare
Sorry, something went wrong.
No results found
0.6.0 (2026-04-26)
Features
eval: add retrieval/RAG benchmark with AWS docs corpus (42badb0 )
v0.5.1
Compare
Sorry, something went wrong.
No results found
0.5.1 (2026-04-25)
Documentation
real benchmark numbers and status line in README (3816a97 )
update README for v0.5.0 retrieval and RAG evaluation (401fbca )
v0.5.0
Compare
Sorry, something went wrong.
No results found
0.5.0 (2026-04-25)
Features
cli: add evaluate-retrieval and evaluate-rag subcommands (f6133ba )
config: extend ThresholdConfig with retrieval and RAG thresholds (725c56e )
engine: add run_rag_evaluation orchestrator and retrieval drift check (202a741 )
judge: add context_relevance and citation_faithfulness judges (8b5074b )
metrics: add retrieval_metrics module with recall, precision, MRR, nDCG (91ac8d0 )
retrieval: add RetrievalAdapter protocol and BM25 in-memory adapter (7109118 )
server: add evaluate_retrieval, evaluate_rag, drift, and poisoned-corpus MCP tools (c8a1ab1 )
types: add retrieval and RAG result types, extend EvalEntry (650fbc0 )
Documentation
design: add v0.5.0 retrieval eval design spec (5bf7e04 )
v0.4.1
Compare
Sorry, something went wrong.
No results found
0.4.1 (2026-04-19)
Documentation
add benchmark workflow and CI flow diagram (e1e0900 )
update readme (46f8de5 )