v6.0.0

What's Changed

• Added monitoring by @ethan-tonic in #167
• Bump tqdm from 4.66.2 to 4.66.4 by @dependabot in #165
• Bump ruff from 0.3.7 to 0.4.3 by @dependabot in #161
• fix: environ setting for azure openai by @gmuffiness in #157

New Contributors

• @gmuffiness made their first contribution in #157

Full Changelog: v5.0.0...v6.0.0

v5.0.0

Now supports Claude and Gemini for scorers

What's Changed

• Bump idna from 3.6 to 3.7 by @dependabot in #139
• Bump openai from 1.14.3 to 1.18.0 by @dependabot in #142
• Bump pydantic from 2.6.4 to 2.7.0 by @dependabot in #140
• Bump typing-extensions from 4.10.0 to 4.11.0 by @dependabot in #135
• Bump ruff from 0.3.4 to 0.3.7 by @dependabot in #141
• LiteLLM Integration by @Ryan-Rishi in #144
• Update README.md by @anuj-tonic in #156

New Contributors

• @anuj-tonic made their first contribution in #156

Full Changelog: v4.0.6...v5.0.0

v4.0.6

What's Changed

• Attempt to disable telemetry on workflows, again by @akamor in #134
• Fix run metadata llm evaluator check by @ethan-tonic in #137
• Fix max exponential backoff by @ethan-tonic in #138

Full Changelog: v4.0.5...v4.0.6

v4.0.5

What's Changed

• Add ChatBees vs OpenAI RAG benchmark to tonic validate examples by @ndb-rkang in #115
• Added field to store the llm used for the eval. by @Ryan-Rishi in #126
• Added parameters to expose LLM prompts from eval by @Ryan-Rishi in #128
• Disable tracking for github workflows by @akamor in #131
• Log runtime and sdk version in telemetry by @akamor in #133
• Track SDK backend in telemetry by @akamor in #132

New Contributors

• @ndb-rkang made their first contribution in #115
• @Ryan-Rishi made their first contribution in #126

Full Changelog: v4.0.4...v4.0.5

v4.0.4

What's Changed

• Swap off alru by @ethan-tonic in #120
• Fix Answer Contains PII by @ethan-tonic in #121
• Add sdk tests using pytest by @ethan-tonic in #122
• Move pytest into dev dependencies by @ethan-tonic in #124
• Added type validation by @ethan-tonic in #123

Full Changelog: v4.0.3...v4.0.4

v4.0.3

Important: This release fixes Validate not working on Python 3.8 and 3.9

What's Changed

• Add slack notification for package publish by @calexander3 in #111
• Add support for tags and loosen typing on Metadata by @akamor in #112
• Fix on Python 3.8 and 3.9 by @ethan-tonic in #119
• Bump openai from 1.14.2 to 1.14.3 by @dependabot in #117
• Bump ruff from 0.3.3 to 0.3.4 by @dependabot in #116
• Adding simple E2E test by @akamor in #118

New Contributors

• @calexander3 made their first contribution in #111

Full Changelog: v4.0.2...v4.0.3

v4.0.2

What's Changed

• Fix alru dependencies by @ethan-tonic in #110

Full Changelog: v4.0.1...v4.0.2

v4.0.1

What's Changed

• Fix TQDM progress bar counts by @ethan-tonic in #106
• Fix bullet point parsing by @ethan-tonic in #107
• V4.0.1 Bump by @ethan-tonic in #108

Full Changelog: v4.0.0...v4.0.1

v4.0.0

What's Changed

• Fix callback being called twice by @ethan-tonic in #99
• Create dependabot.yml by @ethan-tonic in #101
• Bump openai from 1.7.1 to 1.14.2 by @dependabot in #102
• Bump ruff from 0.1.15 to 0.3.3 by @dependabot in #103
• 🚀🚀 Add async to making evaluation 60 times faster by @ethan-tonic in #100
• Bump tiktoken from 0.5.2 to 0.6.0 by @dependabot in #104
• Bump sphinx-rtd-theme from 1.3.0 to 2.0.0 by @dependabot in #105

New Contributors

• @dependabot made their first contribution in #102

Full Changelog: v3.4.0...v4.0.0

v3.4.0

What's Changed

• feat: validate score range by @AzizCode92 in #78
• Add TQDM by @ethan-tonic in #75
• Improve retry logic by @ethan-tonic in #76
• Improve statement_derived_from_context_call by @ethan-tonic in #74
• Validate x LlamaIndex webinar by @ethan-tonic in #77
• Improve readme by @ethan-tonic in #83
• Fix LinkedIn URL by @ethan-tonic in #84
• Add new binary metrics by @ethan-tonic in #69
• Fix doc strings by @ethan-tonic in #88
• F/add sphinx by @lyon-tonic in #87
• feat: add hate speech binary metric by @AzizCode92 in #79
• Add LatencyMetric class by @ethan-tonic in #85
• Update readme to make metrics more clear by @ethan-tonic in #89
• Update README.md by @ethan-tonic in #92
• feat: expose prompts by @AzizCode92 in #90
• Added new metrics to scan for PII in RAG Answer and RAG Context. by @akamor in #94
• Update README.md by @ethan-tonic in #95
• Update README.md by @ethan-tonic in #96
• Speed up uploads by @ethan-tonic in #97
• Version bump to 3.4.0 by @ethan-tonic in #98

New Contributors

• @AzizCode92 made their first contribution in #78
• @lyon-tonic made their first contribution in #87

Full Changelog: v3.3.0...v3.4.0