TruLens Eval v0.28.0

What's Changed

• Meta-eval / feedback functions benchmarking notebooks, ranking-based eval utils, and docs update by @daniel-huang-1230 in #991
• App delete functionality added by @arn-tru in #1061
• Added test coverage to langchain provider by @arn-tru in #1062
• Configurable table prefix by @piotrm0 in #971
• Add example systemd service file by @piotrm0 in #1072

Bug fixes

• Queue fixed for python version lower than 3.9 by @arn-tru in #1066
• Fix test-tru by @piotrm0 in #1070
• Removed broken tests by @arn-tru in #1076
• Fix legacy db missing abstract method by @piotrm0 in #1077
• Release test fixes by @piotrm0 in #1078
• Docs fixes by @piotrm0 in #1075

Examples

• MongoDB Atlas quickstart by @joshreini1 in #1056
• OpenAI Assistants API (quickstart) by @joshreini1 in #1041

Full Changelog: trulens-eval-0.27.2...trulens-eval-0.28.0

trulens-eval-0.27.2

Bug Fix

• add missing pprint import by @joshreini1 in #1054

Full Changelog: trulens-eval-0.27.1...trulens-eval-0.27.2

trulens-eval-0.27.1

What's changed

• Add if_missing. by @piotrm0 in #1038
• Added feedback button to trulens by @arn-tru in #1046

Documentation updates

• pipelines readme by @piotrm0 in #1030
• docs | standards on proper names by @markdavidmc0 in #997
• docs glossary by @piotrm0 in #1029
• Fix TruLens docs link in hybrid retriever notebook by @daniel-huang-1230 in #1035
• docs README by @joshreini1 in #1034
• docs: fix typo by @nicoloboschi in #1036
• more pipelines docs by @piotrm0 in #1033
• Fix azure docs pipeline by @joshreini1 in #1037
• Docs updates for feedback, instrumentation apis, examples by @joshreini1 in #1032
• Proper names and glossary expansion in docs by @piotrm0 in #1042

Bug fixes

• Import improvements, fix version conflicts by @joshreini1 in #1047
• Fix import and favicon by @arn-tru in #1049
• remove pkg_resources and distutils by @piotrm0 in #1052
• pin streamlit-aggrid version by @piotrm0 in #1043

New Contributors

• @markdavidmc0 made their first contribution in #997
• @nicoloboschi made their first contribution in #1036

Full Changelog: trulens-eval-0.27.0...trulens-eval-0.27.1

trulens-eval-0.27.0

What's Changed

• Python 3.12 support by @joshreini1 in #1012
• Design guidelines for contributors @piotrm0 in #1015
• Pull request template by @piotrm0 in #1021
• Handle utf8 encoding issues in trulens database @arn-tru in #1023
• Split system and user prompts for feedback functions by @joshreini1 in #1018
• Enable Meta and Mistral models on AWS Bedrock by @joshreini1 in #1018
• Added support for Langchain MultiQueryRetriever by @sayedsohan in #1014
• Add Vectara Hallucination Detection Model by @Josephrp in #950
• Parametrize temperature for create chat completion by @daniel-huang-1230 in #1026

Examples

• Update LiteLLM quickstart to show TogetherAI model usage by @joshreini1 in #1018
• Add Claude-3 as a feedback provider example by @joshreini1 in #1018
• Notebook to show evaluation of Langchain MultiQueryRetriever by @sayedsohan in #1014
• Added example to show usage of Vectara Hallucination Detection Model by @Josephrp in #950

New Contributors

• @arn-tru made their first contribution in #1023
• @sayedsohan made their first contribution in #1014
• @Josephrp made their first contribution in #950

Full Changelog: trulens-eval-0.26.0...trulens-eval-0.27.0

trulens-eval-0.26.0

What's Changed

• QS Relevance -> Context Relevance by @joshreini1 in #977
• Verify feedback selectors on recorder init by @piotrm0 in #961
• Relax llama version by @joshreini1 in #985
• Allow VirtualRecords to have multiple calls to the same component. by @piotrm0 in #988
• Allow Feedback.run with args even if they had selectors specified. by @piotrm0 in #1003

Documentation

• update doc-building requirements by @piotrm0 in #990
• docs updates/additions by @piotrm0 in #996
• Update feedback docs by @joshreini1 in #999
• doc usage formatting by @piotrm0 in #1002

Examples

• Existing data quickstart by @joshreini1 in #976
• Adds Azure Quickstart for LangChain by @ingridstevens in #984

Bug Fixes

• fix more docs links by @piotrm0 in #987
• Fix broken colab links by @joshreini1 in #994

Full Changelog: trulens-eval-0.25.1...trulens-eval-0.26.0

TruLens Eval v0.25.1

What's Changed

• Examples and Bug Fixes

Examples

• canopy quickstart by @joshreini1 in #925
• add virtual app setup by @joshreini1 in #967

Bug fixes

• Fix release pipeline: allow deserialization for faiss example by @joshreini1 in #978
• Fix TruChain instrumentation issue introduced in 0.24.1 by @joshreini1 in #974
• fix links in docs by @piotrm0 in #968
• Update typo in pip install on azure_openai.ipynb by @ingridstevens in #973
• fix colab link - langchain ensemble notebook by @joshreini1 in #980

Full Changelog: trulens-eval-0.25.0...trulens-eval-0.25.1

TruLens Eval v0.25.0

What's Changed

• add nemo guardrails integrations by @piotrm0 in #824
• Improve regex matching for structured output extraction from LLM feedback function providers by @daniel-huang-1230 in #954
• Better selection of main input/output by @joshreini1 in #938
• makefile targets for release process by @piotrm0 in #934
• Documentation structure and heading pages by @piotrm0 in #945
• update tru virtual docs by @piotrm0 in #949

Bug Fixes

• Fix release test pipeline by @joshreini1 in #962
• Enable test set generation for llama-index by @joshreini1 in #865
• fix links in docs by @piotrm0 in #963
• adjust all_tools symlink by @joshreini1 in #965
• instrumentation notebook updates and fixes by @piotrm0 in #953

Full Changelog: trulens-eval-0.24.1...trulens-eval-0.25.0

TruLens Eval v0.24.1

Library containing evaluations of LLM Applications

Changelog

• Bugfixes.

Bug Fixes

• Updated langchain and llama_index examples to avoid deprecated or outdated functions.
• Removed unneeded debug statements.

trulens-eval-0.24.0

What's Changed

• cost tracking tests and litellm cost tracking by @piotrm0 in #910
• check packages on init by @piotrm0 in #917
• Increase provider test coverage to Huggingface feedback provider by @venkatkakoju in #919
• upgrade Llama-Index integration to 0.10 by @joshreini1 in #891
• Update issue templates by @joshreini1 in #923
• async handling adjustments by @piotrm0 in #918

New Contributors

• @venkatkakoju made their first contribution in #919

Full Changelog: trulens-eval-0.23.0...trulens-eval-0.24.0

trulens-eval-0.23.0

What's Changed

• Improvements to Comprehensiveness Feedback by @joshreini1 in #901
• Deeper Instrumentation for Hybrid Retrievers by @joshreini1 in #873

Examples

• Evaluating Hybrid and Ensemble Retrievers by @joshreini1 in #873

Bug Fixes

• various documentation fixes by @piotrm0 in #907
• fix forward refs with model_rebuild by @piotrm0 in #905
• cleanup by @piotrm0 in #880
• fix for in-memory sqlite params by @piotrm0 in #904
• Fix use case colab links by @joshreini1 in #900
• few site-related fixes to recently merged pr by @piotrm0 in #903
• Capture multi-line criteria and reasons for COT feedbacks by @joshreini1 in #901

Full Changelog: trulens-eval-0.22.2...trulens-eval-0.23.0