TruLens 2.14.0

TruLens 2.14.0 Release Notes

Headline Features

Coding Agent Instrumentation

Instrument IDE hooks and coding agent workflows — including Claude Code and Cursor — with TruLens tracing. Capture the full execution trace of agentic coding sessions for evaluation and observability. (#2732) by @joshreini1

Production Trace Curation into Evaluation Datasets

Curate production traces into persisted evaluation datasets. Select and save real-world traces directly from your running application, building ground-truth datasets for offline evaluation without manual data collection. (#2722) by @Payal2000

Connector-Backed Prompt Management with Version Evaluation

Store, version, and evaluate prompt templates through the connector interface. Track prompt changes over time and evaluate the impact of prompt edits against your existing datasets and feedback functions. (#2717) by @dylanpulver

OTLP-Native Export and OTel Metrics Signal

Export traces and metrics directly via the OpenTelemetry Protocol, enabling integration with any OTLP-compatible backend. (#2681) by @RyanFlowerYes

---

Additional Features

• Streaming token tracking with TTFT and throughput metrics — OTEL instrumentation now captures streaming token counts, time-to-first-token, and throughput for LLM calls. (#2708) by @Abelo9996
• Ollama provider — New trulens-providers-ollama package for running feedback functions against local Ollama models. (#2688) by @connectsudhindra-gif
• Conversation metric customization — additional_instructions and chain-of-thought reasons support for conversation-scoped metrics. (#2710) by @dylanpulver

Dashboard Improvements

• Leaderboard record counts — Leaderboard metrics now show how many records each metric actually scored. (#2698) by @CTWalk
• TTFT in the Records page — Time-to-first-token is now visible in the latency view on the Records page. (#2696) by @connectsudhindra-gif
• Conversation thread column labels — Conversation-scoped thread columns are now labeled "latest" instead of "avg" to reflect their actual semantics. (#2750) by @Abelo9996
• Topic Adherence score consistency — Topic Adherence scores now match between the results table and conversation metrics view. (#2733) by @Abelo9996

Bug Fixes

• LLM judge rating extraction — Fixed a bug where the judge's stated scale description was incorrectly parsed as its numeric rating. (#2726) by @arthi-arumugam-git
• ndcg_at_k shape crash and cohens_kappa label mutation — ndcg_at_k no longer crashes on mismatched shapes; cohens_kappa no longer mutates its input labels. (#2709) by @ClaireXi99
• Ground-truth row ordering — Benchmark scoring now processes ground-truth rows in row order, not index-label order. (#2737) by @arthi-arumugam-git
• IR metric undefined results — IR metrics now report nan instead of 0.0 when undefined (e.g., empty relevance lists). (#2738) by @arthi-arumugam-git
• Duplicate chunk handling in IR metrics — recall_at_k and ndcg_at_k are now correctly bounded when retrieved chunks contain duplicates. (#2747) by @Abelo9996
• Temperature argument for conversation metrics — temperature stays positional on conversation metric calls, fixing a regression. (#2728) by @BetterAndBetterII
• Dataset and ground-truth read round-trips — Fixed serialization round-trip errors when reading datasets and ground-truth entries from the database. (#2735) by @Abelo9996
• Feedback definition re-insert — Feedback definition updates are now persisted correctly on re-insert. (#2743) by @Abelo9996
• compute_metrics message formatting — Status messages now build from metric names, not raw objects. (#2745) by @Abelo9996
• Anthropic cost tracking — Fixed a crash in the Anthropic provider when tracking costs via core_endpoint.Cost. (#2756) by @Abelo9996
• methods_to_instrument existence check — App wrappers now use Lens.get for safer instrumentation target checks. (#2754) by @Abelo9996
• OpenAI moderation_harassment_threatening — This moderation method now reads the correct category from the API response. (#2752) by @Abelo9996
• OTEL tracing default cleanup — Removed stale enablement references; TruLens now warns when OTEL tracing is explicitly disabled. (#2724) by @sfc-gh-jreini
• Ruff version alignment — All three ruff pins are now aligned on 0.16.3. (#2720) by @dylanpulver
• Conda recipe updates — Conda recipes updated for TruLens 2.13.0. (#2713) by @sfc-gh-jreini
• Missing docstring fix — Added missing docstring on public function pytest_collection_modi. (#2706) by @nickhac

New example

• Trace debugging cookbook — New notebook walking through common trace debugging workflows. (#2716) by @dylanpulver

New Contributors

• @dylanpulver made their first contribution in #2710
• @ClaireXi99 made their first contribution in #2709
• @Abelo9996 made their first contribution in #2708
• @nickhac made their first contribution in #2706
• @CTWalk made their first contribution in #2698
• @arthi-arumugam-git made their first contribution in #2726
• @connectsudhindra-gif made their first contribution in #2688
• @BetterAndBetterII made their first contribution in #2728