Skip to content

Releases: ContextJet-ai/awesome-llm-observability

v1.1.0 - 26 skills, tested code, one-command install

Choose a tag to compare

@NishchayMahor NishchayMahor released this 05 Jul 19:48

Major expansion since v1.0.0:

  • 26 Agent Skills (up from 6) covering the full LLM app lifecycle: instrument, build eval sets, evaluate, eval-driven dev, optimize prompts, debug from traces, trace-based testing, collect feedback, annotate/review, reduce cost, resilience/fallbacks, RAG quality, hallucination detection, model comparison, agent success, multi-agent tracing, guardrails, red-teaming, drift alerts, A/B testing, PII-safe tracing, and tool selection.
  • 4 code-backed, unit-tested skills (marked with a gear): a cost calculator, an OTel GenAI span validator, a Luhn-checked PII scrubber, and a self-consistency hallucination check. 22 tests, run in CI.
  • One-command install as a Claude Code plugin, plus a degit one-liner and an installer script.
  • 50+ verified tools with a platform comparison matrix, a finance/compliance guide, and a vendor-neutral OpenTelemetry tracer.
  • Community additions: Azure Monitor and Azure AI Evaluation (Foundry).

Everything is CC0. Contributions welcome.

v1.0.0 — Awesome LLM Observability

Choose a tag to compare

@NishchayMahor NishchayMahor released this 05 Jul 09:49

First release of Awesome LLM Observability — a curated, research-grounded list of 48+ verified tools for LLM tracing, evaluation, monitoring, prompt management, guardrails & LLMOps, plus 6 original Agent Skills, a finance/compliance guide, and a vendor-neutral OpenTelemetry GenAI tracer.

Covers: Langfuse, Arize Phoenix, Comet Opik, LangSmith, Helicone, Latitude, Laminar, MLflow, promptfoo, DeepEval, Ragas, LiteLLM, OpenLLMetry, OpenInference, Guardrails AI, and more.

Contributions welcome — see CONTRIBUTING.md.