Releases: ContextJet-ai/awesome-llm-observability
Releases · ContextJet-ai/awesome-llm-observability
Release list
v1.1.0 - 26 skills, tested code, one-command install
Major expansion since v1.0.0:
- 26 Agent Skills (up from 6) covering the full LLM app lifecycle: instrument, build eval sets, evaluate, eval-driven dev, optimize prompts, debug from traces, trace-based testing, collect feedback, annotate/review, reduce cost, resilience/fallbacks, RAG quality, hallucination detection, model comparison, agent success, multi-agent tracing, guardrails, red-teaming, drift alerts, A/B testing, PII-safe tracing, and tool selection.
- 4 code-backed, unit-tested skills (marked with a gear): a cost calculator, an OTel GenAI span validator, a Luhn-checked PII scrubber, and a self-consistency hallucination check. 22 tests, run in CI.
- One-command install as a Claude Code plugin, plus a degit one-liner and an installer script.
- 50+ verified tools with a platform comparison matrix, a finance/compliance guide, and a vendor-neutral OpenTelemetry tracer.
- Community additions: Azure Monitor and Azure AI Evaluation (Foundry).
Everything is CC0. Contributions welcome.
v1.0.0 — Awesome LLM Observability
First release of Awesome LLM Observability — a curated, research-grounded list of 48+ verified tools for LLM tracing, evaluation, monitoring, prompt management, guardrails & LLMOps, plus 6 original Agent Skills, a finance/compliance guide, and a vendor-neutral OpenTelemetry GenAI tracer.
Covers: Langfuse, Arize Phoenix, Comet Opik, LangSmith, Helicone, Latitude, Laminar, MLflow, promptfoo, DeepEval, Ragas, LiteLLM, OpenLLMetry, OpenInference, Guardrails AI, and more.
Contributions welcome — see CONTRIBUTING.md.