Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks like CrewAI, Langchain, and Autogen

A library for evaluating Retrieval-Augmented Generation (RAG) systems (The traditional ways).

🥤 RAGLite is a Python toolkit for Retrieval-Augmented Generation (RAG) with PostgreSQL or SQLite

Evalica, your favourite evaluation toolkit

Our curated collection of templates. Use these patterns to set up your AI projects for evaluation with Openlayer.

Develop better LLM apps by testing different models and prompts in bulk.

The Modelmetry Python SDK allows developers to easily integrate Modelmetry’s advanced guardrails and monitoring capabilities into their LLM-powered applications.