Skip to content

v0.1.0

Choose a tag to compare

View all tags
@vishwanathakuthota vishwanathakuthota released this 03 May 08:48
· 76 commits to main since this release
v0.1.0
2761c3c

🚀 OpenVals

Open-source AI Model Evaluation, Benchmarking & Trust Layer for LLMs

OpenVals helps you evaluate, compare, and trust AI models (OpenAI, Claude, Gemini, Ollama, open-source models) using standardized metrics + real-world benchmarking + explainable scoring.

⸻

Why OpenVals?

Most AI models look great in demos—but fail in production.

OpenVals answers:

  • Which model is actually best for your use case?
  • How do models compare beyond just “accuracy”?
  • Can I trust this model in production?
  • Which model is fastest, safest, and most reliable?

🧠 What OpenVals Does

OpenVals is a 3-layer AI evaluation system:

  1. 📊 Evaluation Layer
  • Accuracy scoring
  • Semantic similarity
  • Latency measurement
  • Reliability scoring
  • Safety scoring
  1. 🏁 Benchmarking Layer
  • Multi-model comparison (Ollama, OpenAI, Claude, etc.)
  • Normalized scoring across models
  • Ranking engine
  1. 🎯 Recommendation Engine
  • Suggests best model for your dataset
  • Tradeoff-aware ranking (speed vs accuracy vs safety)
  • Use-case based model selection (coming next version)

Full Changelog: v0.0.5...v0.1.0

Assets 2
Loading