(IROS 2020, ECCVW 2020) Official Python Implementation for "3D Multi-Object Tracking: A Baseline and New Evaluation Metrics"

The LLM Evaluation Framework

OCTIS: Comparing Topic Models is Simple! A python package to optimize and evaluate topic models (accepted at EACL2021 demo track)

Python SDK for agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks like CrewAI, Langchain, and Autogen

Evaluate your speech-to-text system with similarity measures such as word error rate (WER)

A Neural Framework for MT Evaluation

A Python wrapper for the ROUGE summarization evaluation package

📈 Implementation of eight evaluation metrics to access the similarity between two images. The eight metrics are as follows: RMSE, PSNR, SSIM, ISSM, FSIM, SRE, SAM, and UIQ.

PyNLPl, pronounced as 'pineapple', is a Python library for Natural Language Processing. It contains various modules useful for common, and less common, NLP tasks. PyNLPl can be used for basic tasks such as the extraction of n-grams and frequency lists, and to build simple language model. There are also more complex data types and algorithms. Mor…

LightEval is a lightweight LLM evaluation suite that Hugging Face has been using internally with the recently released LLM data processing library datatrove and LLM training library nanotron.

An implementation of a full named-entity evaluation metrics based on SemEval'13 Task 9 - not at tag/token level but considering all the tokens that are part of the named-entity

Resources for the "Evaluating the Factual Consistency of Abstractive Text Summarization" paper

CLEval: Character-Level Evaluation for Text Detection and Recognition Tasks

Code base for the precision, recall, density, and coverage metrics for generative models. ICML 2020.

Metrics to evaluate the quality of responses of your Retrieval Augmented Generation (RAG) applications.

Code for "Semantic Object Accuracy for Generative Text-to-Image Synthesis" (TPAMI 2020)

Open-Source Evaluation for LLM Application Pipelines

⚡️A Blazing-Fast Python Library for Ranking Evaluation, Comparison, and Fusion 🐍

PySODEvalToolkit: A Python-based Evaluation Toolbox for Salient Object Detection and Camouflaged Object Detection

Evaluate your biometric verification models literally in seconds.