This repository contains analysis and predictive modeling of household electricity consumption using Python. It includes data cleaning, exploratory data analysis (EDA), time series forecasting (ARIMA, SARIMA, LSTM), and model evaluation to optimize energy usage.

Python SDK for agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks like CrewAI, Langchain, and Autogen

LightEval is a lightweight LLM evaluation suite that Hugging Face has been using internally with the recently released LLM data processing library datatrove and LLM training library nanotron.

Valor is a centralized evaluation store which makes it easy to measure, explore, and rank model performance.

Counting-Stars (★)

Python SDK for running evaluations on LLM generated responses

Activity and Sequence Detection Performance Measures: A package to evaluate activity detection results, including the sequence of events given multiple activity types.

The LLM Evaluation Framework

If you want to learn these topics, i refer you to go through this notebook.

A list of works on evaluation of visual generation models, including evaluation metrics, models, and systems

[NeurIPS'21 Outstanding Paper] Library for reliable evaluation on RL and ML benchmarks, even with only a handful of seeds.

VELOCITI Benchmark Evaluation and Visualisation Code

Open-Source Evaluation for LLM Application Pipelines

CIDEr score implementation for python 3.

Evaluates neuron segmentations in terms of statistics related to the number of splits and merges

Python client for Kolena's machine learning testing platform

Evaluate custom and HuggingFace text-to-image/zero-shot-image-classification models like CLIP, SigLIP, DFN5B, and EVA-CLIP. Metrics include Zero-shot accuracy, Linear Probe, Image retrieval, and KNN accuracy.

The most comprehensive Python package for evaluating survival analysis models.

An implementation of a full named-entity evaluation metrics based on SemEval'13 Task 9 - not at tag/token level but considering all the tokens that are part of the named-entity

Metrics to evaluate the quality of responses of your Retrieval Augmented Generation (RAG) applications.