Skip to content
#

benchmarking-framework

Star

Here are 11 public repositories matching this topic...

Language: Python
Filter by language
All 11 Python 11 C++ 8 Java 3 C# 2 PHP 2 Emacs Lisp 1 Smalltalk 1
Sort: Most stars
Sort options
Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

eXascaleInfolab / PyExPool

Star 163

Python Multi-Process Execution Pool: concurrent asynchronous execution pool with custom resource constraints (memory, timeouts, affinity, CPU cores and caching), load balancing and profiling capabilities of the external apps on NUMA architecture

multiprocessing parallel-computing numa monitoring-server cache-control task-queue application-framework parallel-processing execution-pool benchmarking-framework load-balancing in-memory-computations
  • Updated Aug 28, 2019
  • Python

Psycoy / MixEval

Star 3

MixEval, a ground-truth-based dynamic benchmark derived from off-the-shelf benchmark mixtures, which evaluates LLMs with a highly capable model ranking (i.e., 0.96 correlation with Chatbot Arena) while running locally and quickly (6% the time and cost of running MMLU), with its queries being stably updated every month to avoid contamination.

benchmark machine-learning deep-learning evaluation benchmarking-suite evaluation-framework benchmarking-framework foundation-models large-language-models large-language-model llm-training llm-inference llm-evaluation large-multimodal-models llm-evaluation-framework benchmark-mixture dynamic-benchmark
  • Updated Jun 1, 2024
  • Python

Improve this page

Add a description, image, and links to the benchmarking-framework topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the benchmarking-framework topic, visit your repo's landing page and select "manage topics."

Learn more