Skip to content

AI Benchmark 1.0.0

Latest

Choose a tag to compare

@AnonBOTpl AnonBOTpl released this 23 Sep 22:08
· 5 commits to main since this release

Benchmark for local models served by Ollama: time to first token, tokens/s and VRAM usage while generating, manual classification of answers, history and CSV/HTML export.

Requirements

  • Windows 10/11 (x64)
  • Ollama installed and running separately - it is not bundled. Everything here was measured against Ollama 0.34.3.
  • Optional: an NVIDIA card and driver for the VRAM metrics (NVML). Without one the app still works and only those metrics are skipped.

Install

  1. Download AI-Benchmark_1.0.0_x64-setup.exe below and run it. It installs per user (no administrator prompt) and is removed from "Apps & features" like any other program.
  2. The installer is not code-signed, so SmartScreen shows "Windows protected your PC" the first time you run it: choose More info -> Run anyway.
  3. Models are not bundled either. They are downloaded on demand from the Models tab, and the app shows the size of every one before the download.
SHA-256  49e95110d1487a9019696f7f1921f4d0c59ebe2c5773edbf2a5e03e1009e4bc7

Uninstalling leaves your data alone: %APPDATA%\com.aibenchmark.dev\ (model list, settings, history) and the exports in Downloads\AI Benchmark\.

What is inside

  • Live metrics: TTFT, tokens/s (Ollama's own numbers plus a wall-clock measurement), VRAM peak and the GPU/CPU split.
  • A memory forecast with four states before you download anything: on the card / on the card without headroom / partly on the CPU / will not run.
  • Categories with your own prompts: coding, chat, images (VLM), reasoning, JSON, long context.
  • Manual classification of answers - completed / refused / completed with limits or changes - three buttons, no second model guessing.
  • History of runs, A/B comparison over time, CSV and HTML export.
  • HuggingFace catalogue with per-variant sizes and memory verdicts.
  • Seven interface languages: English, Polish, German, Spanish, French, Portuguese, Italian.

Screenshots and full documentation: README.