Skip to content

Releases: M007-Net/local-model-bench

Release list

Local Model Bench 1.8.0

Local Model Bench 1.8.0 Latest
Latest

Choose a tag to compare

View all tags
@M007-Net M007-Net released this 14 Sep 17:14
v1.8.0
1159871

First public release. Windows 10 or 11, 64-bit.

Before you start

You need LM Studio with at least one model downloaded, and its local server running:

  1. Get LM Studio from https://lmstudio.ai and install it.
  2. Open the Discover tab, pick a model and download it. Something around 4 GB is a fine start.
  3. Open the Developer tab and switch the local server on (port 1234).

This app measures models LM Studio already has. It never downloads one itself.

Install

  1. Download Local-Model-Bench-Setup-1.8.0.exe below.

  2. Check the hash before running it. This installer is not code-signed, so this is the only way to confirm you have the file this release actually published. Open Terminal and run:

    Get-FileHash -Algorithm SHA256 "$HOME\Downloads\Local-Model-Bench-Setup-1.8.0.exe"

    It must match the SHA-256 below. SHA256SUMS.txt is attached and contains the same value.

  3. Run it. Windows will say "Windows protected your PC" — that appears for any program without a paid signing certificate. Click More info, then Run anyway.

It installs per-user and asks for no administrator rights.

SHA-256

Local-Model-Bench-Setup-1.8.0.exe

622410ec7b4f1cc58f9b00f213d3d145267a50db808eefbeace27ca75592431b

What it does

Runs your models against a set of questions at several concurrency levels, times them, checks the answers automatically, and keeps every response so you can read what was actually said.

Bundles three published question sets as locally adapted protocols — GSM8K (1,319 questions), an IFEval subset (205 prompts) and CRUXEval-O (750 items) — plus 12 objective accounting tests. You can import your own questions from a JSON, JSON Lines, CSV or TSV file.

GPU temperature, power and clocks are sampled during a run where the hardware reports them. Only the GPU sensors are read, so no kernel driver is loaded and no elevation is needed.

Results, run history and full responses stay in your own app-data folder. There is no telemetry and no analytics. The only request it can make beyond your LM Studio server is an update check, and that is off until you name a repository in Settings → Updates.

Notes

  • Unofficial community project. Not affiliated with, endorsed by, or supported by LM Studio.
  • Dataset and bundled-library licences are in THIRD_PARTY.md; the app's own source is MIT.
  • Full changes are in CHANGELOG.md.
Assets 4
Loading