Skip to content

ComputeArena CLI 0.1.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 10 Sep 17:05
· 40 commits to main since this release
v0.1.0
7484e71
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

ComputeArena CLI 0.1.0

This is the first public release of the ComputeArena command-line client. It
runs versioned prefill and decode benchmarks through local inference
runtimes, keeps reports offline until you choose to submit them, and publishes
selected results to computearena.ai.

Install

On macOS or Linux:

curl -LsSf https://computearena.ai/install.sh | sh

The installer explains every change before proceeding, verifies the downloaded
archive, installs without sudo, and can configure ~/.local/bin on your
PATH. Release archives and checksums are also available below for manual
installation.

Highlights

  • BaseRT and llama.cpp adapters. Benchmark .base models with
    basert-benchmark-harness or .gguf models with llama-bench. Runtime
    capability checks, executable discovery, model selection, and result
    translation stay isolated behind a common adapter interface.
  • Interactive and scriptable workflows. Run computearena for the
    full-screen terminal interface, or use commands such as computearena basert run, computearena llama-cpp run, computearena list, computearena inspect, computearena verify, and computearena submit.
  • Offline-first signed reports. Every report is saved locally and signed
    with an installation-specific Ed25519 key. Runtime and model files are hashed
    before and after execution, and a changed artifact prevents the report from
    being signed.
  • Explicit submission control. Benchmarking does not require an account.
    When submitting, the client previews the exact public JSON, excludes local
    file paths, asks for confirmation, and handles valid reports separately from
    invalid ones.
  • Automatic telemetry. ComputeArena records available process memory,
    temperature, power-state, NVIDIA, and ROCm observations while using one
    measurement pass with repeated samples rather than separate telemetry
    replays. A capability-gated BaseRT path is ready for native same-run energy,
    allocator, and KV-cache telemetry.
  • Optional thermal conditioning. Standard runs start after warmup;
    thermally controlled runs add recorded cooldown waits. The plan shows the
    runtime-specific workload and time estimate before execution.
  • Runtime and model setup. The client can find installed runtime binaries,
    guide BaseRT model pulls, search and download GGUF files from Hugging Face,
    and install supported prebuilt runtime bundles after showing the source,
    destination, and integrity check.
  • Portable identity and storage. Conservative chip normalization keeps
    known aliases together without guessing unknown hardware. Existing staging
    data is migrated to the standalone ComputeArena data directory without
    changing saved reports, login sessions, signing keys, or installed runtimes.

Supported systems

Prebuilt ComputeArena binaries are available for:

  • macOS on Apple Silicon;
  • Linux x86_64 with glibc 2.31 or newer; and
  • Linux arm64 with glibc 2.31 or newer.

Windows and Intel Macs are not supported in this release. ComputeArena does not
bundle an inference runtime or model; install BaseRT or llama.cpp and obtain a
compatible model before benchmarking.

The macOS binary is not signed with an Apple Developer ID. The release archive
is checksum-verified and carries a Sigstore keyless signature from the release
workflow. Browser-downloaded copies may need their quarantine attribute removed
as described in the README.

Benchmark and integrity notes

The default text benchmark records three repetitions for prefill sizes 128
through 16384 and 128 decode tokens. BaseRT and llama.cpp retain distinct
protocol identifiers, context semantics, runtime versions, backends, and
quantization metadata so unlike measurements are not silently presented as
equivalent.

A valid report signature proves that the report has not changed since the
installation signed it. It does not prove that the client, runtime, driver,
operating system, or machine reported truthful results. Executable and model
hashes identify the files that were checked around the run; they are not remote
attestation.

Accuracy benchmarks, MLX, vLLM, Windows, and Intel macOS support are not part of
v0.1.0.

For usage and verification details, see the
README. Questions and
feedback are welcome in the
ComputeArena Discord.

Assets 14
Loading