Skip to content

ComputeArena CLI 0.1.1

Choose a tag to compare

@github-actions github-actions released this 11 Sep 10:48
· 23 commits to main since this release
eb12e0b

ComputeArena CLI 0.1.1

This release makes model identity verifiable. Reports now state exactly which
published model file was benchmarked, the client checks that claim against
Hugging Face before uploading, and computearena.ai verifies it again on its
own. It also adds update notices, server-driven upgrade guidance, and clearer
BaseRT support on Linux x86-64.

Install or upgrade

On macOS or Linux:

curl -LsSf https://computearena.ai/install.sh | sh

The same command upgrades an existing installation. Saved reports, the
installation signing key, login sessions, and installed runtimes are kept, and
the data directory gains a model-provenance/ folder for the receipts
described below. Release archives and checksums are also available below for
manual installation.

Highlights

  • Verifiable model identity. Every new report carries a
    computearena-model/1 identity: the upstream Hugging Face model, the exact
    artifact (repository, immutable revision, file path, format, and
    runtime-specific quantization), the SHA-256 of the bytes that were
    benchmarked, and how they were obtained. The fields from 0.1.0 remain
    populated, and reports saved by earlier releases still submit.
  • Managed downloads leave a receipt. GGUF downloads are pinned to an
    immutable revision, hashed while streaming, and checked against the Hub's
    published SHA-256. BaseRT models pulled through the picker are matched to
    their exact published file by hash. Models installed before this release are
    matched the same way the first time they are benchmarked; when no published
    file matches, the report records the repository and the hash rather than a
    guessed file name.
  • BaseRT catalogue in the model picker. The BaseRT picker lists BaseRT's
    public catalogue and runs basert pull with a backend-aware target, so
    BaseRT keeps choosing the compatible artifact, downloading split files, and
    converting. Existing .base and .gguf files can still be entered directly.
  • computearena identify. Bind a file you downloaded or copied yourself to
    one exact Hub file: computearena identify ./model.gguf <hf-file-url>
    succeeds only when the local SHA-256 matches the published object. A
    repository URL or a model name is not enough, and a later identify can
    correct an earlier receipt.
  • Checks before submission. Before uploading, the client checks each
    report's artifact against Hugging Face and lists it as verified, unresolved,
    or check unavailable. A file whose hash contradicts the claimed published
    file is set aside as invalid; in the full-screen interface the remaining
    reports are still uploaded, and the summary says how many were left out.
    When computearena.ai has verified an artifact, its outcome is shown after
    the upload.
  • Model families. A repository's base_model link counts only when the
    repository declares itself a quantization of that model. Finetunes,
    adapters, and merges stay their own model, so an instruct model is not folded
    into its pretraining base. computearena.ai decides the final grouping, and a
    difference between the client's record and that grouping never blocks a
    submission.
  • Update notices and upgrade guidance. Interactive sessions check for the
    latest release in the background, keep the answer for a day, and mention a
    newer version when one exists; the check never blocks offline use. API
    requests carry the client version, and when the service retires a release
    the client prints the installed version, the minimum required, and the
    install command, then stops the submission batch.
  • BaseRT on Linux x86-64. ComputeArena warns before entering BaseRT on
    Linux x86-64, where no prebuilt BaseRT runtime is published, and points to
    llama.cpp or a compatible harness you built yourself instead of an installer
    that cannot help there.

Supported systems

Prebuilt ComputeArena binaries are available for:

  • macOS on Apple Silicon;
  • Linux x86_64 with glibc 2.31 or newer; and
  • Linux arm64 with glibc 2.31 or newer.

Windows and Intel Macs are not supported in this release. ComputeArena does not
bundle an inference runtime or model; install BaseRT or llama.cpp and obtain a
compatible model before benchmarking. Prebuilt BaseRT runtimes exist for macOS
on Apple Silicon and for Linux arm64 with CUDA; on Linux x86-64, use llama.cpp
or a BaseRT benchmark harness you built yourself.

The macOS binary is not signed with an Apple Developer ID. The release archive
is checksum-verified and carries a Sigstore keyless signature from the release
workflow. Browser-downloaded copies may need their quarantine attribute removed
as described in the README.

Model identity and trust

A verified artifact means the bytes that were benchmarked are a file published
on Hugging Face, and the model family follows what that file's publisher
declares. It does not prove that the publisher labelled the model correctly,
that the runtime executed those bytes honestly, or that two fine-tunes with
different lineage are equivalent. The client's checks are advisory;
computearena.ai repeats them independently, because a public client is not a
trust boundary. Files with no evidence, gated repositories the service cannot
read, and ambiguous merges stay unresolved rather than being grouped by file
name. Set HF_TOKEN in your environment to download from gated repositories.

Accuracy benchmarks, MLX, vLLM, Windows, and Intel macOS support are not part
of v0.1.1.

For usage and verification details, see the
README and
docs/model-identity.md.
Questions and feedback are welcome in the
ComputeArena Discord.