ComputeArena CLI 0.1.2

ComputeArena CLI 0.1.2

This release changes how a benchmark runs and which benchmarks can be
published. The headline workloads, PP512 and TG128, now run before the rest of
the prefill sweep, reports sign a description of the host they ran on, and
computearena.ai accepts only complete runs, so every published benchmark can be
compared with every other at every size. It also says when the installed
BaseRT is worth updating, and explains what happens when a benchmark you
deleted on the website is submitted again.

Install or upgrade

On macOS or Linux:

curl -LsSf https://computearena.ai/install.sh | sh

The same command upgrades an existing installation. Saved reports, the
installation signing key, login sessions, model receipts, and installed
runtimes are kept. Release archives and checksums are also available below for
manual installation.

BaseRT users should also upgrade BaseRT to 0.2.5 or newer, with
computearena basert install or the official BaseRT installer. ComputeArena
selects the new BaseRT behaviour below by what the installed harness
advertises: with BaseRT 0.2.4 it keeps working exactly as before, the report
records that the older protocol was used, and the client now says so before
the run.

Highlights

• Only complete runs are published. A run is submittable when it contains
  the full default sweep: every default prefill size, PP128 to PP16384, and
  TG128. That is what run does when --pp and --tg are left out. Extra
  prefill sizes are fine, and the number of repetitions is not part of the
  rule. A run with a custom --pp or --tg is still a valid signed report
  that you can save, inspect, and verify, but it is local only, and the client
  says so at every step: in the benchmark plan before the run is confirmed,
  with the workloads it will lack; after the run, in place of the submit hint;
  as LOCAL ONLY in list and in the submission pickers, where such reports
  are not pre-selected; and at submit, where partial runs are listed apart
  from invalid reports and refused before login or any upload, while complete
  runs in the same batch still go through. computearena.ai enforces the same
  rule, so earlier client releases receive the same explanation from the
  server.
• Headline first. PP512 runs first, then TG128, then the remaining prefill
  sizes; a custom sweep without PP512 uses its first size as the headline.
  Prefill starts from an empty context and decode starts from one untimed seed
  token. Warmup, repetitions, timers, and telemetry are unchanged for both
  runtimes.
• BaseRT reserves 4K for the headline. A BaseRT harness that advertises
  features.headline_context_capacity, which BaseRT 0.2.5 is the first
  release to do, is run with --headline-first: the headline is measured in a
  freshly loaded model with a 4K context reservation, and the client checks
  the returned capacity, history, order, and repetitions before signing. These
  reports normalize to computearena-throughput/3. The same harness reports
  power, energy, temperature, and memory from the timed repetitions
  themselves, and the client uses that same-run telemetry in place of
  observing the process from outside. Older harnesses keep their existing
  invocation and recorded protocol, with no new flags passed to them.
• BaseRT update notices. ComputeArena says when the BaseRT it found is
  worth updating, and never refuses to run an older one. A harness that
  predates the headline-first protocol is named before the benchmark plan,
  with what its report will be signed as; that needs no network, because the
  harness describes itself. A newer BaseRT release is mentioned the same way,
  from a background lookup that is remembered for a day and never delays a
  run. The full-screen interface shows the notice above its menu and offers
  u to install the latest release, asking for a second press before it
  replaces anything. A harness chosen with --runtime-path or
  COMPUTEARENA_BASERT_HARNESS is left for you to update, and platforms
  without a prebuilt BaseRT are pointed at the release to build from.
• Stock llama-bench, differences recorded. ComputeArena still runs your
  unmodified llama-bench. It has no independent context reservation option,
  so its native PP+TG+depth request is kept and the report records that the 4K
  target was not applied; -d 4096 is never used as a substitute, because it
  would add real token history. Standard runs use one process for the
  headline, one for decode, and one for the remaining prefill sizes; thermally
  controlled runs still use one process per workload. The workload order,
  warmup, and context requests are part of the signed report.
• The host is part of the signed report. Signed before-and-after
  environment boundaries (computearena-environment/1) record the OS family,
  version and kernel, logical and physical CPU counts, total and available
  memory, swap use, macOS memory pressure or Linux PSI, the power or
  performance mode, and the available GPU configuration. Slow probes run only
  outside the measured process windows, and static macOS GPU details are read
  once rather than at every boundary.
• Deleted benchmarks say so. A benchmark deleted on the website cannot be
  published again from the same saved report. The client says this before a
  submission, labels such a report Failed: previously deleted, keeps
  uploading the other selected reports, and ends with the uploaded, duplicate,
  and failed counts and a nonzero exit status. Local files are unchanged. To
  publish that result again, run the benchmark again with the same settings
  and submit the new report; it counts as a separate benchmark.

Reports and compatibility

Reports saved by 0.1.0 and 0.1.1 remain verifiable. They remain submittable
when they contain the full default sweep, which is what those releases produce
unless --pp or --tg was passed. Benchmarks already published on
computearena.ai are not changed by this release.

Matching workload labels do not promise identical conditions across runtimes.
BaseRT and llama.cpp differ in warmup, context reservation, and process layout;
those differences are recorded in each signed report rather than hidden, and
computearena.ai does not split rankings or add a filter because of them.

Supported systems

Prebuilt ComputeArena binaries are available for:

• macOS on Apple Silicon;
• Linux x86_64 with glibc 2.31 or newer; and
• Linux arm64 with glibc 2.31 or newer.

Windows and Intel Macs are not supported in this release. ComputeArena does not
bundle an inference runtime or model; install BaseRT or llama.cpp and obtain a
compatible model before benchmarking. Prebuilt BaseRT runtimes exist for macOS
on Apple Silicon and for Linux arm64 with CUDA; on Linux x86-64, use llama.cpp
or a BaseRT benchmark harness you built yourself.

The macOS binary is not signed with an Apple Developer ID. The release archive
is checksum-verified and carries a Sigstore keyless signature from the release
workflow. Browser-downloaded copies may need their quarantine attribute removed
as described in the README.

Trust and limits

A valid signature shows that a report was not changed after it was signed. It
does not attest that the runtime executed the benchmark honestly or that the
recorded host description is truthful; computearena.ai recomputes every rate
from the signed raw samples and repeats the model identity checks on its own,
because a public client is not a trust boundary. Unless a harness reports its
own same-run telemetry, telemetry is observed from outside the runtime at
one-second intervals, so short peaks can be missed, and a process-window
reading is not a per-token metric.

Accuracy benchmarks, MLX, vLLM, Windows, and Intel macOS support are not part
of v0.1.2.

For usage and measurement details, see the
README,
docs/benchmark-profiles.md,
docs/runtime-adapters.md,
and
docs/telemetry.md.
Questions and feedback are welcome in the
ComputeArena Discord.