Releases: drivelineresearch/vast-benchmarking
Release list
Vast Benchmarking v0.1.1
Vast Benchmarking v0.1.1
This release changes the writing, not the benchmark. Workload code, acceptance rules,
stored demo results, and dashboard calculations are unchanged from v0.1.0.
Documentation edit
Every tracked Markdown file was reviewed against the Humanizer 2.1.0 checklist. The
edit removed decorative emoji headings, canned transitions, unnecessary bold text, and
sentences that sounded more like generated copy than technical documentation. Commands,
machine IDs, prices, measured results, warnings, and cost limits were left intact.
The same review covered explanatory Python comments in the API client, Vast runner, and
web application. Shebangs and coverage pragmas were left alone because they are machine
directives rather than prose.
Package and repository
- The wheel and source archive include the MIT License and report
MITas their SPDX
license expression. - CI builds and scans both archives before a release is accepted.
mainstill requires thetestandrepository-scanchecks, one approval, resolved
conversations, and linear history.- The repository becomes public only after the v0.1.1 tag, release assets, and checksums
have been verified.
Verification
The release gate runs Ruff, 19 tests, the dashboard JavaScript syntax check, local
Markdown link validation, the public-release scan, and distribution inspection. The
published wheel and source archive are downloaded again and checked against
SHA256SUMS before repository visibility changes.
Vast Benchmarking v0.1.0
Vast Benchmarking v0.1.0
The first packaged release provides a bounded Python benchmark, safe Vast.ai controller,
SQLite result store, and Flask dashboard for comparing container-visible hardware
capacity.
Included
- Concurrent CV and FP16 GPU stress across every visible CUDA device.
- Effective CPU concurrency, single-thread SHA-256, memory copy, and durable disk tests.
- A 540-second standard-profile ceiling with partial-run validation.
- Single-offer and parallel Vast runners with spend limits and exact-instance cleanup.
- Category rankings normalized to the measured leader, with historical price context and
performance-per-dollar sorting. - A professional light dashboard with self-hosted Geist Sans and Geist Mono.
- Wheel and source distributions containing the CLI, dashboard, generated artwork, and
favicon family. - Modular public guides for the measured demo, methodology, Vast.ai execution, and
generic self-hosting, with automated local-link checks. - MIT licensing under Driveline Research.
Demonstrated snapshot
The sanitized demo contains 19 stored runs, 15 accepted machines, and 60 tested GPUs. Its
measured leaders include 118,420.80 CV images/s from eight RTX 4070 Super GPUs, 368.64
effective CPU cores on an EPYC 9654 rental, and 5.333 GB/s single-thread SHA-256 on a
Core Ultra 9 285K rental. The sanitized campaign report and machine annotations are
included in the source archive.
See the versioned demo snapshot
for exact category leaders, campaign cost, acceptance caveats, and cleanup confirmation.
Rental rates shown in the dashboard and reports are historical observations captured at
benchmark time, not current Vast.ai quotes.