Skip to content

Releases: drivelineresearch/vast-benchmarking

Vast Benchmarking v0.1.1

Choose a tag to compare

@kyleboddy kyleboddy released this 22 Aug 20:53

Vast Benchmarking v0.1.1

This release changes the writing, not the benchmark. Workload code, acceptance rules,
stored demo results, and dashboard calculations are unchanged from v0.1.0.

Documentation edit

Every tracked Markdown file was reviewed against the Humanizer 2.1.0 checklist. The
edit removed decorative emoji headings, canned transitions, unnecessary bold text, and
sentences that sounded more like generated copy than technical documentation. Commands,
machine IDs, prices, measured results, warnings, and cost limits were left intact.

The same review covered explanatory Python comments in the API client, Vast runner, and
web application. Shebangs and coverage pragmas were left alone because they are machine
directives rather than prose.

Package and repository

  • The wheel and source archive include the MIT License and report MIT as their SPDX
    license expression.
  • CI builds and scans both archives before a release is accepted.
  • main still requires the test and repository-scan checks, one approval, resolved
    conversations, and linear history.
  • The repository becomes public only after the v0.1.1 tag, release assets, and checksums
    have been verified.

Verification

The release gate runs Ruff, 19 tests, the dashboard JavaScript syntax check, local
Markdown link validation, the public-release scan, and distribution inspection. The
published wheel and source archive are downloaded again and checked against
SHA256SUMS before repository visibility changes.

Vast Benchmarking v0.1.0

Choose a tag to compare

@kyleboddy kyleboddy released this 22 Aug 20:13

Vast Benchmarking v0.1.0

The first packaged release provides a bounded Python benchmark, safe Vast.ai controller,
SQLite result store, and Flask dashboard for comparing container-visible hardware
capacity.

Included

  • Concurrent CV and FP16 GPU stress across every visible CUDA device.
  • Effective CPU concurrency, single-thread SHA-256, memory copy, and durable disk tests.
  • A 540-second standard-profile ceiling with partial-run validation.
  • Single-offer and parallel Vast runners with spend limits and exact-instance cleanup.
  • Category rankings normalized to the measured leader, with historical price context and
    performance-per-dollar sorting.
  • A professional light dashboard with self-hosted Geist Sans and Geist Mono.
  • Wheel and source distributions containing the CLI, dashboard, generated artwork, and
    favicon family.
  • Modular public guides for the measured demo, methodology, Vast.ai execution, and
    generic self-hosting, with automated local-link checks.
  • MIT licensing under Driveline Research.

Demonstrated snapshot

The sanitized demo contains 19 stored runs, 15 accepted machines, and 60 tested GPUs. Its
measured leaders include 118,420.80 CV images/s from eight RTX 4070 Super GPUs, 368.64
effective CPU cores on an EPYC 9654 rental, and 5.333 GB/s single-thread SHA-256 on a
Core Ultra 9 285K rental. The sanitized campaign report and machine annotations are
included in the source archive.

See the versioned demo snapshot
for exact category leaders, campaign cost, acceptance caveats, and cleanup confirmation.

Rental rates shown in the dashboard and reports are historical observations captured at
benchmark time, not current Vast.ai quotes.