Skip to content

Releases: pctutu/ai-benchmark

Release list

1.0.0

1.0.0 Latest
Latest

Choose a tag to compare

View all tags
@pctutu pctutu released this 01 Sep 06:53
v1.0.0
acceffc
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

SADB v1.0.0

This is the first public release of Super AI Desktop Benchmark.

Highlights

  • Frozen sadb/1.0 protocol and deterministic 72-question desktop capability bank.
  • Five-request stability suite with success rate, nearest-rank P50/P95, and structured errors.
  • Official cross-platform sadb CLI for OpenAI-compatible providers.
  • Unlimited provider response time by default; elapsed time remains observable metadata.
  • Public conformance tests and frozen bank SHA-256 verification.
  • English primary documentation and Simplified Chinese README.

Verification

Verify every downloaded archive using SHA256SUMS.txt. The archives contain no API keys or benchmark result artifacts.

Compatibility

Only compare results when all of these match:

protocol:    sadb/1.0
bank:        sadb-bank-1.0
scorer:      sadb-score-1.0
fingerprint: 7133690d67ef91609aaf1810a1d8ebc659cee1c016d381eaf388e015d29b5598

Known Scope Limits

V1 does not benchmark tool execution, agent traces, streaming throughput, multimodal input, cost, or sandbox behavior. These are intentionally reserved for later standards.

Full Changelog: https://github.com/pctutu/ai-benchmark/commits/v1.0.0

Assets 5
Loading