Skip to content

Mogil Bench v0.1.0 (Public Alpha)

Pre-release
Pre-release

Choose a tag to compare

View all tags
@nmogil nmogil released this 15 Jul 10:31
v0.1.0
505ad00
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Mogil Bench v0.1.0 — public alpha

Mogil Bench is a reproducible coding-agent evaluation layer built around versioned benchmark packs, Harbor-managed execution, full Pi trajectories, strict reviewer-safe evidence, and blinded BlindBench review.

This is a public alpha. Expect interfaces and evidence contracts to evolve before a stable release.

Included

  • Versioned benchmark packs and deterministic local sample execution.
  • Harbor 0.18.0 Docker execution and optional Daytona SDK 0.196.0 execution.
  • Pi 0.80.6 trajectory capture in the pinned Daytona runtime.
  • Strict checksummed run bundles and reviewer-safe evidence projections.
  • Deterministic two-arm BlindBench paired-comparison CSV export.
  • Timeout-aware and idempotency-conscious BlindBench evidence uploads.
  • Public contribution, security, compatibility, and release documentation.

Verified release boundaries

  • Python: >=3.12,<3.13
  • Harbor: 0.18.0
  • Daytona SDK: 0.196.0
  • Container Pi runtime: 0.80.6
  • Immutable Daytona runtime:
    ghcr.io/mogilventures/mogil-bench-daytona-runtime@sha256:7728671c38220e066d23f63fd2544cc0722874ec40e1c86c883c8cc4d6c35dfe

Release verification

  • 140 local tests passed; the paid/manual Daytona smoke was intentionally deselected.
  • Explicit mandatory Docker smoke passed.
  • Ruff and strict mypy passed.
  • Wheel and sdist built from merge commit 505ad003f4a0e8bd8375ec475dc71320db08e196.
  • The wheel installed and completed the credential-free sample workflow in a clean Python 3.12 environment with PYTHONPATH unset.
  • Current checkout and Git history credential-pattern scans returned zero findings.
  • The immutable runtime was fetched anonymously and passed its Python, shell, Node/npm, Pi, and credential-residue contract inspection.

Artifacts

SHA-256 digests are attached in SHA256SUMS:

  • mogil_bench-0.1.0-py3-none-any.whl
    6e76c8d2d1ee926756f5e65de8749ef4678f7b161d3845b288468b6b8349c564
  • mogil_bench-0.1.0.tar.gz
    0a3ede88a5dea347362774e754756474f307d2671aba0b85388f8294c765cffa

Installation

Download the wheel from this release and install it in Python 3.12:

python -m pip install ./mogil_bench-0.1.0-py3-none-any.whl
mogil-bench --help

PyPI publication is intentionally deferred pending package-data policy and trusted-publishing approval.

See the README, release runbook, and changelog for details.

Assets 5
Loading