Skip to content

FrontierTrials v0.1.0

Choose a tag to compare

View all tags
@CAOShurong CAOShurong released this 04 Aug 16:24
v0.1.0
190e1d6

FrontierTrials v0.1.0 is the first public release of the no-API, local-first blind evaluation workbench.

Included

  • Exact Markdown response capture with SHA-256 integrity records
  • Deterministic blind aliases, all-pairs comparisons, balanced left/right order, and rater allocation
  • Self-contained offline HTML judging packets with downloadable JSON ballots
  • Validated ballot import and trial state transitions
  • Bradley–Terry relative-strength estimates with task-clustered bootstrap intervals
  • Pointwise rubric summaries, task slices, reviewer agreement, position checks, and verbosity diagnostics
  • Revealed portable HTML reports, protocol and ranking exports, structural audits, and content-addressed evidence seals
  • Seven documented JSON schemas and a fully fictional 191-artifact demonstration
  • 85 automated tests across Windows and Ubuntu on Python 3.11 and 3.13

Evidence

The bundled fictional demonstration contains 8 tasks, 4 candidates, 32 captured responses, 48 pairings, 96 ballots, and 224 sealed files. Its audit reports zero errors and zero warnings.

Evidence root:

sha256:c680bd65cfd31f4a1281582a3ae6b920bb1ef5975b462d982989dff1ea7aa9f5

The demonstration does not measure any real model or provider.

Install

Download the wheel below and run:

python -m pip install frontiertrials-0.1.0-py3-none-any.whl
frontiertrials demo my-fictional-trial
frontiertrials audit --trial my-fictional-trial
frontiertrials verify --trial my-fictional-trial

See the project website, revealed demo report, and blind reviewer packet.

Assets 5
Loading