Skip to content

evaluation-harness v0.4.4

Choose a tag to compare

@github-actions github-actions released this 28 Oct 20:00
· 43 commits to main since this release
6f0b5e4

The catalog says what it is and links out: --about puts a deployment's own paragraph on the
overview, a goal's links are shown, and --repo turns project paths into links. The arms page
now carries the deployment's options like every other page. No result moves.

Added

  • evh catalog --about FILE: a deployment's own paragraph on the overview, under its
    description. It says which project the catalog shows and where its findings are. None by
    default.
  • A goal's links on its page and in evh goals --goal G. The field was in the goal
    schema but shown nowhere. A web address opens; a path in the project is named as text, because
    the static pages cannot know where the project is hosted.
  • evh catalog --repo URL: where the project's files can be read (e.g. a GitHub blob/main/
    address). A project path in a goal's links, or at the start of a guardrail's source, then links
    there. Without it, the path stays text.

Changed

  • The overview says what the catalog is: "pages the harness builds from this project's
    committed runs", and that the inspector tests any arms against a baseline, item by item.

Fixed

  • evh catalog built the arms page without the deployment's options. Run as a command, the
    builder is __main__, and the arms page was rendered by a second copy of it with the
    defaults. So arms.html said "eval catalog" whatever --brand was, and had no --link
    switcher, no --head and the default --api. Every page now carries them.

Install

pip install "evaluation-harness @ git+https://github.com/chipi/eval-harness@v0.4.4#subdirectory=harness"