Skip to content

evaluation-harness v0.4.1

Choose a tag to compare

@github-actions github-actions released this 27 Oct 13:06
· 78 commits to main since this release

A catalog that works on a phone and sits inside a deployment's own site, and an inspector that
no longer runs out of memory. No result moves.

Added

  • evh catalog --head FILE: HTML added to every page's <head>, for a deployment's own
    analytics or tags. The harness adds none of its own.
  • evh catalog --api PATH: where the inspector's API answers, for a catalog served under a
    path (default /api/; e.g. /catalog/api/).
  • evh catalog --brand NAME: the product's name beside the logo (default "eval catalog").

Changed

  • --link LABEL=URL is a switcher between the parts of a publication, not extra nav entries:
    the links head every page in order, the catalog marked (LABEL=. places it; without one it
    comes first as Catalog), and the logo opens the first. The catalog's own pages stay in its nav
    (a ☰ menu at the left of the bar on phones), and its front page is called Overview. The search
    box reads "Search ( / )", and on a phone it is a round search button in the bar that opens it below; a long
    count on the overview shows in short form on a phone (19.5M). The bar's buttons are SVG icons.
  • Tables on a phone: a cell breaks between words, never inside a name, so a narrow screen
    scrolls a wide table sideways; running text keeps a readable width; a long item id is
    shortened in the inspector's per-item table (the whole id is its title); no catalog or
    inspector page is wider than the screen.
  • No inline styles: the charts' legend colours are classes, so a deployment's content
    security policy can forbid style attributes.

Fixed

  • The inspector no longer runs out of memory. The paired test kept the signs it draws for
    every item count it met (~31 MB each), so --warm on a large catalog grew past a 256 MiB
    container limit and the server was killed every few minutes. At most two sign tables are kept
    now, stored as bytes, and at most 4096 estimates; a warmed inspector settles at ~150 MB. The
    p-values are the same, bit for bit.

Install

pip install "evaluation-harness @ git+https://github.com/chipi/eval-harness@v0.4.1#subdirectory=harness"