Repository navigation
Faster answers from evh and its MCP server. No result moves.
Added
evh mcp --write-answers FILEandevh mcp --public --answers FILE: a release's answers written ahead. Each
call to the MCP server runs a freshevhprocess, several seconds of CPU; a deployment that limits the server's CPU
pays it in tens of seconds (the evals server's firstgoalsafter each start, and after each hour, took 33 s at
half a core).--write-answerswrites the answer of every tool that takes no argument and does not depend on the
date (notovervieworactivity); a public server started with--answersserves them without running the
command, for as long as it runs. A line another harness version wrote is not read.
Fixed
- A view and a catalog build read each arm's scores once.
evh goalsread 1,054 arms' scores for 686 arms; that
reading was 79% of its CPU. The answers are byte for byte the same;goals,armsandmodelstake about a
quarter to a third less CPU.
Install
pip install "evaluation-harness @ git+https://github.com/chipi/eval-harness@v0.6.2#subdirectory=harness"