Repository navigation
- New opt-in
FMEV_RUNNER_UP=1: Choice questions with 3+ options get a second question to find the model's runner-up, which gets a larger share of the remaining probability instead of an even split. On jevbench's public items the runner-up was the right answer on 21 of 45 misses (47%, against 34% by chance). The chosen option and its probability are unchanged. It costs 1.5–2× latency, so it's off by default. - Score questions now give the remaining probability mostly to the levels next to the chosen one, so expected scores are no longer pulled toward the middle of the scale.
- The chosen option is always kept strictly above the others, so it never ties.
- Install:
brew install mrubens/tap/fmev
SHA-256 (fmev-0.1.5-macos-arm64.zip): 73eee4df9379d97d66e659e1f05bba463893f5254c3235cffb566ca0d68356f2