Skip to content

WDB 12.9.25-2 - the crowd is out

Choose a tag to compare

@WebDevBarCoder WebDevBarCoder released this 28 Jul 23:23
· 310 commits to webdevbar since this release

Fork build on upstream 12.9.25.

Solstice Clash odds: the crowd is out

The displayed number was reporting high confidence and getting it wrong. 54 scored
predictions made that measurable: it was right 27 times in 54, and scored worse than
always guessing the base rate.

The cause was not subtle. The crowd's betting split is flat across its own confidence,
scores 0.83 logloss standalone against a 0.69 constant, and is a noisy echo of the ladder
rating - the same pick 40 times in 51, and on the 11 disagreements the rating is right 8
and the crowd 3. At weight 0.70 it outvoted every signal that works and supplied nearly
all of the displayed spread.

  • W_CROWD 0.70 to 0.0. Zero, not negative: the fitted slope is indistinguishable from
    zero, so this is "stop paying for it", not "bet against it". Pools and spectator counts
    are still read and recorded, so it stays a weight to revisit.
  • W_HEROES 0.50 to 1.0. The hero model measured no edge at 245 matches and clears its
    bar at 340, in two rounds, on shuffle splits and walk-forward validation, in three
    independent implementations.
  • SIGMA_THETA 0.15 to 0.20, as the optimum drifts up with the corpus.

The block now labels itself ODDS from rating + heroes, naming only what actually moved
the number.

Also

  • Matches pool across themes within an event. The old cross-theme weighting would have
    dropped every collected match to a third of its weight at a theme rotation - starving
    the model at the moment it had the most data.
  • The theme is decided by its dated window alone; a screen read is a hint for backfilling
    and never decides.
  • Fixed a display gate that reported "0 matches for this theme" against hundreds
    collected, whenever a match's ratings OCR failed.

Everything measured, including the ~58 model configurations that lost, is recorded in
docs/solstice-clash/model-findings-ledger.md with its sample size and what would re-open
it.