Judgment Calls

Judgment Calls

• Unpriced items count as unknown, not worthless. Items whose value is an effect the stat table does not price (Mystic Burst, Extra Charge, Healbane, Mystic Vulnerability…) had statValue 0, so a 5 %-usage stat stick like Battle Vest outscored items bought in 100 % of high-rank Lash games. They now get the median stat value of priced items so popularity and win rate decide. Found on the Lash data itself (100 %-usage items missing from the build), fixed before re-reading any agreement score; Infernus went 60 → 66 %, Lash 45 → 48 %, Mina unchanged at 64 %.

• Usage-weighted win lift (second agreement pass). Shrinkage by sample size did not remove the Lash problem above: Crippling Headshot's 63 % is computed from 5,500 matches, so K ≈ 1,070 barely moves it, yet the number is about which games it gets bought in, not what it does. Multiplying the lift by usage is the statement that the win rate is only an unbiased estimate when nearly everyone buys the item. Lash 48 → 54 %, Mina 64 → 67 %, Infernus unchanged at 66 %.

• Stat-table weight 0.8 → 0.2 (same pass; this one is a weight change and is reported as such). With the win-lift fixed, the remaining Lash misses were Headshot Booster, Extra Charge, Stamina Mastery and Tankbuster (81–99 % of high-rank games) losing to Battle Vest and Enchanter's Emblem (5 %). The score breakdown showed why: eff/kit are value-per-soul from a hand-set stat table, normalised by the maximum, and cheap stat sticks reach 1.0 while effect items sit at 0.05–0.2, so 1.6 points of hand-set opinion outweighed 1.0 point of 21k-match expert consensus. Two data-side fixes were tried first and rejected because they made every set worse (percentile-ranking the stat terms: 57/52/64; capping at the 90th percentile: 57/52/64). The stat terms were designed to correct all-rank popularity; once the population is Phantom+ they should be a tie-breaker and hero-fit nudge, not a driver. The full sweep is shown so the choice is transparent, not a picked point:

  eff = kit weight	Infernus	Lash	Mina
  0.8 (before)	66	54	67
  0.6	66	57	67
  0.4	73	64	63
  0.2 (chosen)	73	64	75
  0.1	73	64	75
  0.0	69	64	76

  0.2 is the smallest weight before the stat table stops mattering at all; it is kept because the kit term is the only place hero-specific scaling enters and because removing it costs Infernus. The validation sets were still never read by the generator, but a weight chosen by looking at three held-out scores is fitted to those three players in the ordinary sense; treat these numbers as in-sample for that one constant and out-of-sample for everything else.

• Yndio / Kelvin as a true out-of-sample check. Added after the weight change, with no further edits: 71 % (13 / 21 core). Chounted (leaderboard #1) was run first at 67 % and swapped for Yndio at the user's request; Yndio has 11× the Kelvin games and was 20–10 in the sample, so the core set is cleaner (11 items at ≥ 98 %). Misses: Rapid Recharge, Tankbuster, Extra Charge and Mystic Burst (all ≥ 98 % of Yndio's games), Enchanter's Emblem, Battle Vest, Monster Rounds, Mystic Reverb. Non-core picks: Healbane, Enduring Speed, Spirit Lifesteal, Spirit Burn, Transcendent Cooldown.

• One build instead of three archetypes (user request). Neutral stat multipliers were chosen rather than picking the best-scoring archetype, so the choice does not depend on the validation score. Infernus agreement for the single build is 60 % (the three archetypes were 61/67/63 %).

• Build styles (2026-09-06, user request after Warden sat at 52 %): a hero gets a second build only when a conditional population (games with one anchor item) is ≥ 5 % of high-rank games, drops ≥ 30 % of the hero's staples below half usage, and raises at least two new items to ≥ 50 %. Thresholds were set by looking at the detected styles for plausibility (Warden spirit at 7 % kept, McGinnis' 21-anchor group capped at 4), not by maximising agreement. Each rep is then scored against the closer build. Mean agreement 70.7 % → 72.7 %, Warden 52 % → 63 %. Pair stats stay unconditional because the API's permutation stats are top-600-capped and are not a co-occurrence measure.

• Usage floor of 3 % and shrinkage prior: the first draft picked 0–1 %-usage items purely on inflated win rates. This was fixed on data-quality grounds (selection bias), before looking at agreement scores.

• Popularity is relative to the hero's single most-bought item because the API does not return the hero's total match count.

• Permutation stats trimmed to the 600 most-played pairs and ability sequences to the top 400 per hero to keep the snapshot at ~16 MB.

• Generating from high-rank lobbies rather than all ranks: the tool's goal is a build a top player would recognise, and the all-rank Infernus mix (Rapid Rounds, Titanic Magazine, Ricochet) is not what high-rank lobbies buy (Extra Spirit, Healbane, Mystic Vulnerability, Rapid Recharge). This and the upgrade-chain rule raised Infernus agreement from 48/48/52 % to 61/67/63 % across the then three archetypes; no scoring weight changed.

• Buy order uses average purchase time rather than cost-ascending; that is how the aggregate data describes real games. The average time itself is not shown in the UI: it is skewed by outlier late buys and reads as a "buy at this minute" instruction, which it is not. Only the resulting order and phase are shown.

• Slot cap 8 per category (4 native slots + 4 flex), the most the shop allows of one type. It was 5 before the panel tuning; 5 cost about 1 point of mean agreement because heroes such as Sinclair and Dynamo stack one slot type.

• Standard mode = Unranked + Ranked in the Normal game mode for the held-out players.

• Items are counted once per match regardless of sells; starter items that get sold later still count.

• Description text: the assets API embeds inline SVG/HTML; it is stripped to plain text for the card. {s:sign} prefix tokens render as +.

• Disabled items (78) are kept in the catalog snapshot for completeness but are never recommended.

• Panel tuning (2026-09-06). With a 5-player panel for every hero (190 sets, see Held-out Validation) the user asked for the numbers to be set to maximise agreement across all characters. scripts/tune.ts ran coordinate descent over every value in PARAMS (six passes over fixed grids, keeping any change that raised the mean). Mean agreement 64.0 → 70.7 %, median 64 → 70 %, worst hero (Warden) 49 → 52 %. What changed, in order of effect:

  parameter	before	after	note
  popularity weight	1.0	2.0	the single biggest gain (+4 points on its own)
  slot cap	5	8	lets a build stack one slot type the way some heroes do
  final items / minimum	14 / 12	16 / 14	F1 rewards recall; 16 is the game's slot count, the sweep wanted 17 and was capped
  win-lift weight	1.0	0.25
  eff and kit weights	0.2	0	the hand-set stat table only hurt
  synergy weight	0.5	0.25	with pairs needing ≥ 200 matches (was 20)
  active-item bonus	0.1	0.3
  shrinkage prior	5 %	20 % of the top item's matches
  usage floor	3 %	12 %
  tier-1 minimum	3	4
  upgrade steps	4	6

  In plain terms the build is now "what Phantom+ players buy most on this hero, in the order they buy it", with win rate, synergy and actives as tie-breakers, and the stat table switched off. These numbers are fitted to the panel they are scored on. The landscape is flat near the optimum (several runs with different grids landed between 70.5 and 71.0 % mean with different mixes of the minor weights), so only the popularity weight, slot cap and item count are firm conclusions.

• Rejected on 2026-09-07: everything beyond the per-hero popularity. With the weights at their optimum (a simulated-annealing search from several restarts finds nothing coordinate descent missed), the misses are a small set of cross-hero late-game items (Extra Spirit, Improved Spirit, Superior Cooldown, Healbane…, tier-4 items are 53 % missed against 20 % for tier 1), which suggested new signals. Each was wired in and swept, and each lost: a cross-hero popularity term (mean 73.1 → 72.0–73.0 %, worst hero 62 → 59 %; the joint optimum with the hero popularity weight puts it at 0), a tier bias (73.1 → 57.5 % at the smallest weight), more items or looser slot caps (73.1 → 71.5–72.9 %), a co-occurrence term from the pair counts (72.2–72.9 % everywhere), and every alternative buy-order key. The reason is in Held-out Validation: an oracle built from the panel itself only reaches 75.7 %, so the metric, not the generator, is the limit. Two things were kept from the work: the ceiling script and the consensus metric, which read the same builds as 81.4 % against an 84.3 % ceiling.