Battlegrounds Meta Stats for Heroes

Introduction

Computing stats for all the heroes / tribes combinations in Battlegrounds is not easy. Because of the huge amount of possible combinations, relying on simply the data we have for a given specific combination means we won't have enough data have big enough samples to provide accurate information. So Firestone uses some tricks to try and mitigate this issue, and this page tries to explain what I'm doing.

The goal of this page is twofold:

• help you understand what the app does, so that you can trust its results
• let you criticize how things are done, especially if you have a strong statistics / data science background (which I don't have), so that I can improve the information displayed in the app :)

All tribes

When using the "all tribes" filter, things are straightforward: I consider all games, so building average positions, placement distribution or other stats simply means averaging / computing information across all the games I have.

❓ QUESTION: that said, the data is more or less reliable depending on how many games I have. I'd like to surface this information somehow (a bit like showing the "average placement" and "uncertainty" at the same time). If you can help me figure out how to compute this uncertainty based on the sample size, thank you :)

Using a specific combination of tribes

First, the app can't tell you "this is what the stats look like when X is banned". Instead, it will tell you what stats look like when X, Y and Z are present. The difference is subtle, but it can have an effect on how you read the stats.

Average placement

This is the easiest stat to explain, and I will use it as an example.
Raw stats for a given hero look like this (I've only kept the data related to average placement to make the data easier to read):

{
    "heroCardId": "TB_BaconShop_HERO_37",
    "dataPoints": 10788,
    "averagePosition": 4.353819058212829,
    "tribeStats": [
      {
        "tribe": 14,
        "dataPoints": 3485,
        "averagePosition": 4.343758967001435,
        "impactAveragePosition": -0.010060091211394173
      },
      ... (repeat for all the tribes)
    ],
    "mmrPercentile": 100,
    "timePeriod": "all-time"
  },

The interesting part here is the tribeStats node. It contains, for each tribe:

• how many games were recorded where that tribe was available
• what the average placement was for the hero across all those games
• it then deduces an "impact" value, which is the difference between the hero's global average position and their average position when the tribe is present. In the example above, it means roughly that "when Murlocs (tribe 14) are present, Lord Jaraxxus' final placement is on average 0.01 better".

This impact value is a way to try and weigh how each tribe influence how a hero is doing.

Once you're in the app, you can now select any number of tribes (typically the 5 that are in your current lobby), and we apply the impact of each tribe to the base averagePosition stat to build a final value.

Pros and cons

The obvious benefit is that this lets me split each tribe and study its impact independently from all the others. Which means that we still get a very good sample size for each tribe, which in turn means that we can actually propose some kind of impact study for a combination of tribes.

The downsides are:

• when studying a combination of tribes (eg a specific lobby), the information you get isn't as accurate. You can have synergies between tribes that aren't reflected with this method.
• it makes studying the impact of banned tribes more difficult. To do that, you would need to have "all the games where X isn't present", and apply a similar process. I don't know if this use case is interesting enough to warrant building another set of values for it. Let me know :)

Other metrics

They work in a similar fashion. If you are interested in having the details let me know, and I will add the info here. But the general philosophy is the same.

True rank

I've been looking for a way to integrate the sample size into the average position stat (because the fewer games you have for a hero, the less reliable the stat is).
By default, the app will now show you the "True Rank" of each hero, which is:

trueRank = averagePosition + 3 * standardError

where

standardError = standardDeviation / sqrt(sampleSize)

Conclusion

❓ What do you think? Is this a valid way to proceed? How would you improve it?
Don't hesitate to reach me on Discord for any question / suggestion (link is on the repo's README.md).