These tools help you run Alpha/Beta tests on your equipment choices for multiplayer competitive games. In particular, it is good for checking the KDA, both over time and when examining your performance for certain loadouts.

What?

I play hunt showdown a lot. It's very fun. It's also insanely frustrating sometimes. The game has long matches, very frantic, quick battles, and a wide variey of meaningful character specialization and equipment options. It can take dozens of matches to determine if a loadout is worth it, and there are many loadouts, and a match takes an hour ... In short, it is very hard to get feedback on what equipment loadouts, tactics, or friends are useful.

For that, I keep a journal of matches, and am writing this tool to output some insights on the data gathered.

To use it, you will have to write down match information. But matches last an hour, so that's not much overhead.

Then, you'll have to use the tools this package provides:

• kda-summary will summarize your K, D, and A values (and the usual KDA metric) over the entire journal.
• kda-compare will run will look at the whole dataset and tell you if you're doing significantly differently with different loadouts.
• kda-explore will allow you to look at the conditional distributions of anything with and without anything else.

Get for debian / WSL

If you have cargo (apt install cargo), then cargo install jodavaho/kda-tools.

Otherwise, just grab one of the test debs in releases/ For example 1.3.0

Then, in wsl,

sudo dpkg -i kda-tools_1.3.0_amd64.deb

You can use the example above as-is.

Basics

Keep a match journal like this (fyi this is key value count format ).

<date> [<items or friends initials>] [K|D|A|B]

For example, my Hunt diary looks a little like:

2021-03-12 BAR+Scope pistol K K B alone
2021-03-12 BAR+Scope pistol K D D jb
2021-03-12 Short-Rifle Short-Shotgun K D jb
2021-03-12 BAR+Scope pistol D jp+jb
2021-03-13 BAR+Scope pistol jp D
2021-03-13 BAR+Scope pistol jp B D D A A
2021-03-13 Shotgun pistol jp D
2021-03-13 BAR+Scope pistol jp K
2021-03-14 Short-Rifle akimbo  alone
2021-03-17 LAR Sil pistol  alone
2021-03-17 pistol-stock akimbo  alone
2021-03-17 Short-Shotgun pistol-stock  alone

Whereas my usual wingman JP has one like:

2021-04-19 Martini-Henri_IC1 Caldwell_Conversion_Chain JV B B K
2021-04-19 Martini-Henri_IC1_Deadeye Caldwell_Conversion_Chain JV JB
2021-04-19 Martini-Henri_IC1_Deadeye Caldwell_Conversion_Chain JV JB B B
2021-04-19 Vetterli_71_Karabiner Bornheim_No_3 JV JB B B
2021-04-19 Vetterli_71_Karabiner Caldwell_Conversion JV JB D
2021-05-02 Martini-Henri_IC1 Caldwell_Conversion_Chain JV JB A
2021-05-02 Lebel_1886_Marksman Caldwell_Conversion_Chain JV JB K K
2021-05-02 Lebel_1886_Marksman Caldwell_Conversion_Chain JV JB K K
2021-05-02 Martini-Henri_IC1_Deadeye Caldwell_Conversion_Chain JV JB B B B B K K K

He's much better than I am. You can use whatever you want to denote loadouts or friends ... it'll just run multi-variate hypotheis tests on all of them with the important parts: K D A or B, for example...

Contents of examples/journal.txt:

2021-01-03 K K Sniper
2021-01-03 K D Shotgun
2021-01-04 K K JP Sniper
2021-01-04 K D B Shotgun JB
2021-01-04 K D B Sniper JB

is 5 matches:

1. two kills with a sniper loadout
2. a kill a death with a shotgun loadout
3. two kills with a sniper loadout and team-mate "JP"
4. a kill, a death, a bounty with Shotguns and team-mate "JB"
5. Same, but with Sniper loadout

Automated: KDA-Summary

Let's see the summary over time:

$ <journal.txt kda-summary
    n       Date   K   D   A   B   KDA    sK    sD    sA    sB  mKDA    mK    mD    mA    mB
    1 2021-01-03   2   0   0   0  2.00     2     0     0     0  2.00  2.00  0.00  0.00  0.00
    2 2021-01-03   1   1   0   0  1.00     3     1     0     0  3.00  1.50  0.50  0.00  0.00
    3 2021-01-04   2   0   0   0  2.00     5     1     0     0  5.00  1.67  0.33  0.00  0.00
    4 2021-01-04   1   1   0   1  1.00     6     2     0     1  3.00  1.50  0.50  0.00  0.25
    5 2021-01-04   1   1   0   1  1.00     7     3     0     2  2.33  1.40  0.60  0.00  0.40

Not bad. Notice, kda-summary requires the use of tags K for kills, D for death, B for bounties, and A for assist. It outputs your per-match stats (K, D, A, B columns), the KDA value of (K+A)/D for that match, the running sum of the K, D, A and B, (the next 4 columns) and the mean (avg / match) of KDA, K, D, A, and B over time (the last 5 columns). You can stop here and pull this data into another program if you'd like.

Note the Date field. If you put dates of the form YYYY-MM-DD somewhere per line in the journal, it will populate that field. See example above or kvc. File bug reports there if you don't like how the dates are required to be formatted.

Semi-Automated: KDA-compare

So, what's the better loadout or partner? KDA-compare does some testing for you.

$ <journal.txt kda-compare
Processed. Read: 5 rows and 8 variables

[====================================================] 100.00 % 2696.14/s
met    grp      n/d      val   N     n/d      ~val  M     p
kda    Sniper   5/1      5.00   3    2/2      1.00   2    0.06
kda    JP       2/0      inf    1    5/3      1.67   4    0.49
kda    Shotgun  2/2      1.00   2    5/1      5.00   3    0.77
kda    JB       2/2      1.00   2    5/1      5.00   3    0.78
b/d    Sniper   1/1      1.00   3    1/2      0.50   2    0.24
b/d    Shotgun  1/2      0.50   2    1/1      1.00   3    0.69
b/d    JB       2/2      1.00   2    0/1      0.00   3    NaN
b/d    JP       0/0      NaN    1    2/3      0.67   4    NaN

Let's look. The first row is met grp ... These are

• the metric name (e.g., kda or bounties / death b/d)
• item group (grp)
• value counts (n)
• deaths (d)
• the value of the metric 'val' with the grp
• number of matches where 'grp' was used (N)
• value counts without the grp (n)
• death counts without the grp (d)
• the value of the metric 'val' without the grp
• number of matches without the grp (M)
• and the probability that we'd randomly see that 'val' given the distribution of the metric without the grp.

You can see a pvp metric (kda or (kills + assists)/deaths ), and pve metric (bounties/death), for each item.

That last one, p, is usually called a p-value, and if it's low, you have a signfiicantly better set of rounds with the grp than without it.

In the data above, it appears that rounds where I use the Sniper weapon are significantly better than rounds where I don't, given p is small.

Note, there are some NaN's because when playing with JP I got no bounties or deaths (0/0), which is not a meaningful result to compare against. Less obviously, when playing with JB, I also cannot get a p-value, since in the rounds that I did not play with JB , I never got a bounty. This means there's no meaningful representation for the baseline (without JB) case.

OK, so what? well, draw your own conclusions and try to mix up your loadouts. If you only play snipers wtih your friend JP, and only play shotguns with your friend JB, they will be highly correlated and it may be hard to see if JB or Shotgun makes the most difference.

There is an option -i to ignore certain items. This is useful if you want to see the weapons only.

let's ignore my wingmen JP and JB to check just weapons.

$ <journal.txt kda-compare -i JP JB
Processed. Read: 5 rows and 8 variables

[====================================================] 100.00 % 3450.51/s
met    grp      n/d      val   N     n/d      ~val  M     p
kda    Sniper   5/1      5.00   3    2/2      1.00   2    0.08
kda    Shotgun  2/2      1.00   2    5/1      5.00   3    0.77
b/d    Sniper   1/1      1.00   3    1/2      0.50   2    0.24
b/d    Shotgun  1/2      0.50   2    1/1      1.00   3    0.69

Now we see a slight decrease in Sniper for kda, but no change in b/d.

You can also test pairings (group-size = 2)

$ <journal.txt kda-compare --group-size 2
Processed. Read: 5 rows and 8 variables

[==================================================] 100.00 % 8175.42/s
met    grp         n/d      val   N     n/d      ~val  M     p
kda    JP+Sniper   2/0      inf    1    5/3      1.67   4    0.47
kda    JB+Sniper   1/1      1.00   1    6/2      3.00   4    0.78
kda    JB+Shotgun  1/1      1.00   1    6/2      3.00   4    0.79
b/d    JB+Shotgun  1/1      1.00   1    1/2      0.50   4    0.62
b/d    JB+Sniper   1/1      1.00   1    1/2      0.50   4    0.64
b/d    JP+Sniper   0/0      NaN    1    2/3      0.67   4    NaN

Now we see that there isn't really a difference across friend / weapon pairings. Note the inf result. You may think that infinites are not possible to analyze. Well, since we're using bootstrap methods (see poisson-rate-test), you can actually get meaningful probabilities, and therefore meaningful p-values.

To confirm that friends are a nuisance variable (no statistically significant difference), try ignoring weapons:

$ <journal.txt kda-compare -i Sniper Shotgun
Processed. Read: 5 rows and 8 variables

[====================================================] 100.00 % 3948.08/s
met    grp n/d      val   N     n/d      ~val  M     p
kda    JP  2/0      inf    1    5/3      1.67   4    0.49
kda    JB  2/2      1.00   2    5/1      5.00   3    0.77
b/d    JB  2/2      1.00   2    0/1      0.00   3    NaN
b/d    JP  0/0      NaN    1    2/3      0.67   4    NaN

here we definitely see that the choice of wingman has much less effect than the choice of weapon (compare values from two examples ago)

use

$ kda-compare -h
It *expects* input in kvc format (one match per line), and processs the variables K, D, and A, as a function of *all
other* variables present. It ignores kvc keywords / fields (like dates), but you'll have to specify other things to
ignore manually.


USAGE:
    kda-compare [FLAGS] [OPTIONS]

FLAGS:
    -f               Speed up computation by doing a fewer number of iterations. Helpful for quick looks but the
                     ordering of some sets may change across multiple invocations
    -h, --help       Prints help information
    -n               Display notes about particular test cases in the output
    -V, --version    Prints version information

OPTIONS:
    -g, --group-size <group_size>    Instead of individual items (group_size==1), rank by enumerated groupings that
                                     appear in data of a given size. [default: 1]  [possible values: 1, 2, 3, 4]
    -i <ignore>...                   List of fields to ignore (if they appear in data). You can ignoring fields A B and
                                     C as '-i A,B,C' or '-i A -i B -i C' but not '-i A B C' or '-i A, B, C'. That's
                                     because of shell magic, not becuase of the way it was implemented
    -o <out_format>                  Output format which can be one of Vnlog or Whitespace-,  Tab-, or Comma-seperated.
                                     [default: wsv]  [possible values: wsv, tsv, csv, vnl]

Manual: KDA-Explore

To really dig in, you can evaluate the conditional distribtuions of any variable conditioned on the occurance of any other variable. This is the role of kda-explore.

What I mean is that kda-explore the semantics of 'K' vs 'k' vs "kill" is irrelevant. We explore the data by asking it to analyze variables by name. For example, in the data above, to see kills "K" per match with Sniper and without, you form the "experiment" denoted as "K:Sniper" and ask kda-explore to run that experiment by kda-explore "K : Sniper".

$ < journal.txt kda-explore K:Sniper
Processed. Read: 5 rows and 8 variables
Varibables found: Date Sniper K D Shotgun JP B JB
Debug: processing: K:Sniper
met    grp     n     M     rate  ~n    ~M    ~rate p     notes
K      Sniper  5     3     1.67  2     2     1.00  0.37

This means the probability of a kill given you have a sniper is higher, but not so high that it might not be much different over time (p .37). We're only checking kills here. What about deaths?

$ < journal.txt kda-explore D:Sniper
Processed. Read: 5 rows and 8 variables
Varibables found:
Date Sniper K Shotgun D JP B JB
Debug: processing: D:Sniper
met    grp     n     M     rate  ~n    ~M    ~rate p     notes
D      Sniper  1     3     0.33  2     2     1.00  0.19

You can run many experiments seperated by 'vs' (this may change), against many output varaibles ... All are valid:

• kda-explore "K D : Sniper vs Shotgun" to see kills and deaths compared to sniper and shotguns
• kda-explore "D : K" to see if you die more when you kill stuff or not
• kda-explore "Sniper: JB" to see if you play sniper more or less when you're with JB
• kda-explore K:all to see kill spreads and sorted rate comparisons for all variables

and so on ... each "tag" (item on a line in a journal) is a valid input or output depending on your determination of experiments.

Use

$ kda-explore -h
USAGE:
    kda-explore [OPTIONS] <command>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

OPTIONS:
    -o <out_format>        Output format which can be one of Vnlog or Whitespace-,  Tab-, or Comma-seperated. [default:
                           wsv]  [possible values: wsv, tsv, csv, vnl]

ARGS:
    <command>    The A/B comparison to run, of the form '<some variables : <other variables>'. e.g., 'K: pistol'
                 will check kills with and wtihout pitols [default: K D A : all]

One way to interpret this is "This doesn't make sense". That's true, it's primitive still, and mostly a toy for my own use.

Todo

• document match journal format better. see: github.com/jodavaho/kvc.git
• improve match journal to allow :count. (see kvc again)
• provide linter for match journal
• tool to create / factor matricies in format ammenable to third-party analysis (e.g., R)
• Perform power tests / experiment design
• remove '-c' as mandatory switch ... obsolete when baseline '_' was removed
• Provide a library version in C,
  • C++,
  • Rust

Known issues

• If you have an item you use every game, then you have insufficient data. for every test of the form k:A there must be at least one match without A occuring. It's ok if there's no kills (k).