-
Notifications
You must be signed in to change notification settings - Fork 2
Statistics
Statistics can help you get a deeper understanding of how the simulation behaves with a specific set of variables. It calculates some additional things such as mean, standard deviation, confidence intervals, and hypothesis tests.
It uses the "alpha" value provided in settings for all confidence intervals and hypothesis tests. This means if alpha is 0.05, it will yield 95% confidence intervals and allow for a 5% chance of being incorrect for hypothesis tests.
Statistics runs the simulation for n="Trial Sample Size" * k="Trial Count" for a total of n * k simulation runs. It uses the Central Limit Theorem to create a probability distribution of the amount of wins, losses, and draws based on these runs.
"Test Type" specifies the type of hypothesis test you want to do, "Outcome To Test" specifies the outcome you want to test (win, loss, draw, ...), and "Value to test" is which value you want to test it against. So, if "Test Type" is "greater", "Outcome to Test" is "num_wins", and "value to test" is 0.50, we're going to run a hypothesis test that answers the question, "Can we say with (alpha * 100)% chance of being incorrect that num_wins > 50%?"
Results of statistics are also stored in history
All inputs specific to statistics can be found in the settings menu. They are as follows:
Trial Sample Size: The sample size to use for each trial. We'll be doing multiple trials. (Recommended: 100)
Trial Count: How many trials we will be doing. Do not go below 30 or your results will not be accurate. (Recommended: >=30)
Alpha: This value is used for both confidence intervals AND hypothesis testing. So, if alpha=0.05, you will see 95% confidence intervals, and hypothesis tests will have a margin of error of 5%.
Test Type: Here, you can specify what type of hypothesis test you want to perform. You can either perform a Two Tailed, Greater, or Less than test.
Outcome to test: This is the actual outcome (wins/losses/draws) that you want to test with your hypothesis test.
Value to test: This is the value you want to test against in the hypothesis test.
Gives you statistics for every variable that is detailed in regular Simulation Runs. This includes the mean, standard deviation, and confidence intervals (using the alpha you selected in settings).
At the bottom, the hypothesis tests section can be seen. It will tell you the p-value, and it will reject the null hypothesis only if the p-value is less than the alpha you selected.