-
Notifications
You must be signed in to change notification settings - Fork 2
Usage
NOTE: See Simulation Runs as well. It may be easier to read.
These can help you get a quick, rough idea of the simulation's output for a single set of variables. Simply click "Run Simulation" and the simulation will be executed using the values you specified in the "Settings" menu. You can alter the "Sample Size" variable to alter the number of runs, this getting a more representative percentage of wins/losses/draws. Results are stored in "History"
NOTE: See Statistics as well. It may be easier to read.
Statistics can help you get a deeper understanding of how the simulation behaves with a specific set of variables. It calculates some additional things such as mean, standard deviation, confidence intervals, and hypothesis tests.
It uses the "alpha" value provided in settings for all confidence intervals and hypothesis tests. This means if alpha is 0.05, it will yield 95% confidence intervals and allow for a 5% chance of being incorrect for hypothesis tests.
Statistics runs the simulation for n="Trial Sample Size" * k="Trial Count" for a total of n * k simulation runs. It uses the Central Limit Theorem to create a probability distribution of the amount of wins, losses, and draws based on these runs.
"Test Type" specifies the type of hypothesis test you want to do, "Outcome To Test" specifies the outcome you want to test (win, loss, draw, ...), and "Value to test" is which value you want to test it against. So, if "Test Type" is "greater", "Outcome to Test" is "num_wins", and "value to test" is 0.50, we're going to run a hypothesis test that answers the question, "Can we say with (alpha * 100)% chance of being incorrect that num_wins > 50%?"
Results of statistics are also stored in history
NOTE: See Searching as well. It may be easier to read.
Searching is a method for studying the relationships between initial variables and simulation outcomes. It does this by generating Regression models after doing multiple simulation runs with different values for the variable you're trying to study, which models the relationship between simulation outcomes and the value of the variable you are studying.
"Target Percent" and "Outcome to Target" specify what percentage you're trying to solve for with the linear regression algorithm, and whether you're studying wins/losses/draws, respectively.
"Min value" and "Max value" specify the range of values you want to investigate for this searching run. If just left to defaults (recommended), searching will search the entire domain for the variable you're investigating.
"Steps" is the number of steps between the minimum and maximum values for the variable you're studying. For example, if you set "Steps" to 20, there will be 20 points of data that the regression model is built upon. Beware setting steps too high, as setting it too high can introduce variability into your results and make your regression model less accurate, by amplifying noise from variability. The default of 20 is good in most cases.
"Order" is what order Regression model you want to use. For example, if "Order"=3, it will use a model of the form x -> ax^3 + bx^2 + cx + d to model the relationship between the variable you're studying and the outcome you're targeting. Again, higher is not always better, as a higher Regression model can take noise into account and be less accurate. "Order"=3 seemed to be the sweet spot.
Searching finally finds the roots of the regression model to give you an idea for what value the attribute you're searching needs to be to reach the target percentage of the outcome you're targetting. Select which variable you want to investigate with "Attribute to search". Search results are also stored in "History"
Will run the simulation once utilizing the settings specified in the Settings menu. Will generate two .csv files in the project's root directory; sys_rec.csv and user_rec.csv, which shows the system record and user record between each run of the simulation.
This is useful for getting an idea of how the simulation calculated it's outcome, as it gives an extremely detailed account of the changes in most of the mutable internal simulation variables.
- Settings: Use this menu to change settings and initial values to run the simulation with
- History: Use this menu to see the history of your results, so you can easily reference them later. Stored in an SQLite database by default