From ad0705fe262fca27cd2180791f48a75a64ae612f Mon Sep 17 00:00:00 2001 From: lrennels Date: Wed, 4 Nov 2020 23:26:15 -0800 Subject: [PATCH 1/6] Add defsim clarifications to docs --- docs/src/howto/howto_3.md | 8 +++++--- docs/src/tutorials/tutorial_5.md | 25 ++++++++++++++++--------- 2 files changed, 21 insertions(+), 12 deletions(-) diff --git a/docs/src/howto/howto_3.md b/docs/src/howto/howto_3.md index ecc6180e2..f68b533ab 100644 --- a/docs/src/howto/howto_3.md +++ b/docs/src/howto/howto_3.md @@ -73,13 +73,15 @@ In addition to the distributions available in the `Distributions` package, Mimi ### Apply RVs to model parameters +**For all applications in this section, it is important to note that for each trial, a random variable on the right hand side of an assignment will take on the value of a *single* draw from the given distribution. This means that even if the random variable is applied to more than one parameter on the right hand side (such as assigning to a slice), each of these parameters will be assigned the same value, not different draws from the distribution.** + The macro next defines how to apply the values generated by each RV to model parameters based on a pseudo-assignment operator: - `param = RV` replaces the values in the parameter with the value of the RV for the current trial. -- `param += RV` replaces the values in the parameter with the sum of the original value and the value of the RV for the current trial. -- `param *= RV` replaces the values in the parameter with the product of the original value and the value of the RV for the current trial. +- `param += RV` replaces the values in the parameter with the sum of the previous trial's value and the value of the RV for the current trial. +- `param *= RV` replaces the values in the parameter with the product of the previous trial's value and the value of the RV for the current trial. -Furthermore, in `@defsim`, you can apply distributions to specific slices of array parameters, and you can "bulk assign" distributions to elements of a vector or matrix using a more condensed syntax. +As described below, in `@defsim`, you can apply distributions to specific slices of array parameters, and you can "bulk assign" distributions to elements of a vector or matrix using a more condensed syntax. #### Apply RVs to model parameters: Assigning to array slices diff --git a/docs/src/tutorials/tutorial_5.md b/docs/src/tutorials/tutorial_5.md index caeef333f..8f101a1d1 100644 --- a/docs/src/tutorials/tutorial_5.md +++ b/docs/src/tutorials/tutorial_5.md @@ -2,6 +2,8 @@ This tutorial walks through the Monte Carlo simulation and sensitivity analysis (SA) functionality of Mimi, including core routines and examples. We will start with looking at using the Monte Carlo and SA routines with the multi-region Mimi model built in the second half of Tutorial 3, which is also available in the Mimi repository at `examples/tutorial/02-multi-region-model`. Then we will show some more advanced features using a real Integrated Assessment model, [MimiDICE2010](https://github.com/anthofflab/MimiDICE2010.jl). +**For a more complete understanding of the Monte Carlo and SA Support, we recommend following up by reading How-to Guide 3: Conduct Monte Carlo Simulations and Sensitivity Analysis.** + Working through the following tutorial will require: - [Julia v1.4.0](https://julialang.org/downloads/) or higher @@ -168,7 +170,8 @@ Mimi.Model Built: false ``` -#### Step 2. Define Random Variables +#### Step 2. Define the Simulation + The `@defsim` macro is the first step in the process, and returns a `SimulationDef`. The following syntax allows users to define random variables (RVs) as distributions, and associate model parameters with the defined random variables. There are two ways of assigning random variables to model parameters in the `@defsim` macro. Notice that both of the following syntaxes are used in the following example. @@ -179,19 +182,20 @@ rv(rv1) = Normal(0, 0.8) # create a random variable called "rv1" with the spe param1 = rv1 # then assign this random variable "rv1" to the parameter "param1" in the model ``` -The second is a shortcut, in which you can directly assign the distribution on the right-hand side to the name of the model parameter on the left hand side. With this syntax, a random variable is created under the hood and then assigned to `param1`. +The second is a shortcut, in which you can directly assign the distribution on the right-hand side to the name of the model parameter on the left hand side. With this syntax, a single random variable is created under the hood and then assigned to `param1`. ```julia param1 = Normal(0, 0.8) ``` +**It is important to note** that for each trial, a random variable on the right hand side of an assignment, be it using an explicitly defined random variable with `rv(rv1)` syntax or using shortcut syntax as above, will take on the value of a **single** draw from the given distribution. This means that even if the random variable is applied to more than one parameter on the right hand side (such as assigning to a slice), each of these parameters will be assigned the same value, not different draws from the distribution -The `@defsim` macro also selects the sampling method. Simple random sampling (also called Monte Carlo sampling) is the default. -Other options include Latin Hypercube sampling and Sobol sampling. +The `@defsim` macro also selects the sampling method. Simple random sampling (also called Monte Carlo sampling) is the default. Other options include Latin Hypercube sampling and Sobol sampling. Below we show just one example of a `@defsim` call, but the How-to guide referenced at the beginning of this tutorial gives a more comprehensive overview of the options. ```jldoctest tutorial5; output = false, filter = r".*"s using Mimi using Distributions sd = @defsim begin + # Define random variables. The rv() is only required when defining correlations # or sharing an RV across parameters. Otherwise, you can use the shortcut syntax # to assign a distribution to a parameter name. @@ -202,14 +206,17 @@ sd = @defsim begin # If using LHS, you can define correlations like this: sampling(LHSData, corrlist=[(:name1, :name2, 0.7), (:name1, :name3, 0.5)]) - # Exclude the sampling() call, or use the following for simple random sampling: - # sampling(MCSData) - - # For Sobol sampling, specify N, and calc_second_order, which defaults to true. - # sampling(SobolData, N=10000, calc_second_order=true) + # Exclude the sampling() call, since it will default to MCSData, or use the + # following for simple random sampling: + sampling(MCSData) # assign RVs to model Parameters share = Uniform(0.2, 0.8) + + # you can use the *= operator to replace the values in the parameter with the + # product of the previous trial's value and the value of the RV for the current + # trial (note that in both lines below, all indexed values will be mulitplied by the + # same draw from the given random parameter (name2 or Uniform(0.8, 1.2)) sigma[:, Region1] *= name2 sigma[2020:5:2050, (Region2, Region3)] *= Uniform(0.8, 1.2) From a8cb57f0a56f5d14586859a426f456e7c1c69fe4 Mon Sep 17 00:00:00 2001 From: lrennels Date: Thu, 5 Nov 2020 10:16:01 -0800 Subject: [PATCH 2/6] Fix sampling call error --- docs/src/tutorials/tutorial_5.md | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/docs/src/tutorials/tutorial_5.md b/docs/src/tutorials/tutorial_5.md index 8f101a1d1..7cbebff0f 100644 --- a/docs/src/tutorials/tutorial_5.md +++ b/docs/src/tutorials/tutorial_5.md @@ -203,13 +203,10 @@ sd = @defsim begin rv(name2) = Uniform(0.75, 1.25) rv(name3) = LogNormal(20, 4) - # If using LHS, you can define correlations like this: + # Define the sampling strategy, and if you are using LHS, you can define + # correlations like this: sampling(LHSData, corrlist=[(:name1, :name2, 0.7), (:name1, :name3, 0.5)]) - # Exclude the sampling() call, since it will default to MCSData, or use the - # following for simple random sampling: - sampling(MCSData) - # assign RVs to model Parameters share = Uniform(0.2, 0.8) From cd32434a5ff8d31e118401593b6452a49d85bdfe Mon Sep 17 00:00:00 2001 From: lrennels Date: Mon, 9 Nov 2020 19:49:04 -0800 Subject: [PATCH 3/6] Clarify docs on update_timesteps keyword argument --- docs/src/tutorials/tutorial_3.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/src/tutorials/tutorial_3.md b/docs/src/tutorials/tutorial_3.md index 18d832de0..4f66e32d1 100644 --- a/docs/src/tutorials/tutorial_3.md +++ b/docs/src/tutorials/tutorial_3.md @@ -108,7 +108,9 @@ set_dimension!(m, :time, years) ``` -Next, create a dictionary `params` with one entry `(k, v)` per external parameter by name `k` to value `v`. Each key `k` must be a symbol or convert to a symbol matching the name of an external parameter that already exists in the model definition. Part of this dictionary may look like: +Now that you have changed the time dimension, you have a mismatch between the time labels attached to your parameters and the time lables used by the model. Thus, **you must update at least all parameters with a `:time`** dimension and use the special `update_timesteps=true` flag to get the time labels on the parameters to match those in the model. This is required even in cases where you do not want to change the parameter values themselves (see the forum question [here](https://forum.mimiframework.org/t/update-time-index/134/5)) for an in-depth explanation of this case. You may of course also update parameters without a `:time` dimension as desired. + +Create a dictionary `params` with one entry `(k, v)` per external parameter by name `k` to value `v`. Each key `k` must be a symbol or convert to a symbol matching the name of an external parameter that already exists in the model definition. Part of this dictionary may look like: ```julia params = Dict{Any, Any}() @@ -126,7 +128,7 @@ update_params!(m, params, update_timesteps=true) run(m) ``` -Note that here we use the `update_timesteps` flag and set it to `true`, because since we have changed the time index we want the time labels on the parameters to change, not simply their values. +Note again that here we use the `update_timesteps` flag and set it to `true`, because since we have changed the time index we want the time labels on the parameters to change, not simply their values. ## Component and Structural Modifications: The API From 86611fca6c4a31f32209fb363b8a1df5957592f8 Mon Sep 17 00:00:00 2001 From: lrennels Date: Mon, 9 Nov 2020 20:35:20 -0800 Subject: [PATCH 4/6] update_timesteps docs --- docs/src/tutorials/tutorial_3.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/src/tutorials/tutorial_3.md b/docs/src/tutorials/tutorial_3.md index 4f66e32d1..7fb8ac697 100644 --- a/docs/src/tutorials/tutorial_3.md +++ b/docs/src/tutorials/tutorial_3.md @@ -108,7 +108,7 @@ set_dimension!(m, :time, years) ``` -Now that you have changed the time dimension, you have a mismatch between the time labels attached to your parameters and the time lables used by the model. Thus, **you must update at least all parameters with a `:time`** dimension and use the special `update_timesteps=true` flag to get the time labels on the parameters to match those in the model. This is required even in cases where you do not want to change the parameter values themselves (see the forum question [here](https://forum.mimiframework.org/t/update-time-index/134/5)) for an in-depth explanation of this case. You may of course also update parameters without a `:time` dimension as desired. +Now that you have changed the time dimension, you have a mismatch between the time labels attached to your parameters and the time lables used by the model. Thus, **you must update at least all parameters with a `:time`** dimension and use the explicit `update_timesteps=true` flag to get the time labels on the parameters to match those in the model. This is required even in cases where you do not want to change the parameter values themselves (see the forum question [here](https://forum.mimiframework.org/t/update-time-index/134/5)) for an in-depth explanation of this case. You may of course also update parameters without a `:time` dimension as desired. Create a dictionary `params` with one entry `(k, v)` per external parameter by name `k` to value `v`. Each key `k` must be a symbol or convert to a symbol matching the name of an external parameter that already exists in the model definition. Part of this dictionary may look like: From cd6888d7484af5d17713b41c22b72fb5c87c403f Mon Sep 17 00:00:00 2001 From: lrennels Date: Tue, 10 Nov 2020 14:36:04 -0800 Subject: [PATCH 5/6] Minor docs fixes --- docs/src/howto/howto_3.md | 2 +- docs/src/tutorials/tutorial_3.md | 2 +- docs/src/tutorials/tutorial_5.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/src/howto/howto_3.md b/docs/src/howto/howto_3.md index f68b533ab..f5ae223ff 100644 --- a/docs/src/howto/howto_3.md +++ b/docs/src/howto/howto_3.md @@ -73,7 +73,7 @@ In addition to the distributions available in the `Distributions` package, Mimi ### Apply RVs to model parameters -**For all applications in this section, it is important to note that for each trial, a random variable on the right hand side of an assignment will take on the value of a *single* draw from the given distribution. This means that even if the random variable is applied to more than one parameter on the right hand side (such as assigning to a slice), each of these parameters will be assigned the same value, not different draws from the distribution.** +**For all applications in this section, it is important to note that for each trial, a random variable on the right hand side of an assignment will take on the value of a *single* draw from the given distribution. This means that even if the random variable is applied to more than one parameter on the left hand side (such as assigning to a slice), each of these parameters will be assigned the same value, not different draws from the distribution.** The macro next defines how to apply the values generated by each RV to model parameters based on a pseudo-assignment operator: diff --git a/docs/src/tutorials/tutorial_3.md b/docs/src/tutorials/tutorial_3.md index 7fb8ac697..7c58cbbe9 100644 --- a/docs/src/tutorials/tutorial_3.md +++ b/docs/src/tutorials/tutorial_3.md @@ -108,7 +108,7 @@ set_dimension!(m, :time, years) ``` -Now that you have changed the time dimension, you have a mismatch between the time labels attached to your parameters and the time lables used by the model. Thus, **you must update at least all parameters with a `:time`** dimension and use the explicit `update_timesteps=true` flag to get the time labels on the parameters to match those in the model. This is required even in cases where you do not want to change the parameter values themselves (see the forum question [here](https://forum.mimiframework.org/t/update-time-index/134/5)) for an in-depth explanation of this case. You may of course also update parameters without a `:time` dimension as desired. +Now that you have changed the time dimension, you have a mismatch between the time labels attached to your parameters and the time labels used by the model. Thus, **you must update at least all parameters with a `:time`** dimension and use the explicit `update_timesteps=true` flag to get the time labels on the parameters to match those in the model. This is required even in cases where you do not want to change the parameter values themselves (see the forum question [here](https://forum.mimiframework.org/t/update-time-index/134/5)) for an in-depth explanation of this case. You may of course also update parameters without a `:time` dimension as desired. Create a dictionary `params` with one entry `(k, v)` per external parameter by name `k` to value `v`. Each key `k` must be a symbol or convert to a symbol matching the name of an external parameter that already exists in the model definition. Part of this dictionary may look like: diff --git a/docs/src/tutorials/tutorial_5.md b/docs/src/tutorials/tutorial_5.md index 7cbebff0f..214b193d1 100644 --- a/docs/src/tutorials/tutorial_5.md +++ b/docs/src/tutorials/tutorial_5.md @@ -186,7 +186,7 @@ The second is a shortcut, in which you can directly assign the distribution on t ```julia param1 = Normal(0, 0.8) ``` -**It is important to note** that for each trial, a random variable on the right hand side of an assignment, be it using an explicitly defined random variable with `rv(rv1)` syntax or using shortcut syntax as above, will take on the value of a **single** draw from the given distribution. This means that even if the random variable is applied to more than one parameter on the right hand side (such as assigning to a slice), each of these parameters will be assigned the same value, not different draws from the distribution +**It is important to note** that for each trial, a random variable on the right hand side of an assignment, be it using an explicitly defined random variable with `rv(rv1)` syntax or using shortcut syntax as above, will take on the value of a **single** draw from the given distribution. This means that even if the random variable is applied to more than one parameter on the left hand side (such as assigning to a slice), each of these parameters will be assigned the same value, not different draws from the distribution The `@defsim` macro also selects the sampling method. Simple random sampling (also called Monte Carlo sampling) is the default. Other options include Latin Hypercube sampling and Sobol sampling. Below we show just one example of a `@defsim` call, but the How-to guide referenced at the beginning of this tutorial gives a more comprehensive overview of the options. From 529f7ed16d37b2f0fc604ceb14909981b844ecbf Mon Sep 17 00:00:00 2001 From: lrennels Date: Tue, 10 Nov 2020 15:12:17 -0800 Subject: [PATCH 6/6] Revert docs changes for *= and += --- docs/src/howto/howto_3.md | 4 ++-- docs/src/tutorials/tutorial_5.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/src/howto/howto_3.md b/docs/src/howto/howto_3.md index f5ae223ff..3a7daa7fe 100644 --- a/docs/src/howto/howto_3.md +++ b/docs/src/howto/howto_3.md @@ -78,8 +78,8 @@ In addition to the distributions available in the `Distributions` package, Mimi The macro next defines how to apply the values generated by each RV to model parameters based on a pseudo-assignment operator: - `param = RV` replaces the values in the parameter with the value of the RV for the current trial. -- `param += RV` replaces the values in the parameter with the sum of the previous trial's value and the value of the RV for the current trial. -- `param *= RV` replaces the values in the parameter with the product of the previous trial's value and the value of the RV for the current trial. +- `param += RV` replaces the values in the parameter with the sum of the original value and the value of the RV for the current trial. +- `param *= RV` replaces the values in the parameter with the product of the original value and the value of the RV for the current trial. As described below, in `@defsim`, you can apply distributions to specific slices of array parameters, and you can "bulk assign" distributions to elements of a vector or matrix using a more condensed syntax. diff --git a/docs/src/tutorials/tutorial_5.md b/docs/src/tutorials/tutorial_5.md index 214b193d1..258270a39 100644 --- a/docs/src/tutorials/tutorial_5.md +++ b/docs/src/tutorials/tutorial_5.md @@ -211,7 +211,7 @@ sd = @defsim begin share = Uniform(0.2, 0.8) # you can use the *= operator to replace the values in the parameter with the - # product of the previous trial's value and the value of the RV for the current + # product of the original value and the value of the RV for the current # trial (note that in both lines below, all indexed values will be mulitplied by the # same draw from the given random parameter (name2 or Uniform(0.8, 1.2)) sigma[:, Region1] *= name2