level() can be used to create more complicated patterns of nesting. For example, when creating lower level data, it is possible to use a different N for each of the values of the higher level data:
add_level() can be used to create more complicated patterns of nesting. For example, when creating lower level data, it is possible to use a different N for each of the values of the higher level data:
variable_data <-
fabricate(
- cities = level(N = 2, elevation = runif(n = N, min = 1000, max = 2000)),
- citizens = level(N = c(2, 4), age = runif(N, 18, 70))
+ cities = add_level(N = 2, elevation = runif(n = N, min = 1000, max = 2000)),
+ citizens = add_level(N = c(2, 4), age = runif(N, 18, 70))
)
variable_dataHere, each city has a different number of citizens. And the value of N used to create the age variable automatically updates as needed. The result is a dataset with 6 citizens, 2 in the first city and 4 in the second. As long as N is either a number, or a vector of the same length of the current lowest level of the data, level() will know what to do.
Here, each city has a different number of citizens. And the value of N used to create the age variable automatically updates as needed. The result is a dataset with 6 citizens, 2 in the first city and 4 in the second. As long as N is either a number, or a vector of the same length of the current lowest level of the data, add_level() will know what to do.
It is also possible to provide a function to N, enabling a random number of citizens per city:
my_data <-
fabricate(
- cities = level(N = 2, elevation = runif(n = N, min = 1000, max = 2000)),
- citizens = level(N = sample(1:6, size = 2, replace = TRUE),
- age = runif(N, 18, 70))
+ cities = add_level(N = 2, elevation = runif(n = N, min = 1000, max = 2000)),
+ citizens = add_level(N = sample(1:6, size = 2, replace = TRUE),
+ age = runif(N, 18, 70))
)
my_data| cities | population | citizens | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 1 | 90 | 001 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1.1 | 1 | 90 | 002 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1.2 | 1 | 90 | 003 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1.3 | 1 | 90 | 004 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1.4 | 1 | 90 | 005 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1.5 | 1 | 90 | 006 | @@ -275,10 +268,10 @@
my_data <-
-data_frame(Y = sample(1:10, 2)) %>%
- fabricate(lower_level = level(N = 3, Y2 = Y + rnorm(N)))
+data_frame(Y = sample(1:10, 2)) %>%
+ fabricate(lower_level = add_level(N = 3, Y2 = Y + rnorm(N)))
my_data| depth | mag | stations | +ID | fatalities | insurance_cost | 562 | 4.8 | 41 | +0001 | 611 | 1.0e+09 | @@ -191,6 +193,7 @@650 | 4.2 | 15 | +0002 | 290 | 3.0e+08 | @@ -200,6 +203,7 @@42 | 5.4 | 43 | +0003 | 427 | 7.8e+08 | @@ -209,6 +213,7 @@626 | 4.1 | 19 | +0004 | 470 | 5.2e+08 | @@ -218,6 +223,7 @@649 | 4.0 | 11 | +0005 | 456 | 7.2e+08 | @@ -227,6 +233,7 @@195 | 4.0 | 12 | +0006 | 390 | 7.0e+08 | @@ -240,10 +247,10 @@
|---|
Several things can be observed in this example. First, fabricate knows that your second level() command will be nested under the first level of data. Each level gets its own ID variable, in addition to the variables you create. Second, the meaning of the variable “N” changes. During the level() call for countries, N is 5. During the level() call for provinces, N is 10. And the resulting data, of course, has 50 observations.
Several things can be observed in this example. First, fabricate knows that your second add_level() command will be nested under the first level of data. Each level gets its own ID variable, in addition to the variables you create. Second, the meaning of the variable “N” changes. During the add_level() call for countries, N is 5. During the add_level() call for provinces, N is 10. And the resulting data, of course, has 50 observations.
Finally, the province-level variables are created using the draw_discrete() function. This is a function provided by fabricatr to make simulating discrete random variables simple. When you simulate your own data, you can use fabricatr’s functions, R’s built-ins, or any custom functions you wish. draw_discrete() is explained in our tutorial on variable generation using fabricatr
citizen_data <-
fabricate(
data = country_data,
- citizens = level(N=10,
- salary = rnorm(N,
- mean = gdp_per_capita +
- has_nat_resources * 5000 +
- has_manufacturing * 5000,
+ citizens = add_level(N=10,
+ salary = rnorm(N,
+ mean = gdp_per_capita +
+ has_nat_resources * 5000 +
+ has_manufacturing * 5000,
sd = 10000)))
head(citizen_data)| countries | +gdp_per_capita | +life_expectancy | provinces | has_nat_resources | has_manufacturing | -gdp_per_capita | -life_expectancy | avg_temp |
|---|---|---|---|---|---|---|---|---|
| 1 | +39,398 | +73 | 01 | 1 | 1 | -39,398 | -73 | 33 |
| 1 | +39,398 | +73 | 02 | 1 | 1 | -39,398 | -73 | 33 |
| 1 | +39,398 | +73 | 03 | 0 | 1 | -39,398 | -73 | 33 |
| 1 | +39,398 | +73 | 04 | 0 | 1 | -39,398 | -73 | 33 |
| 1 | +39,398 | +73 | 05 | 1 | 0 | -39,398 | -73 | 33 |
| 1 | +39,398 | +73 | 06 | 0 | 1 | -39,398 | -73 | 33 |
How does level() know whether to modify your data or add a new level? level() uses contextual information – if the name you provide to your level() call is already a field that exists in your data set, level() will treat this as a request to modify this level of data. If, on the other hand, you provide a name not used in the data set, level() will assume you mean to add nested data under the existing data.
We can observe that the new variable is created at the level of aggregation you chose – countries. Also, although N is not specified anywhere, level() knows how large N should be based on the number of countries it finds in the dataset. It is important, then, to ensure that the level() command is correctly assigned to the level of interest.
We can observe that the new variable is created at the level of aggregation you chose – countries. Also, although N is not specified anywhere, modify_level() knows how large N should be based on the number of countries it finds in the dataset. It is important, then, to ensure that the modify_level() command is correctly assigned to the level of interest.
We can also modify more than one level. Recalling our country-province-citizen data from above, the following process is possible:
new_citizen_data <-
fabricate(
data = citizen_data,
- countries = level(avg_temp = runif(N, 30, 80)),
- provinces = level(conflict_zone = draw_discrete(N,
- x=0.2 + has_nat_resources * 0.3,
- type="binary"),
- infant_mortality = runif(N, 0, 10) +
- conflict_zone * 10 +
- (avg_temp > 70) * 10),
- citizens = level(college_degree = draw_discrete(N,
- x=0.4 - (0.3 * conflict_zone),
- type="binary"))
+ countries = modify_level(avg_temp = runif(N, 30, 80)),
+ provinces = modify_level(conflict_zone = draw_discrete(N,
+ x=0.2 + has_nat_resources * 0.3,
+ type="binary"),
+ infant_mortality = runif(N, 0, 10) +
+ conflict_zone * 10 +
+ (avg_temp > 70) * 10),
+ citizens = modify_level(college_degree = draw_discrete(N,
+ x=0.4 - (0.3 * conflict_zone),
+ type="binary"))
)Before assessing what this tells us about level(), let’s consider what the data simulated does. It creates a new variable at the country level, for a country level average temperature. Subsequently, it creates a province level binary indicator for whether the province is an active conflict site. Provinces that have natural resources are more likely to be in conflict in this simulation, drawing on conclusions from literature on “resource curses”. The infant mortality rate for the province is able to depend both on province level data we have just generated, and country-level data: it is higher in high-temperature areas (reflecting literature on increased disease burden near the equator) and also higher in conflict zones. Citizens access to education is also random, but depends on whether they live in a conflict area.
There are a lot of things to learn from this example. First, it’s possible to modify multiple levels. Any new variable created will automatically propagate to the lower level data according – by setting an average temperature for a country, all provinces, and all citizens of those provinces, have the value for the country. Values created from one level() call can be used in subsequent variables of the same call, or subsequent calls.
Before assessing what this tells us about modify_level(), let’s consider what the data simulated does. It creates a new variable at the country level, for a country level average temperature. Subsequently, it creates a province level binary indicator for whether the province is an active conflict site. Provinces that have natural resources are more likely to be in conflict in this simulation, drawing on conclusions from literature on “resource curses”. The infant mortality rate for the province is able to depend both on province level data we have just generated, and country-level data: it is higher in high-temperature areas (reflecting literature on increased disease burden near the equator) and also higher in conflict zones. Citizens access to education is also random, but depends on whether they live in a conflict area.
There are a lot of things to learn from this example. First, it’s possible to modify multiple levels. Any new variable created will automatically propagate to the lower level data according – by setting an average temperature for a country, all provinces, and all citizens of those provinces, have the value for the country. Values created from one modify_level() call can be used in subsequent variables of the same call, or subsequent calls.
Again, we see the use of draw_discrete(). Using this function is covered in our tutorial on generating discrete random variables, linked below.
Consider this example, which takes a dataset containing 2 cities of 3 citizens, and resamples it into a dataset of 3 cities, each containing 5 citizens.
my_data <-
fabricate(
- cities = level(N = 2, elevation = runif(n = N, min = 1000, max = 2000)),
- citizens = level(N = 3, age = runif(N, 18, 70))
+ cities = add_level(N = 2, elevation = runif(n = N, min = 1000, max = 2000)),
+ citizens = add_level(N = 3, age = runif(N, 18, 70))
)
my_data_2 <- resample_data(my_data,
@@ -289,6 +289,116 @@
resample_data() will first select the cities to be resampled. Then, for each city, it will continue by selecting the citizens to be resampled. If a higher level unit is used more than once (for example, the same city being chosen twice), and a lower level is subsequently resampled, the choices of which units to keep for the lower level will differ for each copy of the higher level. In this example, if city 1 is chosen twice, then the sets of five citizens chosen for each copy of the city 1 will differ.
You can also specify the levels you wish to resample from using the name arguents to the N parameter, like in this example which does exactly the same thing as the previous example, but specifies the level names in a different way:
my_data <-
+ fabricate(
+ cities = add_level(N = 2, elevation = runif(n = N, min = 1000, max = 2000)),
+ citizens = add_level(N = 3, age = runif(N, 18, 70))
+ )
+
+my_data_2 <- resample_data(my_data,
+ N = c(cities=3, citizens=5))
+my_data_2| cities | +elevation | +citizens | +age | +
|---|---|---|---|
| 2 | +1014 | +6 | +39 | +
| 2 | +1014 | +5 | +51 | +
| 2 | +1014 | +5 | +51 | +
| 2 | +1014 | +6 | +39 | +
| 2 | +1014 | +5 | +51 | +
| 2 | +1014 | +4 | +41 | +
| 2 | +1014 | +6 | +39 | +
| 2 | +1014 | +5 | +51 | +
| 2 | +1014 | +4 | +41 | +
| 2 | +1014 | +5 | +51 | +
| 1 | +1421 | +1 | +64 | +
| 1 | +1421 | +2 | +66 | +
| 1 | +1421 | +3 | +43 | +
| 1 | +1421 | +3 | +43 | +
| 1 | +1421 | +2 | +66 | +
fabricatr provides a convenient helper function, draw_discrete(), which you can use to generate discrete random variables far more easily than using R’s built-in data generation mechanisms. Below, we introduce you to the types of data you can generate using draw_discrete().
The simplest possible type of data, and draw_discrete()’s default, is a binary random variable (also called a bernoulli random variable). Generating a binary random variable requires only one parameter x which specifies the probability that outcomes drawn from this variable are equal to 1. By default, draw_discrete() will generate N = length(x) draws. N can also be specified explicitly. Consider these examples:
draw_discrete_ex = fabricate(N = 3, p = c(0, .5, 1),
binary_1 = draw_discrete(x = p),
@@ -136,9 +135,9 @@
type = "bernoulli",
link = "probit"))
Some researchers may be interested in generating ordered outcomes – for example, Likert scale outcomes. This can be done using the “ordered” type. Ordered variables require a vector of breakpoints, supplied as the argument breaks – points at which the underlying latent variable switches from category to category. The first break should always be below the lower bound of the data, while the final break should always be above the upper bound of the data.
In the following example, each of three observations has a latent variable x which is continuous and unbounded. The variable ordered transforms x into three numeric categories: 1, 2, and 3. All values of x below -1 result in ordered 1; all values of x between -1 and 1 result in ordered 2; all values of x above 1 result in ordered 3:
ordered_example = fabricate(N = 3,
@@ -158,17 +157,17 @@
)
)
draw_discrete() supports Poisson-distributed count outcomes. These require that the user specify the parameter x, equal to the Poisson distribution mean (often referred to as lambda).
count_outcome_example = fabricate(N = 3,
x = c(0, 5, 100),
count = draw_discrete(x, type = "count"))draw_discrete() can also generate non-ordered, categorical data. Users must provide a vector of probabilities for each category (or a matrix, if each observation should have separate probabilities), as well as setting type to be “categorical”.
If probabilities do not sum to exactly one, they will be normalized, but negative probabilities will cause an error.
In the first example, each unit has a different set of probabilities and the probabilities are provided as a matrix:
@@ -192,6 +191,225 @@“categorical” variables can also use link functions, for example to generate multinomial probit data.
library(fabricatr)
house_candidates = fabricate(
- parties = level(
+ parties = add_level(
N = 2,
party_ideology = c(0.5, -0.5),
in_power = c(1, 0),
party_incumbents = c(241, 194)),
- representatives = level(
+ representatives = add_level(
N = party_incumbents,
member_ideology = rnorm(N, party_ideology),
terms_served = draw_discrete(N = N, x = 3, type = "count"),
diff --git a/docs/reference/draw_binary_icc.html b/docs/reference/draw_binary_icc.html
new file mode 100644
index 0000000..198e3c2
--- /dev/null
+++ b/docs/reference/draw_binary_icc.html
@@ -0,0 +1,198 @@
+
+
+
+
+
+
+
+
+fabricatr - Draw binary data with fixed intra-cluster correlation. — draw_binary_icc
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Draw binary data with fixed intra-cluster correlation.
+
+
+
+ Data is generated to ensure inter-cluster correlation 0, intra-cluster
+correlation in expectation \(\rho\). Algorithm taken from Hossein,
+Akhtar. "ICCbin: An R Package Facilitating Clustered Binary Data
+Generation, and Estimation of Intracluster Correlation Coefficient (ICC)
+for Binary Data".
+
+
+ draw_binary_icc(x = 0.5, N = NULL, cluster_ids, rho = 0.5)
+
+ Arguments
+
+
+ x
+ A number or vector of numbers, one probability per cluster.
+
+ N
+ (Optional) A number indicating the number of observations to be
+generated. Must be equal to length(cluster_ids) if provided.
+
+ cluster_ids
+ A vector of factors or items that can be coerced to
+clusters; the length will determine the length of the generated data.
+
+ rho
+ A number indicating the desired RCC.
+
+
+ Value
+
+ A vector of binary numbers corresponding to the observations from
+the supplied cluster IDs.
+
+
+ Examples
+ cluster_ids = rep(1:5, 10)
+draw_binary_icc(cluster_ids = cluster_ids)#> [1] 0 1 0 0 0 1 1 1 0 0 0 0 1 0 0 0 1 1 0 0 0 1 1 1 0 0 1 1 0 0 0 1 1 0 0 1 1 1
+#> [39] 1 0 0 1 1 0 0 0 0 1 0 0draw_binary_icc(x = 0.5, cluster_ids = cluster_ids, rho = 0.5)#> [1] 1 1 0 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 0 1 1 1 1 1 1 1 1 1 1 0 1 1 0 1 1 1 0
+#> [39] 1 1 1 0 1 1 1 0 1 1 1 1
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/reference/draw_discrete.html b/docs/reference/draw_discrete.html
index 9c95dcc..3f0a0d0 100644
--- a/docs/reference/draw_discrete.html
+++ b/docs/reference/draw_discrete.html
@@ -6,7 +6,8 @@
-fabricatr - Draw discrete variables including binary, binomial count, poisson count, ordered, and categorical — draw_discrete
+fabricatr - Draw discrete variables including binary, binomial count, poisson count,
+ordered, and categorical — draw_discrete
@@ -119,15 +120,19 @@
- Draw discrete variables including binary, binomial count, poisson count, ordered, and categorical
+ Draw discrete variables including binary, binomial count, poisson count,
+ordered, and categorical
- Drawing discrete data based on probabilities or latent traits is a common task that can be cumbersome. draw_binary is an alias for draw_discrete(type = "binary") that allows you to draw binary outcomes more easily.
+ Drawing discrete data based on probabilities or latent traits is a common
+task that can be cumbersome. draw_binary is an alias for
+draw_discrete(type = "binary") that allows you to draw binary
+outcomes more easily.
draw_discrete(x, N = length(x), type = "binary", link = "identity",
- breaks = c(-Inf, 0, Inf), break_labels = FALSE, k = 1)
+ breaks = c(-Inf, 0, Inf), break_labels = NULL, k = 1)
draw_binary(x, N = length(x), link = "identity")
@@ -136,19 +141,26 @@ Ar
x
- vector representing either the latent variable used to draw the count outcome (if link is "logit" or "probit") or the probability for the count outcome (if link is "identity"). For cartegorical distributions x is a matrix with as many columns as possible outcomes.
vector representing either the latent variable used to draw the
+count outcome (if link is "logit" or "probit") or the probability for the
+count outcome (if link is "identity"). For cartegorical distributions x is
+a matrix with as many columns as possible outcomes.
N
- number of units to draw. Defaults to the length of the vector x
number of units to draw. Defaults to the length of the vector
+x
type
- type of discrete outcome to draw, one of 'binary' (or 'bernoulli'), 'binomial', 'categorical', 'ordered' or 'count'
type of discrete outcome to draw, one of 'binary'
+(or 'bernoulli'), 'binomial', 'categorical', 'ordered' or 'count'
link
- link function between the latent variable and the probability of a postiive outcome, i.e. "logit", "probit", or "identity". For the "identity" link, the latent variable must be a probability.
link function between the latent variable and the probability of
+a postiive outcome, i.e. "logit", "probit", or "identity". For the "identity"
+link, the latent variable must be a probability.
breaks
@@ -156,7 +168,8 @@ Ar
break_labels
- vector of labels for the breaks for an ordered latent outcome (must be the same length as breaks)
vector of labels for the breaks for an ordered latent
+outcome (must be the same length as breaks)
k
@@ -180,37 +193,38 @@ Examp
#> 3 3 1.0 1#> ID x binary
-#> 1 1 2.55317055 1
-#> 2 2 -24.37263611 0
-#> 3 3 -0.05571287 1
+ binary = draw_discrete(x, type = "bernoulli", link = "probit"))#> ID x binary
+#> 1 1 -5.6 0
+#> 2 2 -5.4 0
+#> 3 3 2.3 1#> ID p binomial
#> 1 1 0.0 0
-#> 2 2 0.5 4
+#> 2 2 0.5 3
#> 3 3 1.0 10
fabricate(N = 3,
x = 5*rnorm(N),
- ordered = draw_discrete(x, type = "ordered", breaks = c(-Inf, -1, 1, Inf)))#> ID x ordered
-#> 1 1 -9.109088 1
-#> 2 2 -1.236627 1
-#> 3 3 -1.220998 1
+ ordered = draw_discrete(x, type = "ordered",
+ breaks = c(-Inf, -1, 1, Inf)))#> ID x ordered
+#> 1 1 -7.0 1
+#> 2 2 1.3 3
+#> 3 3 -2.2 1#> ID x count
#> 1 1 0 0
-#> 2 2 5 4
-#> 3 3 100 119
+#> 2 2 5 6
+#> 3 3 100 111
# Categorical
fabricate(N = 6, p1 = runif(N), p2 = runif(N), p3 = runif(N),
- cat = draw_discrete(cbind(p1, p2, p3), type = "categorical"))#> ID p1 p2 p3 cat
-#> 1 1 0.67838043 0.53021246 0.6364656 1
-#> 2 2 0.73531960 0.69582388 0.4790245 2
-#> 3 3 0.19595673 0.68855600 0.4321713 2
-#> 4 4 0.98053967 0.03123033 0.7064338 3
-#> 5 5 0.74152153 0.22556253 0.9485766 1
-#> 6 6 0.05144628 0.30083081 0.1803388 2
+ cat = draw_discrete(cbind(p1, p2, p3), type = "categorical"))#> ID p1 p2 p3 cat
+#> 1 1 0.219 0.407 0.37 2
+#> 2 2 0.665 0.858 0.92 2
+#> 3 3 0.390 0.518 0.68 3
+#> 4 4 0.046 0.979 0.67 3
+#> 5 5 0.617 0.017 0.76 1
+#> 6 6 0.598 0.673 0.54 1
-
fabricate helps you simulate a dataset before you collect it. You can either start with your own data and add simulated variables to it (by passing data to fabricate()) or start from scratch by defining N. Create hierarchical data with multiple levels of data such as citizens within cities within states using level(). You can use any R function to create each variable. We provide several built-in options to easily draw from binary and count outcomes, draw_binary and draw_discrete.
-Fabricate a Level of Data for Hierarchical Data
+ fabricate helps you simulate a dataset before you collect it. You can
+either start with your own data and add simulated variables to it (by passing
+data to fabricate()) or start from scratch by defining
+N. Create hierarchical data with multiple levels of data such as
+citizens within cities within states using add_level() or modify
+existing hierarchical data using modify_level(). You can use any R
+function to create each variable. We provide several built-in options to
+easily draw from binary and count outcomes, draw_binary,
+draw_binary_icc, and draw_discrete.
- fabricate(data, N, ID_label, ...)
+ fabricate(data = NULL, N = NULL, ID_label = NULL, ...)
-level(N = NULL, ...)
+add_level(N = NULL, ID_label = NULL, working_environment_ = NULL, ...,
+ data_arguments = quos(...), new_hierarchy = FALSE)
+
+modify_level(N = NULL, ID_label = NULL, working_environment_ = NULL, ...,
+ data_arguments = quos(...))
Arguments
Examp
height_ft = runif(N, 3.5, 8)
)
head(df)#> ID height_ft
-#> 1 001 6.945201
-#> 2 002 6.963537
-#> 3 003 7.958205
-#> 4 004 7.867344
-#> 5 005 5.251322
-#> 6 006 5.575339
+#> 1 001 4.2
+#> 2 002 7.7
+#> 3 003 7.9
+#> 4 004 6.7
+#> 5 005 7.5
+#> 6 006 7.9
# Start with existing data
df <- fabricate(
data = df,
@@ -189,21 +223,22 @@ Examp
# Draw a two-level hierarchical dataset
# containing cities within regions
df <- fabricate(
- regions = level(N = 5),
- cities = level(N = 2, pollution = rnorm(N, mean = 5)))
+ regions = add_level(N = 5),
+ cities = add_level(N = 2, pollution = rnorm(N, mean = 5)))
head(df)
#> regions cities pollution
-#> 1 1 01 5.239065
-#> 2 1 02 5.236321
-#> 3 2 03 4.740881
-#> 4 2 04 5.649046
-#> 5 3 05 3.782359
-#> 6 3 06 5.841970
+#> 1 1 01 5.3
+#> 2 1 02 6.4
+#> 3 2 03 4.8
+#> 4 2 04 5.7
+#> 5 3 05 5.1
+#> 6 3 06 5.0
# Start with existing data and add variables to hierarchical data
-# note: do not provide N when adding variables to an existing level
+# at levels which are already present in the existing data.
+# Note: do not provide N when adding variables to an existing level
df <- fabricate(
data = df,
- regions = level(watershed = sample(c(0, 1), N, replace = TRUE)),
- cities = level(runoff = rnorm(N))
+ regions = modify_level(watershed = sample(c(0, 1), N, replace = TRUE)),
+ cities = modify_level(runoff = rnorm(N))
)
- Draw discrete variables including binary, binomial count, poisson count, ordered, and categorical
Draw discrete variables including binary, binomial count, poisson count,
+ordered, and categorical
+
+
+
+
+ Draw normal data with fixed intra-cluster correlation.
-
+
Fabricate data
diff --git a/docs/reference/level.html b/docs/reference/level.html
new file mode 100644
index 0000000..44fae84
--- /dev/null
+++ b/docs/reference/level.html
@@ -0,0 +1,158 @@
+
+
+
+
+
+
+
+
+fabricatr - Deprecated level call function maintained to provide useful error for
+previous fabricatr code. — level
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Deprecated level call function maintained to provide useful error for
+previous fabricatr code.
+
+
+
+ Deprecated level call function maintained to provide useful error for
+previous fabricatr code.
+
+
+ level(N = NULL, ID_label = NULL, ...)
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/reference/resample_data.html b/docs/reference/resample_data.html
index 2653726..68c14f1 100644
--- a/docs/reference/resample_data.html
+++ b/docs/reference/resample_data.html
@@ -156,107 +156,106 @@ Examp
baseline_survey <- fabricate(N = 5, Y_pre = rnorm(N))
bootsrapped_data <- resample_data(baseline_survey, N = 10)
-bootsrapped_data#> ID Y_pre
-#> 2 2 0.3764993
-#> 4 4 1.2412631
-#> 5 5 0.6120909
-#> 2.1 2 0.3764993
-#> 3 3 1.1387077
-#> 2.2 2 0.3764993
-#> 2.3 2 0.3764993
-#> 2.4 2 0.3764993
-#> 1 1 0.1329921
-#> 2.5 2 0.3764993
+bootsrapped_data#> ID Y_pre
+#> 1 2 -0.38
+#> 2 2 -0.38
+#> 3 5 -0.87
+#> 4 1 1.52
+#> 5 4 -0.55
+#> 6 2 -0.38
+#> 7 5 -0.87
+#> 8 1 1.52
+#> 9 3 1.83
+#> 10 3 1.83
# Resample by a single level of a hierarchical dataset (e.g. resampling clusters of observations)
# N specifies a number of clusters to return
clustered_survey <- fabricate(
- clusters = level(N=25),
- cities = level(N=runif(25, 1, 5), population=runif(n = N, min=50000, max=1000000))
+ clusters = add_level(N=25),
+ cities = add_level(N=round(runif(25, 1, 5)), population=runif(n = N, min=50000, max=1000000))
)
# Specify the name of the cluster variable one of two ways
cluster_resample <- resample_data(clustered_survey, N = 5, ID_labels = "clusters")
cluster_resample#> clusters cities population
-#> 21 10 21 372291.19
-#> 22 10 22 772723.08
-#> 23 10 23 60328.84
-#> 24 10 24 109980.47
-#> 29 14 29 773192.59
-#> 30 14 30 783853.01
-#> 15 07 15 329063.25
-#> 13 06 13 667030.88
-#> 14 06 14 201083.08
-#> 4 02 04 835365.36
-#> 5 02 05 739475.76
-#> 6 02 06 398962.57
-#> 7 02 07 547257.00
+#> 1 14 36 980082
+#> 2 14 37 673751
+#> 3 14 38 135356
+#> 4 14 39 649016
+#> 5 07 14 667784
+#> 6 07 15 197530
+#> 7 07 16 532894
+#> 8 07 17 131577
+#> 9 16 42 448512
+#> 10 16 43 520782
+#> 11 16 44 385069
+#> 12 08 18 527875
+#> 13 24 66 521043
cluster_resample_2 <- resample_data(clustered_survey, N=c(clusters = 5))
cluster_resample_2#> clusters cities population
-#> 16 08 16 896720.6
-#> 60 25 60 495511.1
-#> 61 25 61 892055.5
-#> 62 25 62 687021.1
-#> 8 03 08 496385.0
-#> 38 18 38 239050.7
-#> 39 18 39 662802.0
-#> 40 18 40 762727.9
-#> 41 18 41 307428.9
-#> 13 06 13 667030.9
-#> 14 06 14 201083.1
+#> 1 23 62 306210
+#> 2 23 63 513874
+#> 3 23 64 98509
+#> 4 23 65 52353
+#> 5 03 06 648891
+#> 6 19 48 239037
+#> 7 19 49 739971
+#> 8 15 40 882929
+#> 9 15 41 933767
+#> 10 18 47 391962
# Resample a hierarchical dataset on multiple levels
my_data <-
fabricate(
- cities = level(N = 2, elevation = runif(n = N, min = 1000, max = 2000)),
- citizens = level(N = 3, income = round(elevation * rnorm(n = N, mean = 5)))
+ cities = add_level(N = 2, elevation = runif(n = N, min = 1000, max = 2000)),
+ citizens = add_level(N = 3, income = round(elevation * rnorm(n = N, mean = 5)))
)
# Specify the levels you wish to resample one of two ways:
my_data_2 <- resample_data(my_data, N = c(3, 5), ID_labels = c("cities", "citizens"))
my_data_2#> cities elevation citizens income
-#> 1 2 1927.282 5 10398
-#> 2 2 1927.282 4 7181
-#> 3 2 1927.282 5 10398
-#> 4 2 1927.282 5 10398
-#> 5 2 1927.282 4 7181
-#> 6 1 1133.349 1 7082
-#> 7 1 1133.349 1 7082
-#> 8 1 1133.349 3 5500
-#> 9 1 1133.349 3 5500
-#> 10 1 1133.349 2 4927
-#> 11 1 1133.349 1 7082
-#> 12 1 1133.349 1 7082
-#> 13 1 1133.349 2 4927
-#> 14 1 1133.349 2 4927
-#> 15 1 1133.349 2 4927
+#> 1 1 1701 3 10505
+#> 2 1 1701 3 10505
+#> 3 1 1701 1 8210
+#> 4 1 1701 3 10505
+#> 5 1 1701 1 8210
+#> 6 1 1701 2 8789
+#> 7 1 1701 2 8789
+#> 8 1 1701 2 8789
+#> 9 1 1701 3 10505
+#> 10 1 1701 3 10505
+#> 11 1 1701 3 10505
+#> 12 1 1701 1 8210
+#> 13 1 1701 1 8210
+#> 14 1 1701 2 8789
+#> 15 1 1701 3 10505
my_data_3 <- resample_data(my_data, N = c(cities=3, citizens=5))
my_data_3#> cities elevation citizens income
-#> 1 2 1927.282 4 7181
-#> 2 2 1927.282 6 5911
-#> 3 2 1927.282 5 10398
-#> 4 2 1927.282 6 5911
-#> 5 2 1927.282 5 10398
-#> 6 2 1927.282 5 10398
-#> 7 2 1927.282 5 10398
-#> 8 2 1927.282 4 7181
-#> 9 2 1927.282 6 5911
-#> 10 2 1927.282 5 10398
-#> 11 1 1133.349 3 5500
-#> 12 1 1133.349 2 4927
-#> 13 1 1133.349 3 5500
-#> 14 1 1133.349 2 4927
-#> 15 1 1133.349 3 5500
+#> 1 2 1040 4 6786
+#> 2 2 1040 6 5508
+#> 3 2 1040 4 6786
+#> 4 2 1040 5 4613
+#> 5 2 1040 4 6786
+#> 6 2 1040 6 5508
+#> 7 2 1040 4 6786
+#> 8 2 1040 6 5508
+#> 9 2 1040 5 4613
+#> 10 2 1040 6 5508
+#> 11 1 1701 1 8210
+#> 12 1 1701 2 8789
+#> 13 1 1701 1 8210
+#> 14 1 1701 2 8789
+#> 15 1 1701 3 10505
# Transparently pass through all units at a given level
# This example will resample 2 citizens at each of the cities:
passthrough_resample_data <- resample_data(my_data, N = c(cities=ALL, citizens=2))
passthrough_resample_data#> cities elevation citizens income
-#> 1 1 1133.349 2 4927
-#> 2 1 1133.349 2 4927
-#> 3 2 1927.282 5 10398
-#> 4 2 1927.282 6 5911
+#> 1 1 1701 1 8210
+#> 2 1 1701 1 8210
+#> 3 2 1040 4 6786
+#> 4 2 1040 5 4613
diff --git a/man/cross_level.Rd b/man/cross_level.Rd
new file mode 100644
index 0000000..17cc716
--- /dev/null
+++ b/man/cross_level.Rd
@@ -0,0 +1,27 @@
+% Generated by roxygen2: do not edit by hand
+% Please edit documentation in R/fabricate.R
+\name{cross_level}
+\alias{cross_level}
+\title{Creates cross-classified (partially non-nested, joined data) with a fixed
+correlation structure.}
+\usage{
+cross_level(N = NULL, ID_label = NULL, working_environment_ = NULL,
+ by = NULL, ..., data_arguments = quos(...))
+}
+\arguments{
+\item{N}{(required) The number of observations in the resulting data frame}
+
+\item{ID_label}{Internal keyword used to sepcify the name of the ID variable created for the new level. If left empty, this will be the name the level is assigned to as part of a \code{fabricate()} call.}
+
+\item{working_environment_}{Internal keyword not for end user use.}
+
+\item{by}{The result of a call to \code{join()} which specifies how the cross-classified data will be created}
+
+\item{...}{A variable or series of variables to add to the resulting data frame after the cross-classified data is created.}
+
+\item{data_arguments}{Internal keyword not for end user use.}
+}
+\description{
+Creates cross-classified (partially non-nested, joined data) with a fixed
+correlation structure.
+}
diff --git a/man/draw_binary_icc.Rd b/man/draw_binary_icc.Rd
index 7b2b519..4486dfe 100644
--- a/man/draw_binary_icc.Rd
+++ b/man/draw_binary_icc.Rd
@@ -4,10 +4,11 @@
\alias{draw_binary_icc}
\title{Draw binary data with fixed intra-cluster correlation.}
\usage{
-draw_binary_icc(x = 0.5, N = NULL, clusters, rho = 0.5)
+draw_binary_icc(x = 0.5, N = NULL, clusters, rho = 0)
}
\arguments{
-\item{x}{A number or vector of numbers, one probability per cluster.}
+\item{x}{A number or vector of numbers, one probability per cluster. If none
+is provided, will default to 0.5.}
\item{N}{(Optional) A number indicating the number of observations to be
generated. Must be equal to length(clusters) if provided.}
@@ -15,7 +16,8 @@ generated. Must be equal to length(clusters) if provided.}
\item{clusters}{A vector of factors or items that can be coerced to
clusters; the length will determine the length of the generated data.}
-\item{rho}{A number indicating the desired RCC.}
+\item{rho}{A number indicating the desired ICC, if none is provided will
+default to 0.}
}
\value{
A vector of binary numbers corresponding to the observations from
diff --git a/man/draw_discrete.Rd b/man/draw_discrete.Rd
index 8a23400..a9f0adf 100644
--- a/man/draw_discrete.Rd
+++ b/man/draw_discrete.Rd
@@ -3,30 +3,42 @@
\name{draw_discrete}
\alias{draw_discrete}
\alias{draw_binary}
-\title{Draw discrete variables including binary, binomial count, poisson count, ordered, and categorical}
+\title{Draw discrete variables including binary, binomial count, poisson count,
+ordered, and categorical}
\usage{
draw_discrete(x, N = length(x), type = "binary", link = "identity",
- breaks = c(-Inf, 0, Inf), break_labels = FALSE, k = 1)
+ breaks = c(-Inf, 0, Inf), break_labels = NULL, k = 1)
draw_binary(x, N = length(x), link = "identity")
}
\arguments{
-\item{x}{vector representing either the latent variable used to draw the count outcome (if link is "logit" or "probit") or the probability for the count outcome (if link is "identity"). For cartegorical distributions x is a matrix with as many columns as possible outcomes.}
+\item{x}{vector representing either the latent variable used to draw the
+count outcome (if link is "logit" or "probit") or the probability for the
+count outcome (if link is "identity"). For cartegorical distributions x is
+a matrix with as many columns as possible outcomes.}
-\item{N}{number of units to draw. Defaults to the length of the vector \code{x}}
+\item{N}{number of units to draw. Defaults to the length of the vector
+\code{x}}
-\item{type}{type of discrete outcome to draw, one of 'binary' (or 'bernoulli'), 'binomial', 'categorical', 'ordered' or 'count'}
+\item{type}{type of discrete outcome to draw, one of 'binary'
+(or 'bernoulli'), 'binomial', 'categorical', 'ordered' or 'count'}
-\item{link}{link function between the latent variable and the probability of a postiive outcome, i.e. "logit", "probit", or "identity". For the "identity" link, the latent variable must be a probability.}
+\item{link}{link function between the latent variable and the probability of
+a postiive outcome, i.e. "logit", "probit", or "identity". For the "identity"
+link, the latent variable must be a probability.}
\item{breaks}{vector of breaks to cut an ordered latent outcome}
-\item{break_labels}{vector of labels for the breaks for an ordered latent outcome (must be the same length as breaks)}
+\item{break_labels}{vector of labels for the breaks for an ordered latent
+outcome (must be the same length as breaks)}
\item{k}{the number of trials (zero or more)}
}
\description{
-Drawing discrete data based on probabilities or latent traits is a common task that can be cumbersome. \code{draw_binary} is an alias for \code{draw_discrete(type = "binary")} that allows you to draw binary outcomes more easily.
+Drawing discrete data based on probabilities or latent traits is a common
+task that can be cumbersome. \code{draw_binary} is an alias for
+\code{draw_discrete(type = "binary")} that allows you to draw binary
+outcomes more easily.
}
\examples{
fabricate(N = 3,
@@ -47,7 +59,8 @@ fabricate(N = 3,
fabricate(N = 3,
x = 5*rnorm(N),
- ordered = draw_discrete(x, type = "ordered", breaks = c(-Inf, -1, 1, Inf)))
+ ordered = draw_discrete(x, type = "ordered",
+ breaks = c(-Inf, -1, 1, Inf)))
fabricate(N = 3,
x = c(0,5,100),
diff --git a/man/draw_normal_icc.Rd b/man/draw_normal_icc.Rd
index 9275822..8b0e57e 100644
--- a/man/draw_normal_icc.Rd
+++ b/man/draw_normal_icc.Rd
@@ -4,10 +4,11 @@
\alias{draw_normal_icc}
\title{Draw normal data with fixed intra-cluster correlation.}
\usage{
-draw_normal_icc(x = 0, N = NULL, clusters, sd = 1, rho = 0.5)
+draw_normal_icc(x = 0, N = NULL, clusters, sd = 1, rho = 0)
}
\arguments{
-\item{x}{A number or vector of numbers, one mean per cluster.}
+\item{x}{A number or vector of numbers, one mean per cluster. If none is
+provided, will default to 0.}
\item{N}{(Optional) A number indicating the number of observations to be
generated. Must be equal to length(clusters) if provided.}
@@ -18,7 +19,8 @@ clusters; the length will determine the length of the generated data.}
\item{sd}{A number or vector of numbers, indicating the standard deviation of
each cluster's error terms}
-\item{rho}{A number indicating the desired RCC.}
+\item{rho}{A number indicating the desired ICC. If none is provided,
+will default to 0.}
}
\value{
A vector of numbers corresponding to the observations from
diff --git a/man/fabricate.Rd b/man/fabricate.Rd
index 756b5b8..e0c0ba0 100644
--- a/man/fabricate.Rd
+++ b/man/fabricate.Rd
@@ -1,30 +1,57 @@
% Generated by roxygen2: do not edit by hand
-% Please edit documentation in R/fabricate.R, R/level.R
+% Please edit documentation in R/fabricate.R
\name{fabricate}
\alias{fabricate}
-\alias{level}
+\alias{add_level}
+\alias{modify_level}
\title{Fabricate data}
\usage{
-fabricate(data, N, ID_label, ...)
+fabricate(data = NULL, ..., N = NULL, ID_label = NULL)
-level(N = NULL, ...)
+add_level(N = NULL, ID_label = NULL, working_environment_ = NULL, ...,
+ data_arguments = quos(...), nest = TRUE)
+
+modify_level(N = NULL, ID_label = NULL, working_environment_ = NULL, ...,
+ data_arguments = quos(...))
}
\arguments{
-\item{data}{(optional) user-provided data that forms the basis of the fabrication, i.e. you can add variables to existing data. Provide either \code{N} or \code{data} (\code{N} is the number of rows of the data if \code{data} is provided).}
+\item{data}{(optional) user-provided data that forms the basis of the
+fabrication, i.e. you can add variables to existing data. Provide either
+\code{N} or \code{data} (\code{N} is the number of rows of the data if
+\code{data} is provided).}
+
+\item{...}{Variable or level-generating arguments, such as
+\code{my_var = rnorm(N)}. For \code{fabricate}, you may also pass
+\code{add_level()} or \code{modify_level()} arguments, which define a level
+of a multi-level dataset. See examples.}
-\item{N}{(optional) number of units to draw. If provided as \code{fabricate(N = 5)}, this determines the number of units in the single-level data. If provided in \code{level}, i.e. \code{fabricate(cities = level(N = 5))}, \code{N} determines the number of units in a specific level of a hierarchical dataset.}
+\item{N}{(optional) number of units to draw. If provided as
+\code{fabricate(N = 5)}, this determines the number of units in the
+single-level data. If provided in \code{add_level}, i.e.
+\code{fabricate(cities = add_level(N = 5))}, \code{N} determines the number
+of units in a specific level of a hierarchical dataset.}
\item{ID_label}{(optional) variable name for ID variable, i.e. citizen_ID}
-\item{...}{Variable or level-generating arguments, such as \code{my_var = rnorm(N)}. For \code{fabricate}, you may also pass \code{level()} arguments, which define a level of a multi-level dataset. See examples.}
+\item{working_environment_}{Internal argument, not intended for end-user use.}
+
+\item{data_arguments}{Internal argument, not intended for end-user use.}
+
+\item{nest}{(Default TRUE) Boolean determining whether data in an \code{add_level()} call will be nested under the current working data frame or create a separate hierarchy of levels. See our vignette for cross-classified, non-nested data for details.}
}
\value{
data.frame
}
\description{
-\code{fabricate} helps you simulate a dataset before you collect it. You can either start with your own data and add simulated variables to it (by passing \code{data} to \code{fabricate()}) or start from scratch by defining \code{N}. Create hierarchical data with multiple levels of data such as citizens within cities within states using \code{level()}. You can use any R function to create each variable. We provide several built-in options to easily draw from binary and count outcomes, \code{\link{draw_binary}} and \code{\link{draw_discrete}}.
-
-Fabricate a Level of Data for Hierarchical Data
+\code{fabricate} helps you simulate a dataset before you collect it. You can
+either start with your own data and add simulated variables to it (by passing
+\code{data} to \code{fabricate()}) or start from scratch by defining
+\code{N}. Create hierarchical data with multiple levels of data such as
+citizens within cities within states using \code{add_level()} or modify
+existing hierarchical data using \code{modify_level()}. You can use any R
+function to create each variable. We provide several built-in options to
+easily draw from binary and count outcomes, \code{\link{draw_binary}},
+\code{\link{draw_binary_icc}}, and \code{\link{draw_discrete}}.
}
\examples{
@@ -48,16 +75,17 @@ df <- fabricate(
# Draw a two-level hierarchical dataset
# containing cities within regions
df <- fabricate(
- regions = level(N = 5),
- cities = level(N = 2, pollution = rnorm(N, mean = 5)))
+ regions = add_level(N = 5),
+ cities = add_level(N = 2, pollution = rnorm(N, mean = 5)))
head(df)
# Start with existing data and add variables to hierarchical data
-# note: do not provide N when adding variables to an existing level
+# at levels which are already present in the existing data.
+# Note: do not provide N when adding variables to an existing level
df <- fabricate(
data = df,
- regions = level(watershed = sample(c(0, 1), N, replace = TRUE)),
- cities = level(runoff = rnorm(N))
+ regions = modify_level(watershed = sample(c(0, 1), N, replace = TRUE)),
+ cities = modify_level(runoff = rnorm(N))
)
}
diff --git a/man/join.Rd b/man/join.Rd
new file mode 100644
index 0000000..48536b5
--- /dev/null
+++ b/man/join.Rd
@@ -0,0 +1,22 @@
+% Generated by roxygen2: do not edit by hand
+% Please edit documentation in R/fabricate.R
+\name{join}
+\alias{join}
+\title{Helper function handling specification of which variables to join a
+cross-classified data on, and what kind of correlation structure needed}
+\usage{
+join(..., rho = 0, sigma = NULL, data_arguments = quos(...))
+}
+\arguments{
+\item{...}{A series of two or more variable names, unquoted, to join on in order to create cross-classified data.}
+
+\item{rho}{A fixed (Spearman's rank) correlation coefficient between the variables being joined on: note that if it is not possible to make a correlation matrix from this coefficient (i.e. if you are joining on three or more variables and rho is negative) then the \code{cross_level()} call will fail.}
+
+\item{sigma}{A matrix with dimensions equal to the number of variables you are joining on, specifying the correlation for the resulting joined data. Only one of rho and sigma should be provided.}
+
+\item{data_arguments}{Internal, not for end-user use.}
+}
+\description{
+Helper function handling specification of which variables to join a
+cross-classified data on, and what kind of correlation structure needed
+}
diff --git a/man/level.Rd b/man/level.Rd
new file mode 100644
index 0000000..a4592ee
--- /dev/null
+++ b/man/level.Rd
@@ -0,0 +1,14 @@
+% Generated by roxygen2: do not edit by hand
+% Please edit documentation in R/fabricate.R
+\name{level}
+\alias{level}
+\title{Deprecated level call function maintained to provide useful error for
+previous fabricatr code.}
+\usage{
+level(N = NULL, ID_label = NULL, ...)
+}
+\description{
+Deprecated level call function maintained to provide useful error for
+previous fabricatr code.
+}
+\keyword{internal}
diff --git a/man/resample_data.Rd b/man/resample_data.Rd
index 3f024e3..7ebf656 100644
--- a/man/resample_data.Rd
+++ b/man/resample_data.Rd
@@ -31,8 +31,8 @@ bootsrapped_data
# N specifies a number of clusters to return
clustered_survey <- fabricate(
- clusters = level(N=25),
- cities = level(N=runif(25, 1, 5), population=runif(n = N, min=50000, max=1000000))
+ clusters = add_level(N=25),
+ cities = add_level(N=round(runif(25, 1, 5)), population=runif(n = N, min=50000, max=1000000))
)
# Specify the name of the cluster variable one of two ways
@@ -47,8 +47,8 @@ cluster_resample_2
my_data <-
fabricate(
- cities = level(N = 2, elevation = runif(n = N, min = 1000, max = 2000)),
- citizens = level(N = 3, income = round(elevation * rnorm(n = N, mean = 5)))
+ cities = add_level(N = 2, elevation = runif(n = N, min = 1000, max = 2000)),
+ citizens = add_level(N = 3, income = round(elevation * rnorm(n = N, mean = 5)))
)
# Specify the levels you wish to resample one of two ways:
diff --git a/tests/testthat/test-crossclassified.R b/tests/testthat/test-crossclassified.R
new file mode 100644
index 0000000..8a570a1
--- /dev/null
+++ b/tests/testthat/test-crossclassified.R
@@ -0,0 +1,149 @@
+context("Fabricate")
+
+test_that("Cross-classified data", {
+ # Example draw setup
+ students = fabricate(
+ primary_schools = add_level(N = 100,
+ ps_quality = runif(n=N, 1, 100),
+ ps_hasband = draw_binary(0.5, N=N),
+ ps_testscores = ps_quality * 5 + rnorm(N, 30, 5)),
+ secondary_schools = add_level(N = 50,
+ ss_quality = runif(n=N, 1, 100),
+ ss_hascomputers = draw_binary(ss_quality/100, N=N),
+ ss_testscores = ss_quality * 5 + rnorm(N, 30, 5),
+ nest = FALSE),
+ students = cross_level(N = 1000,
+ by = join(ps_quality, ss_quality, rho=0.5),
+ student_score = ps_testscores * 5 + ss_testscores * 10 + rnorm(N, 10, 5),
+ student_score_2 = student_score * 2,
+ extracurricular = ps_hasband + ss_hascomputers
+ )
+ )
+
+ # Within a reasonable "tolerance"
+ expect_gte(cor(students$ps_quality, students$ss_quality), 0.3)
+ expect_lte(cor(students$ps_quality, students$ss_quality), 0.7)
+
+ # Uncorrelated
+ students_uncorr = fabricate(
+ primary_schools = add_level(N = 100,
+ ps_quality = runif(n=N, 1, 100),
+ ps_hasband = draw_binary(0.5, N=N),
+ ps_testscores = ps_quality * 5 + rnorm(N, 30, 5)),
+ secondary_schools = add_level(N = 50,
+ ss_quality = runif(n=N, 1, 100),
+ ss_hascomputers = draw_binary(ss_quality/100, N=N),
+ ss_testscores = ss_quality * 5 + rnorm(N, 30, 5),
+ nest = FALSE),
+ students = cross_level(N = 1000,
+ by = join(ps_quality, ss_quality, rho=0),
+ student_score = ps_testscores * 5 + ss_testscores * 10 + rnorm(N, 10, 5),
+ student_score_2 = student_score * 2,
+ extracurricular = ps_hasband + ss_hascomputers
+ )
+ )
+
+ # Again, within tolerance
+ expect_gte(cor(students_uncorr$ps_quality, students_uncorr$ss_quality), -0.1)
+ expect_lte(cor(students_uncorr$ps_quality, students_uncorr$ss_quality), 0.1)
+
+
+ # Specifying sigma in lieu of rho
+ test_next = fabricate(
+ l1 = add_level(N = 50, j1 = rnorm(N)),
+ l2 = add_level(N = 50, j2 = rnorm(N), nest=FALSE),
+ joined = cross_level(N = 200,
+ by = join(j1, j2, sigma=matrix(c(1, 0.5, 0.5, 1), ncol=2)))
+ )
+
+ expect_gte(cor(test_next$j1, test_next$j2), 0.3)
+ expect_lte(cor(test_next$j1, test_next$j2), 0.7)
+})
+
+test_that("Code path without mvnfast", {
+ # Need to directly call joint_draw_ecdf because we don't let users voluntarily
+ # override the use_f argument
+ dl = list(j1 = rnorm(100),
+ j2 = rnorm(500))
+ result = fabricatr:::joint_draw_ecdf(dl, N = 100, rho = 0.3, use_f = FALSE)
+ data = cbind(dl$j1[result[[1]]],
+ dl$j2[result[[2]]])
+ expect_gte(cor(data[, 1], data[, 2]), 0.1)
+ expect_lte(cor(data[, 1], data[, 2]), 0.5)
+})
+
+test_that("Deliberate failures in join_dfs", {
+ df1 = fabricate(N=100, j1 = rnorm(100))
+ df2 = fabricate(N=100, j2 = rnorm(100))
+ df3 = fabricate(N=100, j3 = rnorm(100))
+
+ expect_error(fabricatr:::join_dfs(df1, c("j1"), N=100, rho=0.5))
+ expect_error(fabricatr:::join_dfs(list(df1, df2), c("j1"), N=100, rho=0.5))
+ expect_error(fabricatr:::join_dfs(list(df1), c("j1"), N=100, rho=0.5))
+ expect_error(fabricatr:::join_dfs(list(df1, df2), c("j1", "j2"), N=-1, rho=0.5))
+ expect_error(fabricatr:::join_dfs(list(df1, df2), c("j1"), N=c(3, 10), rho=0.5))
+ expect_error(fabricatr:::join_dfs(list(df1, df2, df3), c("j1", "j2", "j3"), N=100, rho=-0.5))
+ expect_error(fabricatr:::join_dfs(list(df1, df2), c("j1", "j2"), N=100, rho=c(0.5, 0.3)))
+
+ expect_error(fabricatr:::join_dfs(list(df1, df2), c("j1", "j2"),
+ N=100,
+ sigma=matrix(c(1, 0.3, 0.3, 0.3, 1, 0.3, 0.3, 0.3, 1),
+ ncol = 3
+ )))
+})
+
+test_that("Deliberate failures in cross_level", {
+ expect_error(
+ fabricate(
+ l1 = add_level(N = 50, j1 = rnorm(N)),
+ l2 = add_level(N = 50, j2 = rnorm(N), nest=FALSE),
+ joined = cross_level(N = 200,
+ by = join(j1,
+ j_error,
+ sigma=matrix(c(1, 0.5, 0.5, 1),
+ ncol=2)
+ )
+ )
+ )
+ )
+
+ expect_error(
+ fabricate(
+ l1 = add_level(N = 50, j1 = rnorm(N)),
+ l2 = add_level(N = 50, j_var = rnorm(N), j1 = runif(N, 1, 3), nest=FALSE),
+ joined = cross_level(N = 200,
+ by = join(j1, j_var, sigma=matrix(c(1, 0.5, 0.5, 1), ncol=2)))
+ )
+ )
+
+ expect_error(
+ fabricate(
+ l1 = add_level(N = 50, j1 = rnorm(N)),
+ l2 = add_level(N = 50, j2 = rnorm(N), nest=FALSE),
+ joined = cross_level(N = 200)
+ )
+ )
+
+ expect_error(
+ fabricate(
+ l1 = add_level(N = 50),
+ joined = cross_level(N = 200)
+ )
+ )
+
+ expect_error(
+ fabricate(
+ l1 = add_level(N = 50, v1 = rnorm(N), v2 = rnorm(N), v3 = rnorm(N)),
+ l2 = add_level(N = 30, v4 = rnorm(N), nest=FALSE),
+ joined = cross_level(N = 100, by=join(v1, v2))
+ )
+ )
+
+ expect_error(
+ fabricate(
+ l1 = add_level(N = 50, v1 = rnorm(N), v2 = rnorm(N), v3 = rnorm(N)),
+ l2 = add_level(N = 30, v4 = rnorm(N), nest=FALSE),
+ joined = cross_level(N = 100, by=join(v1, v4, v1))
+ )
+ )
+})
diff --git a/tests/testthat/test-fabrication.R b/tests/testthat/test-fabrication.R
index 33b207b..cb93181 100644
--- a/tests/testthat/test-fabrication.R
+++ b/tests/testthat/test-fabrication.R
@@ -26,84 +26,96 @@ test_that("Fabricate", {
ID_label = "ID"
)
- fabricate(regions = level(N = 5, gdp = rnorm(N)))
+ fabricate(regions = add_level(N = 5, gdp = rnorm(N)))
fabricate(
- regions = level(N = 5, gdp = rnorm(N)),
- cities = level(N = sample(1:5), subways = gdp + 10)
+ regions = add_level(N = 5, gdp = rnorm(N)),
+ cities = add_level(N = sample(1:5), subways = gdp + 10)
)
- fabricate(regions = level(N = 5),
- cities = level(N = sample(1:5), subways = rnorm(N, mean = 5)))
+ fabricate(regions = add_level(N = 5),
+ cities = add_level(N = sample(1:5), subways = rnorm(N, mean = 5)))
fabricate(
- regions = level(N = 5, gdp = runif(N)),
- cities = level(N = sample(1:5), subways = rnorm(N, mean = 5))
+ regions = add_level(N = 5, gdp = runif(N)),
+ cities = add_level(N = sample(1:5), subways = rnorm(N, mean = 5))
)
# User provides matrix, test conversion.
- fabricate(data = matrix(rep(c(1, 2, 3), 3), byrow=TRUE, ncol=3, nrow=3))
+ fabricate(data = matrix(rep(c(1, 2, 3), 3),
+ byrow=TRUE,
+ ncol=3,
+ nrow=3))
})
-test_that("use a function to choose N of a level", {
+test_that("choose N of a level based on data from higher levels", {
fabricate(
- regions = level(N = 2, gdp = runif(N)),
- cities = level(
- N = function(x)
- return(round(gdp) * 10 + 1),
+ regions = add_level(N = 2, gdp = runif(N)),
+ cities = add_level(
+ N = round(gdp) * 10 + 1,
subways = rnorm(N, mean = 5)
)
)
})
+test_that("Import data, single level var modification, with/without ID", {
+ expect_equal(
+ ncol(fabricate(datasets::BOD, dd = demand * 2, ID_label="Time")),
+ 3)
+
+ expect_equal(
+ ncol(fabricate(datasets::BOD, dd = demand * 2)),
+ 4)
+
+ expect_equal(
+ ncol(fabricate(datasets::BOD, dd = demand * 2, ID_label="Jello")),
+ 4)
+})
+
test_that("trigger errors", {
+ # User didn't provide a name for a level, and let's make sure that we also
+ # didn't interpret the unnamed level as any of the special arguments contextually
expect_error(fabricate(
- regions = level(),
- cities = level(N = sample(1:5), subways = rnorm(N, mean = 5))
- ))
+ data = NULL,
+ N = NULL,
+ ID_label = NULL,
+ countries = add_level(N = 10),
+ add_level(N = 5, population = rnorm(N))))
expect_error(fabricate(
- regions = level(N = c(1, 2)),
- cities = level(N = sample(1:5), subways = rnorm(N, mean = 5))
+ regions = add_level(),
+ cities = add_level(N = sample(1:5), subways = rnorm(N, mean = 5))
))
expect_error(fabricate(
- regions = level(N = 2),
- cities = level(N = c(5, 5, 5), subways = rnorm(N, mean = 5))
+ regions = add_level(N = c(1, 2)),
+ cities = add_level(N = sample(1:5), subways = rnorm(N, mean = 5))
))
expect_error(fabricate(
- regions = level(N = 2),
- cities = level(N = "N that is a character vector", subways = rnorm(N, mean = 5))
+ regions = add_level(N = 2),
+ cities = add_level(N = c(5, 5, 5), subways = rnorm(N, mean = 5))
))
- region_data <- data.frame(capital = c(1, 0, 0, 0, 0))
- expect_error(fabricatr:::fabricate_data_single_level(data = region_data, N = 5, gdp = runif(N)))
-
expect_error(fabricate(
- regions = level(N = rep(5, 2)),
- cities = level(N = c(5, 5, 5), subways = rnorm(N, mean = 5))
+ regions = add_level(N = 2),
+ cities = add_level(N = "N that is a character vector", subways = rnorm(N, mean = 5))
))
- expect_error(fabricatr:::fabricate_data_single_level(
- N = c(5, 2),
- gdp = runif(N),
- ID_label = "my-level"
+ expect_error(fabricate(
+ regions = add_level(N = rep(5, 2)),
+ cities = add_level(N = c(5, 5, 5), subways = rnorm(N, mean = 5))
))
# you must provide name for levels
expect_error(fabricate(level(N = 5,
gdp = rnorm(N)),
- level(
+ add_level(
N = sample(1:5),
subways = rnorm(N, mean = gdp)
)))
- # same for a single level
- expect_error(fabricate(level(N = 5,
- gdp = rnorm(N))))
-
# No N, no data
expect_error(fabricate(test1 = runif(10), test2 = test1 * 3 * runif(10, 1, 2)))
@@ -120,9 +132,8 @@ test_that("trigger errors", {
# Negative N
expect_error(fabricate(N = -1, test1=runif(10)))
- # must send a data frame to data
- expect_error(user_data <- fabricate(data = c(5)))
-
+ # Scalar as data
+ expect_error(fabricate(data = c(5)))
# Vector as ID_label
expect_error(fabricate(N=10, test1=rnorm(10), test2=rpois(10, lambda=2), ID_label=c("invalid", "id")))
# Matrix as ID_label
@@ -133,9 +144,50 @@ test_that("trigger errors", {
fabricate(N=10, test1=rnorm(10), test2=rpois(10, lambda=2), ID_label="hello")
fabricate(N=10, test1=rnorm(10), test2=rpois(10, lambda=2), ID_label=c("hello"))
# Symbol as ID_label
- fabricate(N=10, test1=rnorm(10), test2=rpois(10, lambda=2), ID_label=test1)
- fabricate(N=10, test1=rnorm(10), test2=rpois(10, lambda=2), ID_label=test3)
+ expect_error(fabricate(N=10, test1=rnorm(10), test2=rpois(10, lambda=2), ID_label=test1))
+ expect_error(fabricate(N=10, test1=rnorm(10), test2=rpois(10, lambda=2), ID_label=test3))
# Unusual test with implicit data argument
expect_error(fabricate(N=10, 1:N))
+
+})
+
+test_that("unusual pass of add_level call to single level generation as data matrix", {
+ expect_error(fabricate(add_level(N = 5,
+ gdp = rnorm(N))))
+})
+
+test_that("modify_level call when you probably meant add_level", {
+ expect_error(fabricate(countries = modify_level(N = 10, new_var = rnorm(N))))
+})
+
+test_that("modify_level call where you don't specify which level", {
+ expect_error(fabricate(countries = add_level(N=20),
+ modify_level(ID_new = as.numeric(ID) * 2)))
+})
+
+test_that("nest_level call when there was no data to nest", {
+ # No import data, nest level
+ expect_error(fabricate(countries = nest_level(N = 10, new_var = rnorm(N))))
+
+ # Import data, should be able to nest level
+ fabricate(datasets::BOD, units = nest_level(N = 2, dd = demand * 2))
+})
+
+
+test_that("multiple non-nested data frames, again and again", {
+ fabricate(
+ l1 = add_level(N = 100),
+ l2 = add_level(N = 200, nest=FALSE),
+ l3 = add_level(N = 100, nest=FALSE),
+ l4 = add_level(N = 300, nest=FALSE)
+ )
+})
+
+test_that("importing data and then specifying a level ID variable that is in data.", {
+ df = fabricate(N = 100, d1 = rnorm(N), ID_label = "hello")
+ df2 = fabricate(df, ID_label = "hello", new_var1 = d1 * 2)
+ expect_equal(length(colnames(df2)), 3)
+ df3 = fabricate(df, new_var1 = d1 * 2)
+ expect_equal(length(colnames(df3)), 4)
})
diff --git a/tests/testthat/test-helper-functions.R b/tests/testthat/test-helper-functions.R
new file mode 100644
index 0000000..56f5c74
--- /dev/null
+++ b/tests/testthat/test-helper-functions.R
@@ -0,0 +1,80 @@
+context("Helper functions")
+
+test_that("Error handlers: handle_data", {
+ # User passed in data that isn't a data frame (no dimensionality)
+ expect_error(handle_data(1:10))
+
+ # It's not a data frame and the user didn't specify the argument name
+ # this is a weird case involving arguments being interpreted by position
+ # from fabricate and weird stuff flipping through to handle_data
+ expect_error(handle_data(matrix(1:9, nrow=3, ncol=3)))
+
+ # Test coercion of dimensional but non-df objects -- a matrix is most common
+ # example -- this is also a working use case.
+ handle_data(data = matrix(1:9, nrow=3, ncol=3))
+
+ # Really stupid test -- object has dimensionality but won't coerce to df
+ # this should almost never happen except for very poorly behaved objects
+ X = 1:10
+ Y = X*2 + rnorm(10)
+ df = data.frame(X = X, Y=Y)
+ model = lm(Y ~ X, df)
+ dim(model) = c(3, 4) # This will break the model object, but fine for test
+ expect_error(handle_data(data = model))
+})
+
+test_that("Error handlers: handle_id", {
+ # Cartoon scenario where we're asked to generate an ID
+ # but our 6 default variables are all taken
+ data = data.frame(ID = 1:10,
+ fab_ID_1 = 11:20,
+ fab_ID_2 = 21:30,
+ fab_ID_3 = 31:40,
+ fab_ID_4 = 41:50,
+ fab_ID_5 = 51:60)
+ expect_error(handle_id(NULL, data))
+
+ # And verify that the waterfall works as expected
+ ID = handle_id(NULL, data[, 1:2])
+ expect_equal(ID, "fab_ID_2")
+})
+
+test_that("Error handlers: handle_n", {
+ # Passed closure as N, didn't evaluate it
+ expect_error(handle_n(N = function(x) { x*2 }))
+
+ # Passed closure as N, did evaluate it
+ func = function(x) { x*2 }
+ handle_n(N = func(4))
+
+ # Non-numeric type where coercion gives a warning
+ expect_error(handle_n(N = "hello"))
+
+ # Non-numeric type where coercion gives an explicit error
+ expect_error(handle_n(N = list(Z = Y ~ X)))
+})
+
+test_that("Error handlers: check_rectangular", {
+ # Everything is either length N or length 1
+ test = list(X = 1:10,
+ Y = 11:20,
+ Z = 4)
+ N = 10
+ check_rectangular(test, N)
+
+ test[["K"]] = 5:7
+ expect_error(check_rectangular(test, N))
+})
+
+test_that("get_unique_variables_by_level", {
+ df = datasets::ChickWeight
+ expect_equal(length(get_unique_variables_by_level(df, "Diet")), 0)
+
+ df$DietVar = as.numeric(df$Diet) * 3
+ expect_equal(length(get_unique_variables_by_level(df, "Diet")), 1)
+})
+
+test_that("Advance lookahead symbol evaluator", {
+ my_quos = rlang::quos(J = KK * LMNOP * max(F, G, H, 20, (((K)))))
+ expect_equal(length(get_symbols_from_quosure(my_quos)[[1]]), 6)
+})
diff --git a/tests/testthat/test-hierarchical.R b/tests/testthat/test-hierarchical.R
index efbdf14..fa13265 100644
--- a/tests/testthat/test-hierarchical.R
+++ b/tests/testthat/test-hierarchical.R
@@ -3,13 +3,13 @@ context("hierarchical")
test_that("hierarchical data is created correctly when you have a vector variable that is of length N per level",{
hierarchy <- fabricate(
- regions = level(N = 3, gdp = rnorm(N)),
- districts = level(
+ regions = add_level(N = 3, gdp = rnorm(N)),
+ districts = add_level(
N = 2,
var1 = c("recent", "ancient"),
var2 = ifelse(var1 == "recent", gdp, 5)
),
- cities = level(N = 2, subways = rnorm(N, mean = gdp))
+ cities = add_level(N = 2, subways = rnorm(N, mean = gdp))
)
df_2 <- unique(hierarchy[,c("regions", "var2")])
@@ -25,17 +25,17 @@ df_2 <- unique(hierarchy[,c("regions", "var2")])
test_that("creating variables", {
population <- fabricate(
- block = level(
+ block = add_level(
N = 5,
block_effect = rnorm(N)
),
- individuals = level(N = 2, noise = rnorm(N))
+ individuals = add_level(N = 2, noise = rnorm(N))
)
expect_error(fabricate(
N = 10,
noise = rnorm(N),
- block = level(
+ block = add_level(
N = 5,
block_effect = rnorm(N)
)
diff --git a/tests/testthat/test-resampling.R b/tests/testthat/test-resampling.R
index 8c79340..0b69fba 100644
--- a/tests/testthat/test-resampling.R
+++ b/tests/testthat/test-resampling.R
@@ -2,8 +2,8 @@ context("Resampling")
test_that("Resampling", {
two_levels <- fabricate(
- regions = level(N = 5, gdp = rnorm(N)),
- cities = level(N = 5, subways = rnorm(N, mean = gdp))
+ regions = add_level(N = 5, gdp = rnorm(N)),
+ cities = add_level(N = 5, subways = rnorm(N, mean = gdp))
)
# Example with data.table codepath
@@ -24,8 +24,8 @@ test_that("Resampling", {
test_that("Error handling of Resampling", {
two_levels <- fabricate(
- regions = level(N = 5, gdp = rnorm(N)),
- cities = level(N = sample(1:5), subways = rnorm(N, mean = gdp))
+ regions = add_level(N = 5, gdp = rnorm(N)),
+ cities = add_level(N = sample(1:5), subways = rnorm(N, mean = gdp))
)
resampled_two_levels <- resample_data(two_levels) # Missing N
@@ -36,14 +36,20 @@ test_that("Error handling of Resampling", {
expect_error(resample_data(two_levels, c(100, 10), ID_labels = c("regions")))
# Negative N
expect_error(resample_data(two_levels, c(-1), ID_labels = c("regions")))
- # Non-numeric
+ # Non-numeric N
expect_error(resample_data(two_levels, c("hello world"), ID_labels = c("regions")))
+ # Non-numeric N in direct call of resample_single_level. This is unlikely to
+ # arise normally since we don't export it and the code paths that call it have
+ # separate error handling
+ expect_error(resample_single_level(two_levels, N=c(1, 2), ID_label = "regions"))
+ expect_error(resample_single_level(two_levels, N=1.5, ID_label = "regions"))
+ expect_error(resample_single_level(two_levels, N="hello", ID_label = "regions"))
})
test_that("Direct resample_single_level", {
two_levels <- fabricate(
- regions = level(N = 5, gdp = rnorm(N)),
- cities = level(N = sample(1:5), subways = rnorm(N, mean = gdp))
+ regions = add_level(N = 5, gdp = rnorm(N)),
+ cities = add_level(N = sample(1:5), subways = rnorm(N, mean = gdp))
)
null_data = two_levels[two_levels$gdp > 100, ]
@@ -55,15 +61,45 @@ test_that("Direct resample_single_level", {
expect_error(resample_single_level(two_levels, ID_label="invalid-id", N=10))
})
+test_that("Extremely deep resampling", {
+ rect_data = fabricate(
+ N = 10,
+ xA = 1:10,
+ xB = 11:20,
+ xC = 21:30,
+ xD = 31:40,
+ xE = 41:50,
+ xF = 51:60,
+ xG = 61:70,
+ xH = 71:80,
+ xI = 81:90,
+ xJ = 91:100,
+ xK = 101:110
+ )
+
+ expect_error(resample_data(rect_data,
+ N = c(xA = 5,
+ xB = 3,
+ xC = 6,
+ xD = 7,
+ xE = 3,
+ xF = 1,
+ xG = 2,
+ xH = ALL,
+ xI = 2,
+ xJ = 4,
+ xK = 9)))
+})
+
test_that("Extremely high volume data creation.", {
skip("Slows build substantially.")
deep_dive_data = fabricate(
- countries = level(N = 100, gdp = rlnorm(N)),
- states = level(N = 50, population = rlnorm(N)),
- cities = level(N = 50, holiday = runif(N, 1, 365)),
- neighborhoods = level(N = 5, stoplights = draw_binary(x=0.5, N)),
- houses = level(N = 5, population = runif(N, 1, 5)),
- people = level(N = population, sex = ifelse(draw_binary(x=0.5, N), "M", "F"))
+ countries = add_level(N = 100, gdp = rlnorm(N)),
+ states = add_level(N = 50, population = rlnorm(N)),
+ cities = add_level(N = 50, holiday = runif(N, 1, 365)),
+ neighborhoods = add_level(N = 5, stoplights = draw_binary(x=0.5, N)),
+ houses = add_level(N = 5, population = runif(N, 1, 5)),
+ people = add_level(N = population, sex = ifelse(draw_binary(x=0.5, N), "M", "F"))
)
test_resample = resample_data(deep_dive_data,
@@ -73,8 +109,8 @@ test_that("Extremely high volume data creation.", {
test_that("Providing ID_labels through names of N.", {
two_levels <- fabricate(
- regions = level(N = 5, gdp = rnorm(N)),
- cities = level(N = sample(1:5), subways = rnorm(N, mean = gdp))
+ regions = add_level(N = 5, gdp = rnorm(N)),
+ cities = add_level(N = sample(1:5), subways = rnorm(N, mean = gdp))
)
resample_data(two_levels, N=c(regions=3, cities=5))
@@ -85,18 +121,23 @@ test_that("Providing ID_labels through names of N.", {
expect_error(resample_data(two_levels,
N=c(invalidid=3, cities=5)))
+ expect_error(resample_data(two_levels,
+ N=c(3, cities=5)))
+
expect_error(resample_data(two_levels,
N=c(3, 5)))
})
test_that("Passthrough resampling.", {
two_levels <- fabricate(
- regions = level(N = 5, gdp = rnorm(N)),
- cities = level(N = sample(1:5), subways = rnorm(N, mean = gdp))
+ regions = add_level(N = 5, gdp = rnorm(N)),
+ cities = add_level(N = sample(1:5), subways = rnorm(N, mean = gdp))
)
resample_data(two_levels, N=c(regions=ALL, cities=3))
# Warning when final level resampled has passthrough -- this is superfluous
expect_warning(resample_data(two_levels, N=c(regions=ALL, cities=ALL)))
+
+
})
diff --git a/tests/testthat/test-start-with-existing-data.R b/tests/testthat/test-start-with-existing-data.R
index 858e9fa..32f348f 100644
--- a/tests/testthat/test-start-with-existing-data.R
+++ b/tests/testthat/test-start-with-existing-data.R
@@ -1,17 +1,22 @@
context("Start with existing multi-level data and add variables")
+test_that("Import data plus single-level add variables.", {
+ result = fabricate(mtcars, zort = drat * 2)
+ expect_equal(dim(result), c(32, 13))
+})
+
test_that("Start with existing multi-level data and add variables",{
user_data <-
fabricate(
- regions = level(N = 5, gdp = rnorm(N)))
+ regions = add_level(N = 5, gdp = rnorm(N)))
expect_equal(dim(user_data), c(5, 2))
user_data <-
fabricate(
- regions = level(N = 5, gdp = rnorm(N)),
- cities = level(N = sample(1:5), subways = rnorm(N, mean = gdp)))
+ regions = add_level(N = 5, gdp = rnorm(N)),
+ cities = add_level(N = sample(1:5), subways = rnorm(N, mean = gdp)))
expect_equal(dim(user_data), c(15, 4))
@@ -19,31 +24,38 @@ test_that("Start with existing multi-level data and add variables",{
user_data_2 <-
fabricate(data = user_data,
- regions = level(rob = paste0(regions, "r")))
+ regions = modify_level(rob = paste0(regions, "r")))
expect_equal(dim(user_data_2), c(15, 5))
## add a variable at the cities level
user_data_3 <-
fabricate(data = user_data,
- cities = level(rob = paste0(cities, "c")))
+ cities = modify_level(rob = paste0(cities, "c")))
expect_equal(dim(user_data_3), c(15, 5))
user_data_4 <-
fabricate(data = user_data,
- regions = level(rob = paste0(regions, "r")),
- cities = level(bob = paste0(cities, "c")))
+ regions = modify_level(rob = paste0(regions, "r")),
+ cities = modify_level(bob = paste0(cities, "c")))
expect_equal(dim(user_data_4), c(15, 6))
user_data_5 <-
fabricate(data = user_data,
- regions = level(rob = paste0(regions, "r")),
- cities = level(bob = paste0(cities, "c")),
- neighborhoods = level(N = 10, tmp = rnorm(N)))
+ regions = modify_level(rob = paste0(regions, "r")),
+ cities = modify_level(bob = paste0(cities, "c")),
+ neighborhoods = add_level(N = 10, tmp = rnorm(N)))
expect_equal(dim(user_data_5), c(150, 8))
})
+
+test_that("Modify var at wrong level", {
+ df = fabricate(country = add_level(N = 50, population = runif(N, 10000, 20000)),
+ state = add_level(N = 10, latitude = runif(N, 40, 50)),
+ town = add_level(N = 5, stop_lights = draw_binary(x = 0.7, N=N)))
+ expect_error(fabricate(df, state = modify_level(crime = 0.5 + stop_lights + latitude)))
+})
diff --git a/tests/testthat/test-variables.R b/tests/testthat/test-variables.R
index 496deb8..f31d435 100644
--- a/tests/testthat/test-variables.R
+++ b/tests/testthat/test-variables.R
@@ -2,14 +2,14 @@ context("Variable functions")
test_that("Variable functions", {
# Single-level data, logit link, inherit or implicit N
- fabricate(my_level = level(
+ fabricate(my_level = add_level(
N = 10,
Y1 = rnorm(N),
Y2 = draw_binary(Y1, link = "logit")
))
# Single level, count, inherit or implicit N
- fabricate(my_level = level(
+ fabricate(my_level = add_level(
N = 10,
Y1 = rnorm(N, 5),
Y2 = draw_discrete(Y1, type = "count", k = 3)
@@ -143,14 +143,8 @@ test_that("Ordered data invalid tests", {
breaks=NA, break_labels=NA)) # Need to specify breaks
expect_error(draw_discrete(x=rnorm(5), type="ordered",
breaks=c("invalid", "break", "test"), break_labels=NA)) # Non-numeric breaks
- expect_error(draw_discrete(x=rnorm(5), type="ordered",
- breaks=c(1, 2), break_labels=NA)) # Insufficient number of breaks
expect_error(draw_discrete(x=rnorm(5), type="ordered",
breaks=c(1, 3, 2), break_labels=NA)) # Breaks out of order
- expect_error(draw_discrete(x=rnorm(5), type="ordered",
- breaks=c(10, 20, 30), break_labels=NA)) # Break endpoints above data
- expect_error(draw_discrete(x=rnorm(5), type="ordered",
- breaks=c(-50, -40, -30), break_labels=NA)) # Break endpoints below data
expect_error(draw_discrete(x=rnorm(5), type="ordered",
breaks=matrix(rep(c(0, 1, 2), 3), byrow=TRUE, ncol=3, nrow=3))) # Non-vector breaks
expect_error(draw_discrete(x=rnorm(5), type="ordered",
@@ -170,3 +164,103 @@ test_that("Ordered data valid tests", {
break_labels = c("A", "B"),
link="probit")
})
+
+test_that("Binary ICCs", {
+ clusters = rep(1:5, 10)
+ # Single probability
+ draw_binary_icc(clusters = clusters)
+ # Probability = length(cluster ids)
+ draw_binary_icc(x = c(0.3, 0.5, 0.7, 0.8, 0.9), clusters = clusters)
+
+ # Invalid cluster IDs
+ expect_error(draw_binary_icc(clusters = data.frame(X=1:10, Y=1:10)))
+ # X doesn't match cluster IDs
+ expect_error(draw_binary_icc(x = c(0.5, 0.8), clusters = clusters))
+ # X isn't a vector
+ expect_error(draw_binary_icc(x = data.frame(j = c(0.1, 0.2),
+ k = c(0.2, 0.4),
+ m = c(0.3, 0.6),
+ o = c(0.4, 0.8),
+ p = c(0.5, 1.0)),
+ clusters = clusters))
+ # X isn't numeric
+ expect_error(draw_binary_icc(x = "hello", clusters = clusters))
+ # X isn't a probability
+ expect_error(draw_binary_icc(x = -0.5, clusters = clusters))
+ # rho isn't a single number
+ expect_error(draw_binary_icc(clusters = clusters, rho = c(0.5, 0.8)))
+ # rho isn't a probability
+ expect_error(draw_binary_icc(clusters = clusters, rho = 2))
+ # rho isn't a number
+ expect_error(draw_binary_icc(clusters = clusters, rho = "hello"))
+ # Non-numeric N
+ expect_error(draw_binary_icc(clusters = clusters, N = "hello"))
+ # N provided but doesn't match
+ expect_error(draw_binary_icc(clusters = clusters, N = 20))
+})
+
+test_that("Likert data example", {
+ set.seed(19861108)
+ latent = rnorm(n=100, mean=3, sd=10)
+ cutpoints = c(-15, -7, -3, 3, 7, 15)
+ likert = draw_discrete(x=latent,
+ type="ordered",
+ breaks = cutpoints)
+ expect_equal(length(unique(likert)), 7)
+ expect_equal(max(likert), 7)
+ expect_equal(min(likert), 1)
+
+ draw_discrete(x=latent,
+ type="ordered",
+ breaks = cutpoints,
+ break_labels = c("Strongly Disagree",
+ "Disagree",
+ "Lean Disagree",
+ "No Opinion",
+ "Lean Agree",
+ "Agree",
+ "Strongly Agree"))
+})
+
+test_that("Normal ICC", {
+ clusters = rep(1:5, 10)
+ # Single mean
+ draw_normal_icc(clusters = clusters)
+ # Means = length(cluster ids)
+ draw_normal_icc(x = c(-1, -0.5, 0, 0.5, 1), clusters = clusters)
+
+ # Invalid cluster IDs
+ expect_error(draw_normal_icc(clusters = data.frame(X=1:10, Y=1:10)))
+ # X doesn't match cluster IDs
+ expect_error(draw_normal_icc(x = c(0.5, 0.8), clusters = clusters))
+ # X isn't a vector
+ expect_error(draw_normal_icc(x = data.frame(j = c(0.1, 0.2),
+ k = c(0.2, 0.4),
+ m = c(0.3, 0.6),
+ o = c(0.4, 0.8),
+ p = c(0.5, 1.0)),
+ clusters = clusters))
+ # X isn't numeric
+ expect_error(draw_normal_icc(x = "hello", clusters = clusters))
+ # rho isn't a single number
+ expect_error(draw_normal_icc(clusters = clusters, rho = c(0.5, 0.8)))
+ # rho isn't a probability
+ expect_error(draw_normal_icc(clusters = clusters, rho = 2))
+ # rho isn't a number
+ expect_error(draw_normal_icc(clusters = clusters, rho = "hello"))
+ # Non-numeric N
+ expect_error(draw_normal_icc(clusters = clusters, N = "hello"))
+ # N provided but doesn't match
+ expect_error(draw_normal_icc(clusters = clusters, N = 20))
+ # SD is wrong length
+ expect_error(draw_normal_icc(clusters = clusters, sd = c(1, 2)))
+ # SD is non-numeric
+ expect_error(draw_normal_icc(clusters = clusters, sd = "hello"))
+ # SD is not a vector
+ expect_error(draw_normal_icc(sd = data.frame(j = c(0.1, 0.2),
+ k = c(0.2, 0.4),
+ m = c(0.3, 0.6),
+ o = c(0.4, 0.8),
+ p = c(0.5, 1.0)),
+ clusters = clusters))
+})
diff --git a/vignettes/advanced_features.Rmd b/vignettes/advanced_features.Rmd
index c6627e8..d3163af 100644
--- a/vignettes/advanced_features.Rmd
+++ b/vignettes/advanced_features.Rmd
@@ -16,13 +16,13 @@ library(fabricatr)
# More complicated level creation with variable numbers of observations
-[`level()`](../reference/level.html) can be used to create more complicated patterns of nesting. For example, when creating lower level data, it is possible to use a different `N` for each of the values of the higher level data:
+[`add_level()`](../reference/add_level.html) can be used to create more complicated patterns of nesting. For example, when creating lower level data, it is possible to use a different `N` for each of the values of the higher level data:
```{r echo=TRUE, results="hide"}
variable_data <-
fabricate(
- cities = level(N = 2, elevation = runif(n = N, min = 1000, max = 2000)),
- citizens = level(N = c(2, 4), age = runif(N, 18, 70))
+ cities = add_level(N = 2, elevation = runif(n = N, min = 1000, max = 2000)),
+ citizens = add_level(N = c(2, 4), age = runif(N, 18, 70))
)
variable_data
```
@@ -30,16 +30,16 @@ variable_data
knitr::kable(variable_data)
```
-Here, each city has a different number of citizens. And the value of `N` used to create the age variable automatically updates as needed. The result is a dataset with 6 citizens, 2 in the first city and 4 in the second. As long as N is either a number, or a vector of the same length of the current lowest level of the data, [`level()`](../reference/level.html) will know what to do.
+Here, each city has a different number of citizens. And the value of `N` used to create the age variable automatically updates as needed. The result is a dataset with 6 citizens, 2 in the first city and 4 in the second. As long as N is either a number, or a vector of the same length of the current lowest level of the data, [`add_level()`](../reference/add_level.html) will know what to do.
It is also possible to provide a function to N, enabling a *random* number of citizens per city:
```{r echo=TRUE, results="hide"}
my_data <-
fabricate(
- cities = level(N = 2, elevation = runif(n = N, min = 1000, max = 2000)),
- citizens = level(N = sample(1:6, size = 2, replace = TRUE),
- age = runif(N, 18, 70))
+ cities = add_level(N = 2, elevation = runif(n = N, min = 1000, max = 2000)),
+ citizens = add_level(N = sample(1:6, size = 2, replace = TRUE),
+ age = runif(N, 18, 70))
)
my_data
```
@@ -53,8 +53,8 @@ Finally, it is possible to define `N` on the basis of higher level variables the
```{r echo=TRUE, results="hide"}
variable_n_function = fabricate(
- cities = level(N = 5, population = runif(N, 10, 200)),
- citizens = level(N = round(population * 0.3))
+ cities = add_level(N = 5, population = runif(N, 10, 200)),
+ citizens = add_level(N = round(population * 0.3))
)
head(variable_n_function)
```
@@ -70,10 +70,10 @@ You may want to include the mean value of a variable within a group defined by a
```{r echo=TRUE, results="hide"}
ave_example = fabricate(
- cities = level(N = 2),
- citizens = level(N = 1:2,
- income = rnorm(N),
- income_mean_city = ave(income, cities))
+ cities = add_level(N = 2),
+ citizens = add_level(N = 1:2,
+ income = rnorm(N),
+ income_mean_city = ave(income, cities))
)
ave_example
```
@@ -92,8 +92,8 @@ library(dplyr)
my_data <-
fabricate(
- cities = level(N = 2, elevation = runif(n = N, min = 1000, max = 2000)),
- citizens = level(N = c(2, 3), age = runif(N, 18, 70))
+ cities = add_level(N = 2, elevation = runif(n = N, min = 1000, max = 2000)),
+ citizens = add_level(N = c(2, 3), age = runif(N, 18, 70))
) %>%
group_by(cities) %>%
mutate(pop = n())
@@ -106,8 +106,8 @@ knitr::kable(my_data)
```{r echo=TRUE, results="hide"}
my_data <-
-data_frame(Y = sample(1:10, 2)) %>%
- fabricate(lower_level = level(N = 3, Y2 = Y + rnorm(N)))
+data_frame(Y = sample(1:10, 2)) %>%
+ fabricate(lower_level = add_level(N = 3, Y2 = Y + rnorm(N)))
my_data
```
```{r echo=FALSE}
diff --git a/vignettes/getting_started.Rmd b/vignettes/getting_started.Rmd
index 65ff1a6..a285685 100644
--- a/vignettes/getting_started.Rmd
+++ b/vignettes/getting_started.Rmd
@@ -66,10 +66,10 @@ The most powerful use of **fabricatr** is to create hierarchical ("nested") data
```{r echo=TRUE, results="hide"}
country_data <-
fabricate(
- countries = level(N = 5,
+ countries = add_level(N = 5,
gdp_per_capita = runif(N, min=10000, max=50000),
life_expectancy = 50 + runif(N, 10, 20) + ((gdp_per_capita > 30000) * 10)),
- provinces = level(N = 10,
+ provinces = add_level(N = 10,
has_nat_resources = draw_discrete(x=0.3, N=N, type="bernoulli"),
has_manufacturing = draw_discrete(x=0.7, N=N, type="bernoulli"))
)
@@ -80,7 +80,7 @@ knitr::kable(head(country_data), format.args=list(big.mark = ","))
```
-Several things can be observed in this example. First, fabricate knows that your second [`level()`](../reference/level.html) command will be nested under the first level of data. Each level gets its own ID variable, in addition to the variables you create. Second, the meaning of the variable "N" changes. During the [`level()`](../reference/level.html) call for countries, N is 5. During the [`level()`](../reference/level.html) call for provinces, N is 10. And the resulting data, of course, has 50 observations.
+Several things can be observed in this example. First, fabricate knows that your second [`add_level()`](../reference/add_level.html) command will be nested under the first level of data. Each level gets its own ID variable, in addition to the variables you create. Second, the meaning of the variable "N" changes. During the [`add_level()`](../reference/add_level.html) call for countries, N is 5. During the [`add_level()`](../reference/level.html) call for provinces, N is 10. And the resulting data, of course, has 50 observations.
Finally, the province-level variables are created using the [`draw_discrete()`](../reference/draw_discrete.html) function. This is a function provided by **fabricatr** to make simulating discrete random variables simple. When you simulate your own data, you can use **fabricatr**'s functions, R's built-ins, or any custom functions you wish. [`draw_discrete()`](../reference/draw_discrete.html) is explained in [our tutorial on variable generation using **fabricatr**](variable_generation.html)
@@ -94,11 +94,11 @@ Imagine importing the country-province data simulated in the previous example. B
citizen_data <-
fabricate(
data = country_data,
- citizens = level(N=10,
- salary = rnorm(N,
- mean = gdp_per_capita +
- has_nat_resources * 5000 +
- has_manufacturing * 5000,
+ citizens = add_level(N=10,
+ salary = rnorm(N,
+ mean = gdp_per_capita +
+ has_nat_resources * 5000 +
+ has_manufacturing * 5000,
sd = 10000)))
head(citizen_data)
```
@@ -106,14 +106,13 @@ head(citizen_data)
knitr::kable(head(citizen_data), format.args=list(big.mark = ","))
```
-
In this example, we add a third level of data; for each of our 50 country-province observations, we now have 10 citizen-level observations. Citizen-level covariates like salary can draw from both the country-level covariate and the province-level covariate.
Notice that the syntax for adding a new nested level to existing data is different than the syntax for adding new variables to the original dataset.
# Modifying existing levels
-Suppose you have hierarchical data, and wish to simulate variables at a higher level of aggregation. For example, imagine you import a dataset containing citizens within countries, but you wish to simulate additional country-level variables. In **fabricatr**, you can do this using the [`level()`](../reference/level.html) command.
+Suppose you have hierarchical data, and wish to simulate variables at a higher level of aggregation. For example, imagine you import a dataset containing citizens within countries, but you wish to simulate additional country-level variables. In **fabricatr**, you can do this using the [`modify_level()`](../reference/modify_level.html) command.
Let's use our country-province data from earlier:
@@ -121,7 +120,7 @@ Let's use our country-province data from earlier:
new_country_data <-
fabricate(
data = country_data,
- countries = level(avg_temp = runif(N, 30, 80))
+ countries = modify_level(avg_temp = runif(N, 30, 80))
)
head(new_country_data)
@@ -131,9 +130,7 @@ knitr::kable(head(new_country_data), format.args=list(big.mark = ","))
```
-How does [`level()`](../reference/level.html) know whether to modify your data or add a new level? [`level()`](../reference/level.html) uses contextual information -- if the name you provide to your [`level()`](../reference/level.html) call is already a field that exists in your data set, [`level()`](../reference/level.html) will treat this as a request to modify this level of data. If, on the other hand, you provide a name not used in the data set, [`level()`](../reference/level.html) will assume you mean to add nested data under the existing data.
-
-We can observe that the new variable is created at the level of aggregation you chose -- countries. Also, although N is not specified anywhere, [`level()`](../reference/level.html) knows how large N should be based on the number of countries it finds in the dataset. It is important, then, to ensure that the [`level()`](../reference/level.html) command is correctly assigned to the level of interest.
+We can observe that the new variable is created at the level of aggregation you chose -- countries. Also, although N is not specified anywhere, [`modify_level()`](../reference/level.html) knows how large N should be based on the number of countries it finds in the dataset. It is important, then, to ensure that the [`modify_level()`](../reference/level.html) command is correctly assigned to the level of interest.
We can also modify more than one level. Recalling our country-province-citizen data from above, the following process is possible:
@@ -141,22 +138,22 @@ We can also modify more than one level. Recalling our country-province-citizen d
new_citizen_data <-
fabricate(
data = citizen_data,
- countries = level(avg_temp = runif(N, 30, 80)),
- provinces = level(conflict_zone = draw_discrete(N,
- x=0.2 + has_nat_resources * 0.3,
- type="binary"),
- infant_mortality = runif(N, 0, 10) +
- conflict_zone * 10 +
- (avg_temp > 70) * 10),
- citizens = level(college_degree = draw_discrete(N,
- x=0.4 - (0.3 * conflict_zone),
- type="binary"))
+ countries = modify_level(avg_temp = runif(N, 30, 80)),
+ provinces = modify_level(conflict_zone = draw_discrete(N,
+ x=0.2 + has_nat_resources * 0.3,
+ type="binary"),
+ infant_mortality = runif(N, 0, 10) +
+ conflict_zone * 10 +
+ (avg_temp > 70) * 10),
+ citizens = modify_level(college_degree = draw_discrete(N,
+ x=0.4 - (0.3 * conflict_zone),
+ type="binary"))
)
```
-Before assessing what this tells us about [`level()`](../reference/level.html), let's consider what the data simulated does. It creates a new variable at the country level, for a country level average temperature. Subsequently, it creates a province level binary indicator for whether the province is an active conflict site. Provinces that have natural resources are more likely to be in conflict in this simulation, drawing on conclusions from literature on "resource curses". The infant mortality rate for the province is able to depend both on province level data we have just generated, and country-level data: it is higher in high-temperature areas (reflecting literature on increased disease burden near the equator) and also higher in conflict zones. Citizens access to education is also random, but depends on whether they live in a conflict area.
+Before assessing what this tells us about [`modify_level()`](../reference/modify_level.html), let's consider what the data simulated does. It creates a new variable at the country level, for a country level average temperature. Subsequently, it creates a province level binary indicator for whether the province is an active conflict site. Provinces that have natural resources are more likely to be in conflict in this simulation, drawing on conclusions from literature on "resource curses". The infant mortality rate for the province is able to depend both on province level data we have just generated, and country-level data: it is higher in high-temperature areas (reflecting literature on increased disease burden near the equator) and also higher in conflict zones. Citizens access to education is also random, but depends on whether they live in a conflict area.
-There are a lot of things to learn from this example. First, it's possible to modify multiple levels. Any new variable created will automatically propagate to the lower level data according -- by setting an average temperature for a country, all provinces, and all citizens of those provinces, have the value for the country. Values created from one [`level()`](../reference/level.html) call can be used in subsequent variables of the same call, or subsequent calls.
+There are a lot of things to learn from this example. First, it's possible to modify multiple levels. Any new variable created will automatically propagate to the lower level data according -- by setting an average temperature for a country, all provinces, and all citizens of those provinces, have the value for the country. Values created from one [`modify_level()`](../reference/level.html) call can be used in subsequent variables of the same call, or subsequent calls.
Again, we see the use of [`draw_discrete()`](../reference/draw_discrete.html). Using this function is covered in our tutorial on [generating discrete random variables](variable_generation.html), linked below.
diff --git a/vignettes/resampling.Rmd b/vignettes/resampling.Rmd
index 5572404..8604e39 100644
--- a/vignettes/resampling.Rmd
+++ b/vignettes/resampling.Rmd
@@ -54,8 +54,8 @@ Consider this example, which takes a dataset containing 2 cities of 3 citizens,
```{r echo=TRUE, results="hide"}
my_data <-
fabricate(
- cities = level(N = 2, elevation = runif(n = N, min = 1000, max = 2000)),
- citizens = level(N = 3, age = runif(N, 18, 70))
+ cities = add_level(N = 2, elevation = runif(n = N, min = 1000, max = 2000)),
+ citizens = add_level(N = 3, age = runif(N, 18, 70))
)
my_data_2 <- resample_data(my_data,
@@ -68,3 +68,20 @@ knitr::kable(my_data_2)
```
[`resample_data()`](../reference/resample_data.html) will first select the cities to be resampled. Then, for each city, it will continue by selecting the citizens to be resampled. If a higher level unit is used more than once (for example, the same city being chosen twice), and a lower level is subsequently resampled, the choices of which units to keep for the lower level will differ for each copy of the higher level. In this example, if city 1 is chosen twice, then the sets of five citizens chosen for each copy of the city 1 will differ.
+
+You can also specify the levels you wish to resample from using the name arguents to the `N` parameter, like in this example which does exactly the same thing as the previous example, but specifies the level names in a different way:
+
+```{r echo=TRUE, results="hide"}
+my_data <-
+ fabricate(
+ cities = add_level(N = 2, elevation = runif(n = N, min = 1000, max = 2000)),
+ citizens = add_level(N = 3, age = runif(N, 18, 70))
+ )
+
+my_data_2 <- resample_data(my_data,
+ N = c(cities=3, citizens=5))
+my_data_2
+```
+```{r echo=FALSE}
+knitr::kable(my_data_2)
+```
diff --git a/vignettes/variable_generation.Rmd b/vignettes/variable_generation.Rmd
index 44f622e..5dcd66e 100644
--- a/vignettes/variable_generation.Rmd
+++ b/vignettes/variable_generation.Rmd
@@ -18,7 +18,7 @@ library(fabricatr)
**fabricatr** provides a convenient helper function, [`draw_discrete()`](../reference/draw_discrete.html), which you can use to generate discrete random variables far more easily than using R's built-in data generation mechanisms. Below, we introduce you to the types of data you can generate using [`draw_discrete()`](../reference/draw_discrete.html).
-# Binary and binomial outcomes
+## Binary and binomial outcomes
The simplest possible type of data, and [`draw_discrete()`](../reference/draw_discrete.html)'s default, is a binary random variable (also called a bernoulli random variable). Generating a binary random variable requires only one parameter `x` which specifies the probability that outcomes drawn from this variable are equal to 1. By default, [`draw_discrete()`](../reference/draw_discrete.html) will generate `N = length(x)` draws. `N` can also be specified explicitly. Consider these examples:
@@ -59,7 +59,7 @@ bernoulli_probit = fabricate(N = 3, x = 10*rnorm(N),
link = "probit"))
```
-# Ordered outcomes
+## Ordered outcomes
Some researchers may be interested in generating ordered outcomes -- for example, Likert scale outcomes. This can be done using the "ordered" type. Ordered variables require a vector of breakpoints, supplied as the argument `breaks` -- points at which the underlying latent variable switches from category to category. The first break should always be below the lower bound of the data, while the final break should always be above the upper bound of the data.
@@ -88,7 +88,7 @@ ordered_probit_example = fabricate(N = 3,
)
```
-# Count outcomes
+## Count outcomes
[`draw_discrete()`](../reference/draw_discrete.html) supports Poisson-distributed count outcomes. These require that the user specify the parameter `x`, equal to the Poisson distribution mean (often referred to as `lambda`).
@@ -98,7 +98,7 @@ count_outcome_example = fabricate(N = 3,
count = draw_discrete(x, type = "count"))
```
-# Categorical data
+## Categorical data
[`draw_discrete()`](../reference/draw_discrete.html) can also generate non-ordered, categorical data. Users must provide a vector of probabilities for each category (or a matrix, if each observation should have separate probabilities), as well as setting `type` to be "categorical".
@@ -127,3 +127,118 @@ warn_draw_discrete_example = fabricate(N = 6,
```
"categorical" variables can also use link functions, for example to generate multinomial probit data.
+
+# Fabricating cluster-correlated random variables.
+
+We also provide helper functions to generate cluster-correlated random variables with fixed intra-cluster correlation (ICC) values. Our two functions `draw_binary_icc()` and `draw_normal_icc()` allow you to generate both discrete binary data with fixed ICCs and normal data with fixed ICCs.
+
+## Binary data with fixed ICCs
+
+`draw_binary_icc()` takes three required arguments: `x`, a probability or vector of probabilities which determine the chance a given observation will be a 1; `clusters`, a map of units to clusters (required to generate the correlation structure); and `rho`, the fixed intra-cluster correlation (from 0 to 1). Users may optionally specify `N`; if it is not specified, `draw_binary_icc()` will determine it based on the length of the `clusters` vector.
+
+Consider the following example, which models whether individuals smoke:
+
+```{r echo=FALSE}
+set.seed(19861108)
+```
+```{r echo=TRUE, results="hide"}
+# 100 individual population, 10 each in each of 10 clusters
+clusters = rep(1:10, 10)
+
+# Individuals have a 20% chance of smoking, but clusters are highly correlated
+# in their tendency to smoke
+smoker = draw_binary_icc(x = 0.2,
+ clusters = clusters,
+ rho = 0.7)
+
+# Observe distribution of smokers and non-smokers
+table(smoker)
+```
+```{r echo=FALSE}
+knitr::kable(as.matrix(t(table(smoker))))
+```
+
+We see that approximately 20% of the population smokes, in line with our specification, but what patterns of heterogeneity do we see by cluster?
+
+```{r echo=TRUE, results="hide"}
+table(clusters, smoker)
+```
+```{r echo=FALSE}
+knitr::kable(table(clusters, smoker))
+```
+
+We observe that 7 clusters have no smokers at all, two clusters are overwhelming smokers, and one cluster is overwhelmingly non-smokers.
+
+We can also specify separate means for each cluster; but it is worth noting that the higher the ICC, the more the cluster mean will depart from the nominal cluster mean.
+
+If you do not specify a vector of probabilities or a correlation coefficient, the default values are probability 0.5 for each cluster and ICC (rho) of 0.5. If you do not specify cluster IDs, the function will return an error.
+
+## Normal data with fixed ICCs
+
+`draw_normal_icc()` takes four required arguments: `x`, a mean or vector of means, one for each cluster; `clusters`, a map of units to clusters (required to generate the correlation structure); `rho`, the fixed intra-cluster correlation coefficient; and `sd`, a standard deviation or vector of standard deviations, one for each cluster. Users can optionally specify `N`, a number of units, but if it is not supplied `draw_normal_icc()` will determine it based on the length of the `clusters` vector.
+
+If `sd` is not supplied, each cluster will be assumed to have a within-cluster standard deviation of 1. If `x` is not supplied, each cluster will be assumed to be mean zero. If `rho` is not supplied, it will be set to 0.5.
+
+Here, we model student academic performance by cluster:
+```{r echo=FALSE}
+set.seed(19861108)
+```
+```{r echo=TRUE, results="hide"}
+# 100 students, 10 each in 10 clusters
+clusters = rep(1:10, 10)
+
+numeric_grade = draw_normal_icc(x = 80,
+ clusters = clusters,
+ rho = 0.5,
+ sd = 15)
+
+letter_grade = draw_discrete(x = numeric_grade,
+ type = "ordered",
+ breaks = c(-Inf, 60, 70, 80, 90, Inf),
+ break_labels = c("F", "D", "C", "B", "A"))
+
+mean(numeric_grade)
+```
+`r mean(numeric_grade)`
+
+The mean grade matches the population mean. Now let's look at the relationship between cluster and letter grade to observe the cluster pattern:
+
+```{r echo=TRUE, results="hide"}
+table(letter_grade, clusters)
+```
+```{r echo=FALSE}
+knitr::kable(table(clusters, letter_grade))
+```
+
+It is obvious upon inspection that some clusters are higher performing than others despite having identical cluster means in expectation.
+
+## Technical Appendix
+
+When generating binary data with a fixed ICC, we follow this formula, where $i$ is a cluster and $j$ is a unit in a cluster:
+
+$$
+\begin{aligned}
+ z_i &\sim \text{Bern}(p_i) \\
+ u_{ij} &\sim \text{Bern}(\sqrt{\rho}) \\
+ x_{ij} &=
+ \begin{cases}
+ x_{ij} \sim \text{Bern}(p_i) & \quad \text{if } u_{ij} = 1 \\
+ z_i & \quad \text{if } u_{ij} = 0
+ \end{cases}
+\end{aligned}
+$$
+
+In expectation, this guarantees an intra-cluster correlation of $\rho$ and a cluster proportion of $p_i$. This approach derives from Hossain, Akhtar and Chakraborti, Hrishikesh. "ICCBin: Facilitates Clustered Binary Data Generation, and Estimation of Intracluster Correlation Coefficient (ICC) for Binary Data", available on [https://cran.r-project.org/web/packages/ICCbin/index.html](CRAN) or [https://github.com/akhtarh/ICCbin](GitHub)
+
+When generating normal data with a fixed ICC, we follow this formula, again with $i$ as a cluster and $j$ as a unit in the cluster:
+
+$$
+\begin{aligned}
+ \sigma^2_{\alpha i} &= \frac{(\rho * \sigma^2_{\epsilon i})}{(1 - \rho)} \\
+ \alpha_i &\sim \mathcal{N}(0, \sigma^2_{\alpha i}) \\
+ \mu_{ij} &\sim \mathcal{N}(\mu_i, \sigma^2_{\epsilon i}) \\
+ x_{ij} &= \mu_{ij} + \alpha_i
+\end{aligned}
+$$
+
+In expectation, this approach guarantees an intra-cluster correlation of $\rho$, a cluster mean of $\mu_{i}$, and a cluster-level variance in error terms of $\sigma^2_{\epsilon i}$. This approach is specified on [https://stats.stackexchange.com/questions/263451/create-synthetic-data-with-a-given-intraclass-correlation-coefficient-icc](StatsExchange).