/ BicliqueRT Public

## History

280 lines (209 loc) · 16.8 KB

280 lines (209 loc) · 16.8 KB

# BicliqueRT

R package for randomization tests of causal effects under general interference. The full package is under construction, but the main functions are up and running!

This is an implementation of the clique-based randomization test developed in the paper A Graph-Theoretic Approach to Randomization Tests of Causal Effects Under General Interference. If used, please cite the paper.

## Installation

To use, first install devtools:

install.packages("devtools")

and then, install the package:

library(devtools)
install_github("dpuelz/BicliqueRT")

## General Work Flow of the Test

Suppose we are going to test the null: Yi(z)=Yi(z') for all i and all z,z' such that fi(z)~fi(z'), where fi(z) is the exposure of unit i under treatment assignment z, and fi(z)~fi(z') expresses the equivalence of two exposures.

The test consists of two parts as described in the paper: do biclique decomposition on the null exposure graph to find a conditional clique, and do randomization test on the conditional clique.

### 1. Biclique decomposition

The function in this step is biclique.decompose(Z, hypothesis, controls=list(method="greedy", mina=10, num_randomizations=2000), stop_Zobs=F). Z is the observed treatment assignment vector of length N where N is the number of units. hypothesis is a list that contains three functions specifying the experiment design and null hypothesis: design_fn, exposure_i and null_equiv.

• design_fn specifies the experiment design. It should be a function that returns a realization of treatment assignment for the whole sample. It can depends on other global variables such as the number of units. For example, if the design is that each unit has equal probability of 0.2 to receive the treatment independently, we can write design_fn = function() { rbinom(num_units, 1, prob=0.2) }. Note that the return of design_fn() should also be a vector of length N as Z is.
• exposure_i should be a function that returns exposure fi(z) of unit i under treatment z for the whole sample. The inputs of the function are an index i and a vector z. For example, if the exposure of i under z is the treatment it receives, then we can write exposure_i = function(z, i) { z[i] }. See examples below for more instances of exposure_i.
• null_equiv expresses the equivalence relationship between two exposures. It should be a function that takes two inputs from exposure_i and determines whether they are equivalent according to the null hypothesis. See examples below for more instances of null_equiv.

The list controls specifies parameters for the biclique decomposition algorithm. method could be either "bimax" or "greedy" that uses the two biclique decomposition algorithm. If "bimax" is used, minr and minc  should be supplied that specify the minimum number of units and assignments in the bicliques found by the algorithm. If "greedy" is used, mina  should be supplied. num_randomizations specifies the number of randomizations to perform. Larger the number, more computation time is needed but we may get a larger conditional clique to do randomization which gives the test more power.

stop_Zobs is either TRUE or FALSE that determines whether the biclique decomposition should end when we find a biclique that contains the observed treatment assignment vector. Setting it to TRUE can speed up the decomposition a bit but may not get the whole biclique decomposition of the null exposure graph.

The biclique.decompose function will return a (partial) biclique decomposition of the null exposure graph where Z is in one of them.

### 2. Randomization Test

The output of the biclique.decompose, written as biclique_decom, should be passed to clique_test(Y, Z, teststat, biclique_decom, alpha=0.05, one_sided=T) to do the randomization test. Here Y is the observed outcome vector of length N, Z is the observed treatment assignment vector of same length. alpha specifies the significance level.

teststat is a function that specifies the test statistic used in the conditional clique. The function should contain at least (with order) y, z, focal_unit_indicator as inputs, where y is the outcome vector, z is the treatment vector and focal_unit_indicator is a 0-1 vector indicating whether a unit is focal (=1) or not (=0). All three inputs should have length equal to number of units and have the same ordering. Other global variables can be used in the function, such as the ones about a network of interference.

We provide several default test statistics for no-interference null in Athey et al. 2018 Exact p-Values for Network Interference that can be generated using function gen_tstat(G, type), where G is the N by N adjacency matrix of a network, type could be one of "elc","score","htn". See section 5 in Athey et al. 2018 and the documentation of gen_tstat for a detailed description of the these test statistics.

Note that by default (one_sided=T), we calculate p-value by a one-sided p-value as in the function one_sided_test. This requires that the test statistic is defined in a way such that a large value of the test statistic provides evidence against the null hypothesis. Note that using the one-sided p-value does not necessarily restrict it to be a one-sided test. For example if we want to test that the means in two groups are the same (H0) vs not the same (H1), we can take the test statistic to be the absolute value of differences in means between the two groups and use the one-sided p-value. Now a large test statistic indicates rejection. However, the user can choose other statistics that do not necessarily satisfy this requirement and use the two-sided p-value by setting one_sided=F, which calculates the p-value by the function two_sided_test.

## Example: Spatial Interference

The following simulation example illustrates spatial inference on a small synthetic network with 500 nodes. We are testing the null that potential outcomes are the same for all units no matter (it is untreated but within a radius of a treated unit), or (it is untreated but not within a radius of any treated unit).

# generated network - 3 clusters of 2D Gaussians
# loads in the 500x500 matrix Dmat
# Dmat just encodes all pairwise Euclidean distances between network nodes, and
# this is used to define the spillover hypothesis below.

library(BicliqueRT)
set.seed(1)

N = 500 # number of units
thenetwork = out_example_network(N)
D = thenetwork$D # simulating an outcome vector and a treatment realization Y = rnorm(N) Z = rbinom(N, 1, prob=0.2) # simulation parameters num_randomizations = 5000 radius = 0.02 # To use the package: # 1. The design function: here the experimental design is Bernoulli with prob=0.2 design_fn = function() { rbinom(N, 1, prob=0.2) } # 2. The exposure function: exposure for each unit is (w_i, z_i) where # w_i = 1{\sum_{j\neq i} g_{ij}^r z_j > 0 } and g_{ij}^r = 1{d(i,j)<r} Gr = (D<radius) * 1; diag(Gr) = 0 exposure_i = function(z, i) { c(as.numeric(sum(Gr[i,]*z) > 0), z[i]) } # 3. The null null_hypothesis = list(c(0,0), c(1,0)) null_equiv = function(exposure_z1, exposure_z2) { (list(exposure_z1) %in% null_hypothesis) & (list(exposure_z2) %in% null_hypothesis) } # Then we can decompose the null exposure graph: H0 = list(design_fn=design_fn, exposure_i=exposure_i, null_equiv=null_equiv) bd = biclique.decompose(Z, H0, controls= list(method="greedy", mina=20, num_randomizations = 2e3)) m = bd$MNE # this gives the biclique decomposition

# To do randomization test, firstly generate a test statistic. Here we use the absolute value of differences in means between units with exposure (0,0) and exposure (1,0)
Tstat = function(y, z, is_focal) {
exposures = rep(0, N)
for (unit in 1:N) {
exposures[unit] = exposure_i(z, unit)[1]
}
stopifnot("all focals have same exposures" = (length(unique(exposures[is_focal]))>1) )
abs(mean(y[is_focal & (exposures == 1)]) - mean(y[is_focal & (exposures == 0)]))
}

# Then run the test
testout = clique_test(Y, Z, Tstat, bd)

The output testout is a list that contains p-value, the test statistic, the randomization distribution of the test statistic, test method and the conditional clique.

names(testout)
# [1] "p.value"            "statistic"          "statistic.dist"     "method"             "conditional.clique"
testout$p.value # p-value of the test ## Example: Clustered Interference The following simulation example illustrates clustered inference with 2000 units equally divided into 500 clusters. We assign at random one unit in half of the clusters to be treated, and we are testing that potential outcomes are the same for all units no matter (it is untreated in a cluster without any treated unit), or (it is untreated in a cluster with a treated unit). We follow the same procedure: library(BicliqueRT) set.seed(1) N = 2000 # total number of units K = 500 # total number of households, i.e., number of clusters housestruct = out_house_structure(N, K, T) # The design function: design_fn = function(){ treatment_list = out_treat_household(housestruct, K1 = K/2) # one unit from half of the households would be treated. treatment = out_Z_household(N, K, treatment_list, housestruct) return(treatment[,'treat']) } # The exposure function: exposure for each unit i is z_i + \sum_{j \in [i]} z_j where [i] represents the cluster i is in. exposure_i = function(z, i) { # find the household that i is in house_ind = cumsum(housestruct) which_house_i = which.min(house_ind < i) # find lower and upper index of [i] in z if (which_house_i == 1) {lower_ind = 1} else {lower_ind = house_ind[which_house_i-1] + 1} upper_ind = house_ind[which_house_i] # calculate exposure exposure_z = z[i] + sum(z[lower_ind:upper_ind]) exposure_z } # The null null_equiv = function(exposure_z1, exposure_z2) { ((exposure_z1 == 1) | (exposure_z1 == 0)) & ((exposure_z2 == 1) | (exposure_z2 == 0)) } # Generate a treatment realization and outcome Z = design_fn() # randomly get one realization # Generate exposure under the realized Z Z_exposure = rep(0, N); for (i in 1:N) { Z_exposure[i] = exposure_i(Z, i) } # Generate observed outcomes based on exposures under Z Y = out_bassefeller(N, K, Z_exposure, tau_main = 0.4, housestruct = housestruct)$Yobs
# here we assume that potential outcomes are 0.4 higher if an untreated unit is in a cluster
# with a treated unit compared to in a cluster without any treated unit,
# i.e., a spillover effect of 0.4 is assumed

# Do biclique decomposition on the null exposure graph
H0 = list(design_fn=design_fn, exposure_i=exposure_i, null_equiv=null_equiv)
bd = biclique.decompose(Z, H0, controls= list(method="greedy", mina=30, num_randomizations = 5e3))

# Define a test statistic, here we use the absolute value of differences in means between units with exposure 1 (untreated in treated cluster) and exposure 0 (untreated in untreated cluster)
Tstat = function(y, z, is_focal) {
exposures = rep(0, N)
for (unit in 1:N) {
exposures[unit] = exposure_i(z, unit)[1]
}
stopifnot("all focals have same exposures" = (length(unique(exposures[is_focal]))>1) )
abs(mean(y[is_focal & (exposures == 1)]) - mean(y[is_focal & (exposures == 0)]))
}

# Then run the test
testout = clique_test(Y, Z, Tstat, bd)