The Q Test

This is the github repo for the Q test proposed by Fan et al. (2022) that tests overidentifying restrictions (or instrumental variable validity) with high-dimensional data, which is robust to heteroskedastic errors. The method is called the Q test since it is based on estimation and inference for quadratic functionals of high-dimensional vectors. The paper is available at https://arxiv.org/abs/2205.00171.

Required packages

You can install the required package with

install.pacakges(c("glmnet,MASS,igraph"))

In addition, users need to install the "Rmosek" package manually. The instructions for installation are available at Installation of MOSEK Rmosek package.

"Lasso.R" and "Projection.R" contain necessary R functions for Lasso estimators and projection direction vectors for bias correction.

"QTest.R" is the function to implement the Q test.

"example.R" provides an example that is shown below.

Example

This is a basic example that shows you how to use the Q test.

rm(list = ls())

library(glmnet)
library(Rmosek)
library(MASS)
library(igraph)

source("Lasso.R")
source("Projection.R")
source("QTest.R")

n = 300
px = 150
pz = 100 

p = px + pz
Sigma = diag(p)

phi <- c(seq(0.1,0.5,0.1),rep(0,145))
psi <- c(seq(0.3,0.7,0.1),rep(0,145))

s1 = 10 # num of relevant IV
gamma <- matrix(c(rep(0.5,10),rep(0,pz-10)),ncol = 1)

beta = 1 

set.seed(2022)

## covariates and instruments 
W <- MASS::mvrnorm(n=n, rep(1, p), Sigma)
if (px == 0){
  X <- NULL
  Z <- W
}else{
  X <- W[,1:px]
  Z <- W[,(px+1):p]
}

## error terms 
err <- MASS::mvrnorm(n=n, rep(0, 2), matrix(1.5 * c(1, .5, .5,1),2))
e <- err[,1]
eps2 <- err[,2]

Test results with valid instruments

D <-  X%*%psi  + Z %*% gamma + eps2
Y <-   D*beta + X%*%phi + e

QTest(Y,D,Z,X) 
# $invalid
# [1] FALSE
# 
# $sig.level
# [1] 0.05
# 
# $pval
# [1] 0.6518306

Test results with invalid instruments

pi <-  c(seq(0.1,1,0.1),rep(0,90))
Y <- D*beta + X%*%phi + Z %*% pi + e

QTest(Y,D,Z,X) 
# $invalid
# [1] TRUE
# 
# $sig.level
# [1] 0.05
# 
# $pval
# [1] 9.769963e-15

Reference

Fan, Q., Guo, Z., & Mei, Z. (2022). Testing Overidentifying Restrictions with High-Dimensional Data and Heteroskedasticity. arXiv preprint arXiv:2205.00171.