Merge branch 'master' into DRtest

* master:
  Add zero dim input check (#130)
  Add guidance message for p/d input (#129)
  Add OR Forthcoming (#127)

README.md

@@ -3,7 +3,7 @@
 [![CRANstatus](https://www.r-pkg.org/badges/version/policytree)](https://cran.r-project.org/package=policytree)
 [![Build Status](https://dev.azure.com/grf-labs/grf/_apis/build/status/grf-labs.policytree?branchName=master)](https://dev.azure.com/grf-labs/grf/_build/latest?definitionId=1&branchName=master)
 
-A package for learning optimal policies via doubly robust empirical welfare maximization over trees. Many practical policy applications require interpretable predictions. For example, a drug prescription guide that follows a simple 2-question Yes/No checklist can be encoded as a depth 2 decision tree (does the patient have a heart condition - etc.). This package implements the multi-action doubly robust approach of Zhou et al. (2018) in the case where we want to learn policies that belong to the class of depth _k_ decision trees.
+A package for learning simple rule-based policies, where the rule takes the form of a shallow decision tree. Applications include settings which require interpretable predictions, such as for example a medical treatment prescription. This package uses doubly robust reward estimates from [grf](https://github.com/grf-labs/grf) to find a shallow, but globally optimal decision tree.
 
 Some helpful links for getting started:
 
@@ -21,18 +21,14 @@ install.packages("policytree")
 
 To install the latest development version from source:
 
-```
-git clone https://github.com/grf-labs/policytree.git && cd policytree
-Rscript -e 'install.packages("r-package/policytree", repos = NULL, type = "source")'
-
-# or
+```R
 devtools::install_github("grf-labs/policytree", subdir = "r-package/policytree")
 ```
 
 Installing from source requires a C++ 11 compiler (on Windows Rtools is required as well) together with the R packages
 `Rcpp` and `BH`.
 
-### Multi-action policy learning
+### Multi-action policy learning example
 ```r
 library(policytree)
 n <- 250
@@ -75,8 +71,8 @@ head(predict(opt.tree, X[-train, ]))
 ```
 
 ### Details
-* `policy_tree()`: fits a depth k tree by exhaustive search (_Nxp_ features on _Nxd_ actions). The optimal tree maximizes the sum of rewards.
-* `double_robust_scores()`: generic function computing doubly robust reward estimates for a subset of _grf_ forest types.
+* `policy_tree()`: fits a depth k tree by exhaustive search (_Nxp_ features on _Nxd_ actions). The optimal tree maximizes the sum of rewards. `hybrid_policy_tree()` employs a mix between a optimal/greedy approach and can be used to fit deeper trees.
+* `double_robust_scores()`: computes doubly robust reward estimates for a subset of _grf_ forest types.
 
 ### Contributing
 
@@ -102,5 +98,5 @@ Erik Sverdrup, Ayush Kanodia, Zhengyuan Zhou, Susan Athey, and Stefan Wager.
 [<a href="https://joss.theoj.org/papers/10.21105/joss.02232">paper</a>]
 
 Zhengyuan Zhou, Susan Athey, and Stefan Wager.
-<b>Offline Multi-Action Policy Learning: Generalization and Optimization.</b> 2018.
+<b>Offline Multi-Action Policy Learning: Generalization and Optimization.</b> <i> Operations Research</i>, forthcoming.
 [<a href="https://arxiv.org/abs/1810.04778">arxiv</a>]

experiments/README.md

@@ -13,5 +13,5 @@ Susan Athey and Stefan Wager.
 <a href="https://arxiv.org/abs/1702.02896">arxiv</a>]
 
 Zhengyuan Zhou, Susan Athey, and Stefan Wager.
-<b>Offline Multi-Action Policy Learning: Generalization and Optimization.</b> 2018.
+<b>Offline Multi-Action Policy Learning: Generalization and Optimization.</b> <i> Operations Research</i>, forthcoming.
 [<a href="https://arxiv.org/abs/1810.04778">arxiv</a>]

r-package/policytree/DESCRIPTION

@@ -9,9 +9,9 @@ Authors@R: c(
     person("Stefan", "Wager", role = "aut")
     )
 Description: Learn optimal policies via doubly robust empirical
- welfare maximization over trees. This package implements the multi-action doubly
- robust approach of Zhou, Athey and Wager (2018) <arXiv:1810.04778> in the case where
- we want to learn policies that belong to the class of depth k decision trees.
+ welfare maximization over trees. Given doubly robust reward estimates, this package
+ finds a rule-based treatment prescription policy, where the policy takes the form of
+ a shallow decision tree that is globally (or close to) optimal.
 Depends:
     R (>= 3.5.0)
 License: GPL-3

r-package/policytree/R/data.R

@@ -12,7 +12,7 @@
 #' @return A list with realized action \eqn{a_i}, region \eqn{r_i},
 #'  conditional mean \eqn{\mu}, outcome \eqn{Y} and covariates \eqn{X}
 #' @references Zhou, Zhengyuan, Susan Athey, and Stefan Wager. "Offline multi-action policy learning:
-#'  Generalization and optimization." arXiv preprint arXiv:1810.04778 (2018).
+#'  Generalization and optimization." Operations Research, forthcoming.
 #' @export
 gen_data_mapl <- function(n, p = 10, sigma2 = 4) {
   if (p < 7) {

r-package/policytree/R/hybrid_policy_tree.R

@@ -6,9 +6,11 @@
 #' Builds deeper trees by iteratively using exact tree search to look ahead l splits. For example,
 #' with `depth = 3` and `search.depth = 2`, the root split is determined by a depth 2 exact tree,
 #' and two new depth 2 trees are fit in the two immediate children using exact tree search,
-#' leading to a total depth of 3. Note that the resulting tree may be shallower than the
-#' specified `depth` depending on whether leaf nodes were pruned or not.
-#' This algorithm scales with some coefficient multiple of the runtime of a `search.depth` `policy_tree`.
+#' leading to a total depth of 3 (the resulting tree may be shallower than the
+#' specified `depth` depending on whether leaf nodes were pruned or not).
+#' This algorithm scales with some coefficient multiple of the runtime of a `search.depth` `policy_tree`,
+#' which means that for this approach to be feasible it needs an (n, p, d) configuration in which
+#' a `search.depth` `policy_tree` runs in reasonable time.
 #'
 #' The algorithm: desired depth is given by `depth`. Each node is split using exact tree search
 #' with depth  = `search.depth`. When we reach a node where the current level + `search.depth` is equal to `depth`,

r-package/policytree/R/policy_tree.R

@@ -37,7 +37,7 @@
 #'  "policytree: Policy learning via doubly robust empirical welfare maximization over trees."
 #'   Journal of Open Source Software 5, no. 50 (2020): 2232.
 #' @references Zhou, Zhengyuan, Susan Athey, and Stefan Wager. "Offline multi-action policy learning:
-#'  Generalization and optimization." arXiv preprint arXiv:1810.04778 (2018).
+#'  Generalization and optimization." Operations Research, forthcoming.
 #'
 #' @examples
 #' \donttest{
@@ -99,10 +99,10 @@ policy_tree <- function(X, Gamma, depth = 2, split.step = 1, min.node.size = 1,
     stop(paste("Currently the only supported data input types are:",
                "`matrix`, `data.frame`"))
   }
-  if (!is.numeric(as.matrix(X))) {
+  if (!is.numeric(as.matrix(X)) || any(dim(X) == 0)) {
     stop("The feature matrix X must be numeric")
   }
-  if (!is.numeric(as.matrix(Gamma))) {
+  if (!is.numeric(as.matrix(Gamma)) || any(dim(Gamma) == 0)) {
     stop("The reward matrix Gamma must be numeric")
   }
   if (anyNA(X)) {
@@ -135,6 +135,22 @@ policy_tree <- function(X, Gamma, depth = 2, split.step = 1, min.node.size = 1,
         "see the documentation for details.)"
       ), immediate. = TRUE)
     }
+    if (ncol(X) > 50) {
+      warning(paste0(
+        "The number of covariates exceeds 50. Consider reducing the dimensionality before ",
+        "running policy_tree, by for example using only the Xj's with the ",
+        "highest variable importance (`grf::variable_importance` - the runtime of exact tree ",
+        "search scales with ncol(X)^depth, see the documentation for details)."
+      ), immediate. = TRUE)
+    }
+    if (depth > 2 && nrow(X) > 5000) {
+      warning(paste0(
+        "A depth 3 or deeper policy_tree is only feasible for 'small' n and p. ",
+        "To fit deeper trees, consider using the hybrid greedy approach available in the function ",
+        "`hybrid_policy_tree`. Note that this still requires an (n, p) configuration ",
+        "which is feasible for a depth k=2 policy_tree, see the documentation for details."
+      ), immediate. = TRUE)
+    }
   }
 
   action.names <- colnames(Gamma)

r-package/policytree/R/policytree-package.R

@@ -1,5 +1,5 @@
 #' @description
-#' A package for learning optimal policies via doubly robust empirical welfare maximization over trees. Many practical policy applications require interpretable predictions. For example, a drug prescription guide that follows a simple 2-question Yes/No checklist can be encoded as a depth 2 decision tree (does the patient have a heart condition - etc.). This package implements the multi-action doubly robust approach of Zhou et al. (2018) in the case where we want to learn policies that belong to the class of depth k decision trees.
+#' A package for learning simple rule-based policies, where the rule takes the form of a shallow decision tree. Applications include settings which require interpretable predictions, such as for example a medical treatment prescription. This package uses doubly robust reward estimates from `grf` to find a shallow, but globally optimal decision tree.
 #'
 #' Some helpful links for getting started:
 #'
@@ -8,7 +8,7 @@
 #'
 #' @examples
 #' \donttest{
-#' # Multi-action policy learning.
+#' # Multi-action policy learning example.
 #' n <- 250
 #' p <- 10
 #' X <- matrix(rnorm(n * p), n, p)