Skip to content

dataprep() applies predictors.op to the treated unit but not the controls - #2

Open
soodoku wants to merge 1 commit into
j-hai:mainfrom
finite-sample:apply-predictors-op-to-controls
Open

dataprep() applies predictors.op to the treated unit but not the controls#2
soodoku wants to merge 1 commit into
j-hai:mainfrom
finite-sample:apply-predictors-op-to-controls

Conversation

@soodoku

@soodoku soodoku commented Aug 2, 2026

Copy link
Copy Markdown

dataprep() aggregates predictors over time.predictors.prior separately for the treated unit and the controls. The treated unit uses the operator the caller supplied:

# R/dataprep.R:216
X1 <- apply(X1, 2, paste(predictors.op), na.rm = TRUE)

The controls use a hard-coded mean:

# R/dataprep.R:257
X0 <- sapply(X0, apply, 2, mean, na.rm = TRUE, simplify = TRUE)

So for any predictors.op other than "mean", X1 and X0 are built under different aggregation rules and the predictor-balance matrix that synth() optimises over is internally inconsistent.

Reproduction

Bundled basque data, predictors = "invest", predictors.op = "median", time.predictors.prior = 1964:1969:

library(Synth)
data(basque)
controls <- c(2:16, 18); prior <- 1964:1969

dp <- dataprep(foo = basque, predictors = "invest", predictors.op = "median",
               dependent = "gdpcap", unit.variable = "regionno",
               time.variable = "year", treatment.identifier = 17,
               controls.identifier = controls,
               time.predictors.prior = prior,
               time.optimize.ssr = 1960:1969, time.plot = 1955:1997)

hand <- function(u, f) f(basque$invest[basque$regionno == u & basque$year %in% prior],
                         na.rm = TRUE)

On main (1.2-0):

TREATED unit (regionno 17)
  dataprep X1        : 24.41825
  hand median        : 24.41825   <- operator honoured
  hand mean          : 24.64738

CONTROL units
  max |X0 - hand median| : 2.061043
  max |X0 - hand mean|   : 0.000000   <- controls received the mean

Asking for the median moves only the treated unit's row; every control row is byte-identical to what the "mean" call produces.

Change

One line, mirroring what line 216 already does:

     X0 <- split(X0, X0[,dim(X0)[2]])
-    X0 <- sapply(X0, apply, 2, mean, na.rm = TRUE, simplify = TRUE)
+    X0 <- sapply(X0, apply, 2, paste(predictors.op), na.rm = TRUE, simplify = TRUE)
     X0 <- as.matrix(X0[-dim(X0)[1],])

With it, max |X0 - hand median| becomes 0.000000.

Two things suggested this is an oversight rather than a deliberate asymmetry, though you would know better than I do:

  1. The message printed a few lines earlier in the control branch refers to the argument by name — "We ignore (na.rm = TRUE) all missing values for predictors.op" (R/dataprep.R:252) — in the same block that hard-codes mean.
  2. special.predictors operators are applied to both the treated unit and the controls (R/spec.pred.func.R), so the package currently treats the two kinds of predictor differently.

?dataprep describes predictors.op as the method used to aggregate the predictor variables, with no treated/control distinction.

Verification

  • Added tests/testthat/test-dataprep-predictors-op.R: FAIL 1 / PASS 1 without this change, PASS 2 with it.
  • Full suite via cd tests && NOT_CRAN=true Rscript testthat.R: FAIL 0 | SKIP 3 | PASS 187 both before and after.
  • R/dataprep.R is byte-identical between CRAN 1.1-10 and main at 1.2-0, so this affects the released version too.

What I have not established

Whether the asymmetry is intended — it is possible you want control units summarised by their mean regardless of predictors.op. If so the right change is a documentation note rather than this patch, and I am happy to be told that.

dataprep() aggregated the treated unit with the caller's predictors.op
(R/dataprep.R:216) but the controls with a hard-coded mean (R/dataprep.R:257),
so any operator other than "mean" produced a predictor-balance matrix whose
treated and control rows were computed on different rules.

With the bundled basque data, predictors.op = "median" and
time.predictors.prior = 1964:1969, the treated unit's X1 matches the hand
median exactly while every control's X0 matches the hand mean exactly, off by
up to 2.06 from the median that was requested.

Adds tests/testthat/test-dataprep-predictors-op.R, which fails without this
change and passes with it. Full suite unchanged at FAIL 0 | SKIP 3 | PASS 187.
soodoku added a commit to finite-sample/synth-predictors-op-bug that referenced this pull request Aug 2, 2026
An earlier version of this README stated Synth has no public issue tracker.
That was wrong -- j-hai/Synth exists and is active. I had trusted the URL field
in the DESCRIPTION, which lists only the author's homepage, rather than checking
the maintainer's GitHub account directly.

The fix is now submitted upstream as j-hai/Synth#2.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant