scmBayesPost v0.4.7

scmBayesPost 0.4.7

Changes

• prepare_data_general(): each treated unit's pseudo-panel now contains the
  unit plus only the donors that carry positive weight in its column of W,
  instead of every control. Zero-weight rows contribute nothing to the
  weighted likelihood, so posterior draws are unchanged, while the number of
  rows falls sharply when donor screening leaves many zero weights.
• prepare_data_general() gains w_min (default 0). When w_min > 0, donors
  below that weight are dropped from each pseudo-panel and the remaining
  weights renormalised to the column's original mass. This is an
  approximation, intended for large donor pools (e.g. 4,000 stratified
  onboarders) whose pseudo-panels otherwise exceed memory and the 2^31-row
  limit. On simulated data w_min = 1e-3 shrank the panels about eightfold
  with unit effects correlated 0.9999 with the exact version.
• Fix: the sigma2 update in every sampler used the number of pseudo-panel
  rows as degrees of freedom, so zero-weight donor rows inflated the
  precision whenever donor screening applied. Degrees of freedom now count
  rows with positive weight. Results with no zero-weight donors are
  unaffected; results that had zero-weight donors were over-precise and
  change.
• testthat suite added (tests/testthat/test-prepare-data.R): verifies that
  the positive-weight pseudo-panels reproduce the all-controls draws, and
  that w_min drops donors, preserves weight mass and is inert at 0.

Compatibility

• Default behaviour (w_min = 0) reproduces previous results exactly for any
  weights matrix without zeros.
• budgetsc >= 0.4.7 passes post.w_min through to this version.

scmBayesPost v0.4.6

scmBayesPost v0.4.6 — Bug fix

What was broken

build_iv_formula() was storing treatment, instruments, and
controls as formula attributes using attr(), but as.formula()
silently strips custom attributes when constructing the formula object.
This meant .parse_iv_formula() never found the attributes, always
fell back to the factor() heuristic, and misclassified plain numeric
controls (e.g. income_level) as instruments — giving incorrect
results in instrument_validity_report() and
test_instrument_strength().

What changed

build_iv_formula() now uses structure() to attach attributes after
as.formula(), which preserves them correctly. Attributes can be
verified with:

f.X <- build_iv_formula(
  outcome     = "outcome_var",
  instruments = "peer_adoption_rate",
  controls    = c("income_level", "factor(event_time)")
)

attr(f.X, "instruments")   # "peer_adoption_rate"
attr(f.X, "controls")      # c("income_level", "factor(event_time)")
attr(f.X, "treatment")     # "budgetdummy"

No changes to function signatures or output structure.

Installation

remotes::install_github("andrebonfrer/scmBayesPost@v0.4.6")

scmBayesPost v0.4.5

scmBayesPost v0.4.5

What changed

build_iv_formula() is now a formally exported package function.
Previously it had to be sourced as a standalone script. It is now
available directly after library(scmBayesPost).

Formula attributes (treatment, instruments, controls) are stored
at creation time for reliable downstream parsing by
instrument_validity_report() and test_instrument_strength().

Installation

remotes::install_github("andrebonfrer/scmBayesPost@v0.4.5")

scmBayesPost v0.4.4

scmBayesPost v0.4.4

What changed

build_iv_formula() now stores instrument, control, and treatment
variable names as formula attributes. .parse_iv_formula() reads
these attributes to reliably classify instruments versus controls,
fixing a fragility where plain numeric controls were incorrectly
classified as instruments when passed alongside factor() terms.

No changes to function signatures or output.

Installation

remotes::install_github("andrebonfrer/scmBayesPost@v0.4.4")

scmBayesPost v0.4.3

scmBayesPost v0.4.3

What changed

instrument_validity_report() and test_instrument_strength() now
accept f.X directly from build_iv_formula(). Instruments and
treatment are parsed automatically from the second block of the formula.
fixest::feglm is selected automatically when factor() terms are
detected — no separate fixed_effects argument needed.

The fixed_effects argument introduced in v0.4.2 has been removed.

Installation

remotes::install_github("andrebonfrer/scmBayesPost@v0.4.3")

scmBayesPost v0.4.2

scmBayesPost v0.4.2

What changed

test_instrument_strength() now accepts a fixed_effects argument,
switching from stats::glm to fixest::feglm for large datasets
where standard probit fails to converge. Only time fixed effects
(e.g. "event_time") should be used — unit fixed effects cause
incidental parameters bias in nonlinear models and are already
absorbed by the SCM weights.

Installation

remotes::install_github("andrebonfrer/scmBayesPost@v0.4.2")

scmBayesPost v0.4.1

scmBayesPost v0.4.1 — Documentation fix

What changed

• Fixed broken plot_first_stage.Rd that caused three devtools::check()
  warnings (unknown macro \item, missing \description, undocumented
  title argument). Roxygen block rewritten with minimal safe structure.
• plot_first_stage() function and behaviour unchanged.

Installation

remotes::install_github("andrebonfrer/scmBayesPost@v0.4.1")

scmBayesPost v0.4.0

scmBayesPost v0.4.0 — Architectural overhaul

Why this exists

Versions 0.3.x introduced the Bayesian probit first stage but remained
slow because all Gibbs samplers operated on a stacked block-diagonal
design matrix X_block. On the motivating dataset with J0 = 1,879
treated units and N_obs = 292,199 observations this matrix had
157 million rows, making every matrix multiply inside the Gibbs
loop catastrophically expensive regardless of optimisation. Profiling
consistently showed %*% consuming >90% of iteration time.

What changed

X_block is never built. prepare_data_general() now returns the J0
unit-level matrices and vectors (X_list, y_list, w_list,
row_idx_list) directly. All four Gibbs samplers operate on these
lists, reducing every matrix operation inside the loop to unit-level
arithmetic. For K=1 (the standard single-treatment case) each unit's
beta update is a scalar division and a single normal draw — no matrix
operations at all.

Breaking change

gdata$X_block, gdata$Y_block, and gdata$W no longer exist.
Code accessing these directly must be updated to use gdata$X_list,
gdata$y_list, and gdata$w_list.

Expected performance

The dominant remaining cost is X_fs %*% delta — a 292k × p_fs
dense matrix-vector product for the first-stage probit, which is
unavoidable but benefits from multi-threaded BLAS. All other
operations are now unit-level and fast.

Installation

remotes::install_github("andrebonfrer/scmBayesPost@v0.4.0")

scmBayesPost v0.3.1

scmBayesPost v0.3.1 — Hotfix release

What was broken

Two regressions introduced in v0.3.0 affected large-dataset runs:

1. Pre-sampler hang: .precompute_XtWX_blocks() used Matrix::which()
   to identify non-zero rows per column block, scanning up to 292k rows
   for each of 1879 treated unit blocks before a single Gibbs iteration
   ran. On the motivating dataset this caused a 5-10 minute hang at
   startup and effectively made the sampler unusable.

2. AWS instance crash: the RhpcBLASctl threading block could crash
   R on non-standard BLAS configurations (observed on AWS r-instance
   types).

What changed

• .precompute_XtWX_blocks() now converts X_block to CSC format once
  and reads row indices directly from the column pointer arrays
  (X_csc@p, X_csc@i). Total work is O(nnz) rather than
  O(N_stacked * K * J0). Precompute completes in seconds.

• RhpcBLASctl block wrapped in tryCatch — BLAS configuration
  failures are caught silently and the sampler continues normally.

Installation

remotes::install_github("andrebonfrer/scmBayesPost@v0.3.1")

scmBayesPost v0.3.0

scmBayesPost v0.3.0 — Major performance release

The problem

On datasets with large numbers of treated units (J0 ~ 1879 in the
motivating application), the Gibbs samplers for the Bayesian probit
first stage (selection_probit_bayes) were taking approximately
21 seconds per iteration — around 6 hours for a 2,000-iteration
run. Profiling identified three dominant costs:

1. Cholesky factorisation of a [K*J0 x K*J0] matrix (e.g. 1879 x 1879) at every iteration, despite this matrix being block-diagonal
2. Sparse matrix-vector product X_block %*% beta_bd over a
   [N_stacked x K*J0] matrix at every iteration
3. Per-iteration data.table merge to align nu_hat with X_block
   row ordering

What changed

The samplers now exploit the block-diagonal structure of X_block
throughout. Key changes:

• The [K*J0 x K*J0] Cholesky is replaced by J0 independent
  [K x K] factorisations — for K = 1 (the common single-treatment
  case) this collapses to scalar arithmetic with no matrix operations
• X_block %*% beta_bd is replaced by a unit-by-unit loop over
  precomputed row ranges and cached column slices
• The data.table merge is replaced by a one-time integer index map
• Truncated normal draws use a fast inverse-CDF sampler with
  precomputed treated/untreated splits
• rho and sigma2 updates use direct vector operations rather than
  sparse matrix products

Expected speedup

The theoretical reduction in the Cholesky step alone is from
O((K*J0)^3) to O(J0 * K^3) — for K=1, J0=1879 this is a
factor of 1879^2 ≈ 3.5 million. In practice the overall per-iteration
speedup depends on which operations remain dominant after the fix
(primarily X_fs %*% delta, a 292k × 162 dense matrix-vector product).
Multi-threaded BLAS is recommended for further acceleration:

RhpcBLASctl::blas_set_num_threads(parallel::detectCores(logical = FALSE))

API unchanged

No changes to function signatures, arguments, or output. Existing code
runs without modification.

Installation

remotes::install_github("andrebonfrer/scmBayesPost@v0.3.0")