Skip to content

Fix stanr/stanli backend dropping group-level parameter levels - #2

Open
gravesti wants to merge 1 commit into
andrjohns:stanr-stanlifrom
gravesti:fix/stanr-ranef-truncation
Open

Fix stanr/stanli backend dropping group-level parameter levels#2
gravesti wants to merge 1 commit into
andrjohns:stanr-stanlifrom
gravesti:fix/stanr-ranef-truncation

Conversation

@gravesti

Copy link
Copy Markdown

Thanks for putting together this patch for brms. I came across this bug where the wrong variables were saved compared to cmdstan. LLM generated patch below.

read_stanr_fit_as_stanfit()/.stanfit_from_csfit() computed a correctly filtered 'variables'/'model_pars' list (excluding raw non-centered z_* parameters, keeping lprior/lp__), but never applied that filter to the actual 'samples' data frame before setting fnames_oi <- colnames(samples). This left fnames_oi out of sync with dims_oi/pars_oi, which downstream renaming logic assumes are in sync. Once a grouping factor had more than 2 flattened elements, this silently corrupted parameter names: raw z_1 columns were kept, lprior/lp__ were dropped, and one level of the derived r___ parameter went missing.

Fix: filter 'samples' down to columns matching model_pars/special_vars before computing fnames_oi, mirroring how read_csv_as_stanfit() already pre-filters columns via cmdstanr::read_cmdstan_csv(variables = ...).

Adds a regression test (tests/testthat/tests.stanr-backend.R) that fails on the prior code and passes after this fix.

Reprex:

## Minimal reproducible example for a data-loss bug in the 'stanr' backend
## (which also affects 'stanli', since stanli is embedded inside stanr).
##
## Bug: for a grouping factor with 3+ levels, brmsfit objects fitted via
## backend = "stanr" lose one level of the derived group-level effect
## (r_<group>__<term>[level,coef]) and the 'lprior'/'lp__' scalars, while
## retaining the raw, non-centered 'z_1' parameters that should have been
## dropped. A 'cmdstanr' (or 'rstan') fit of the exact same model is correct.
##
## Root cause (see brms/R/backends.R, read_stanr_fit_as_stanfit() /
## .stanfit_from_csfit()): the filtered 'variables' vector (which correctly
## excludes 'z_1' and keeps 'lprior'/'lp__') is used to build the metadata
## bookkeeping (model_pars/par_dims) but is never applied to filter the
## actual `samples` data frame, so `fnames_oi <- colnames(samples)` reflects
## the wrong (unfiltered) set of columns.

library(brms)

set.seed(6231)
n <- 60
dat <- data.frame(
  y = rnorm(n),
  x = rnorm(n),
  g = factor(sample(c("a", "b", "c"), n, replace = TRUE))
)

fit_common_args <- list(
  formula = y ~ x + (1 | g),
  data = dat,
  chains = 1,
  iter = 40,
  warmup = 20,
  seed = 8213,
  refresh = 0
)

fit_stanr <- do.call(brm, c(fit_common_args, list(backend = "stanr")))
fit_cmdstanr <- do.call(brm, c(fit_common_args, list(backend = "cmdstanr")))

vars_stanr <- variables(fit_stanr)
vars_cmdstanr <- variables(fit_cmdstanr)

cat("stanr variables:\n")
print(vars_stanr)
cat("\ncmdstanr variables:\n")
print(vars_cmdstanr)

r_g_stanr <- grep("^r_g", vars_stanr, value = TRUE)
r_g_cmdstanr <- grep("^r_g", vars_cmdstanr, value = TRUE)
z_leftover_stanr <- grep("^z_1", vars_stanr, value = TRUE)

cat("\nNumber of r_g_Intercept levels -- stanr:", length(r_g_stanr),
    " cmdstanr:", length(r_g_cmdstanr), " (expected: 3 for both)\n")
cat("Leftover raw z_1 parameters in stanr fit (expected: none):",
    length(z_leftover_stanr), "\n")

stopifnot(
  "stanr fit is missing group-level effect levels" =
    length(r_g_stanr) == length(r_g_cmdstanr),
  "stanr fit retains raw z_1 parameters that should have been excluded" =
    length(z_leftover_stanr) == 0
)

read_stanr_fit_as_stanfit()/.stanfit_from_csfit() computed a correctly
filtered 'variables'/'model_pars' list (excluding raw non-centered
z_* parameters, keeping lprior/lp__), but never applied that filter to
the actual 'samples' data frame before setting fnames_oi <- colnames(samples).
This left fnames_oi out of sync with dims_oi/pars_oi, which downstream
renaming logic assumes are in sync. Once a grouping factor had more
than 2 flattened elements, this silently corrupted parameter names:
raw z_1 columns were kept, lprior/lp__ were dropped, and one level of
the derived r_<group>__<term> parameter went missing -- all without
raising an error, since column counts still matched.

Fix: filter 'samples' down to columns matching model_pars/special_vars
before computing fnames_oi, mirroring how read_csv_as_stanfit() already
pre-filters columns via cmdstanr::read_cmdstan_csv(variables = ...).

Adds a regression test (tests/testthat/tests.stanr-backend.R) that
fails on the prior code and passes after this fix.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant