loo_model_weights() rejects PSIS-LOO objects with equal observations but different posterior sample sizes

[FBartos]

Both objects contain pointwise LOO results for the same 32 observations in the same order. The only difference is the number of posterior draws: 1,000 versus 500 after merging chains.

loo_compare() accepts these objects, but loo_model_weights() rejects them because validate_psis_loo_list() requires both dimensions of the psis_loo objects to match.

My understanding is that stacking and pseudo-BMA operate on the pointwise elpd_loo vectors, so the posterior sample sizes need not be identical. Different sample sizes affect Monte Carlo precision but should not make the objects incompatible.

example

library(loo)

log_lik_full <- example_loglik_array()

# Keep the same chains and 32 observations, but use half the iterations.
log_lik_short <- log_lik_full[seq_len(dim(log_lik_full)[1] / 2),,, drop = FALSE]

loo_full <- loo(
  log_lik_full,
  r_eff = relative_eff(exp(log_lik_full))
)

loo_short <- loo(
  log_lik_short,
  r_eff = relative_eff(exp(log_lik_short))
)

dim(loo_full)
#> [1] 1000   32

dim(loo_short)
#> [1] 500  32

# Comparison works despite the different posterior sample sizes.
loo_compare(loo_full, loo_short)

# Model weights do not.
loo_model_weights(list(
  full  = loo_full,
  short = loo_short
))

sessionInfo()

output

> library(loo)
This is loo version 2.10.1
- Online documentation and vignettes at mc-stan.org/loo
- As of v2.0.0 loo defaults to 1 core but we recommend using as many as possible. Use the 'cores' argument or set options(mc.cores = NUM_CORES) for an entire session. 
- Windows 10 users: loo may be very slow if 'mc.cores' is set in your .Rprofile file (see https://github.com/stan-dev/loo/issues/94).
Warning message:
package ‘loo’ was built under R version 4.6.1 
> 
> log_lik_full <- example_loglik_array()
> 
> # Keep the same chains and 32 observations, but use half the iterations.
> log_lik_short <- log_lik_full[seq_len(dim(log_lik_full)[1] / 2),,, drop = FALSE]
> 
> loo_full <- loo(
+   log_lik_full,
+   r_eff = relative_eff(exp(log_lik_full))
+ )
> 
> loo_short <- loo(
+   log_lik_short,
+   r_eff = relative_eff(exp(log_lik_short))
+ )
> 
> dim(loo_full)
[1] 1000   32
> #> [1] 1000   32
> 
> dim(loo_short)
[1] 500  32
> #> [1] 500  32
> 
> # Comparison works despite the different posterior sample sizes.
> loo_compare(loo_full, loo_short)
  model elpd_diff se_diff p_worse diag_diff diag_elpd
 model2       0.0     0.0      NA                    
 model1       0.0     0.1    0.70   N < 100          

Diagnostic flags present.
See ?`loo-glossary` (sections `diag_diff` and `diag_elpd`)
or https://mc-stan.org/loo/reference/loo-glossary.html.
> 
> # Model weights do not.
> loo_model_weights(list(
+   full  = loo_full,
+   short = loo_short
+ ))
Error: Each object in the list must have the same dimensions.

> sessionInfo()
R version 4.6.0 (2026-04-24 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows 11 x64 (build 26200)

Matrix products: default
  LAPACK version 3.12.1

locale:
[1] LC_COLLATE=English_United States.utf8  LC_CTYPE=English_United States.utf8   
[3] LC_MONETARY=English_United States.utf8 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.utf8    

time zone: Europe/Amsterdam
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] loo_2.10.1

loaded via a namespace (and not attached):
 [1] vctrs_0.7.3          cli_3.6.6            knitr_1.51           rlang_1.2.0         
 [5] xfun_0.57            otel_0.2.0           generics_0.1.4       tensorA_0.36.2.1    
 [9] glue_1.8.1           backports_1.5.1      htmltools_0.5.9      distributional_0.7.0
[13] rmarkdown_2.31       evaluate_1.0.5       tibble_3.3.1         abind_1.4-8         
[17] fastmap_1.2.0        yaml_2.3.12          lifecycle_1.0.5      compiler_4.6.0      
[21] posterior_1.7.0      pkgconfig_2.0.3      rstudioapi_0.18.0    digest_0.6.39       
[25] pillar_1.11.1        parallel_4.6.0       magrittr_2.0.5       checkmate_2.3.4     
[29] tools_4.6.0          matrixStats_1.5.0