CALM

CALM — Combine Augmented Linear Models

Overview

CALM() (R: calm()) combines parameters of all possible linear regressions of the response variable on subsets of predictor variables. Models are weighted by an information criterion (IC), producing averaged coefficients with a natural shrinkage effect — variables absent from a given model receive a coefficient of zero in that model, pulling the combined estimate toward zero.

How It Works

1. Generate all subsets — In bruteforce mode, fit all 2^k models (where k is the number of predictors), including the intercept-only model.
2. Compute IC weights — For each model, compute the selected IC (AIC, AICc, BIC, or BICc). Convert to weights: w_i = exp(-0.5 * (IC_i - IC_min)) / sum(exp(-0.5 * (IC_j - IC_min))).
3. Average parameters — Combine coefficients as the weighted average across all models: beta_combined = sum(w_i * beta_i).
4. Shrinkage — Models that don't include a variable have coefficient 0 for that variable, so the combined coefficient is shrunk toward zero proportional to the variable's "importance" (sum of IC weights for models containing it).

Important: Both R and Python take a data frame / dict as input (response in the first column), not a list of pre-fitted models.

Parameters

Parameter	R	Python	Type	Default	Description
data	data	data	data.frame / dict or DataFrame	—	Response in first column, predictors in the rest
ic	ic	ic	character / str	"AICc"	IC to use: "AIC", "AICc", "BIC", "BICc"
bruteforce	bruteforce	bruteforce	logical / bool	FALSE / False	If TRUE/True, fit all 2^k models
silent	silent	silent	logical / bool	TRUE	If FALSE/False, print progress
formula	formula	—	formula / —	NULL	Restrict combination to variables in formula
subset	subset	—	vector / —	NULL	Subset of observations to use
distribution	distribution	distribution	character / str	"dnorm"	Distribution for ALM
parallel	parallel	—	logical / —	FALSE	Use parallel computation
...	...	**kwargs	—	—	Additional arguments passed to alm() / ALM()

Note: Both R and Python default to bruteforce=FALSE/False.

Return Value

Field	R	Python	Type	Description
coefficients	$coefficients	.coefficients	numeric / ndarray	Combined (weighted average) coefficients
vcov	$vcov	.vcov()	matrix / ndarray	Combined variance-covariance matrix
fitted.values/fitted	$fitted.values / $fitted	.fitted	numeric / ndarray	Fitted values on original scale
residuals	$residuals	.residuals	numeric / ndarray	Distribution-specific residuals
mu	$mu	—	numeric / —	Location parameter values
scale	$scale	.scale	numeric / float	Scale parameter
distribution	$distribution	.distribution / .distribution_	character / str	Distribution used
logLik	$logLik	.log_lik / .loglik	numeric / float	Combined (weighted) log-likelihood
IC	$IC	.aic / .aicc / .bic / .bicc	numeric / float	Information criteria
ICType	$ICType	.IC_type	character / str	Type of IC used
df.residual	$df.residual	.df_residual / .df_residual_	numeric / float	Residual degrees of freedom
df	$df	.nparam	numeric / float	Model degrees of freedom
importance	$importance	.importance	numeric / ndarray	Importance of each parameter (sum of IC weights)
combination	$combination	.combination	matrix / ndarray	Matrix: rows = models, columns = variables + IC weights + IC values
weights	—	.weights	— / ndarray	IC weights for each model
timeElapsed	$timeElapsed	.time_elapsed	numeric / float	Computation time (seconds)
intercept	—	.intercept_	— / float	Intercept value
sigma	—	.sigma	— / float	Residual standard error
nobs	—	.nobs	— / int	Number of observations
nparam	—	.nparam	— / float	Effective number of parameters (sum of importances + 1)
loss_	—	.loss_	— / str	Always "likelihood"
loss_value	—	.loss_value	— / float	Negative log-likelihood
actuals	—	.actuals	— / ndarray	Original response values
formula	—	.formula	— / str	Formula string
n_param	—	.n_param	— / dict	{"number": nparam, "df": df_residual}

Methods

Method	Description
summary(level=0.95)	Returns CALMSummary with coefficients, std errors, importance, confidence intervals
predict(X, interval, level, side)	Returns PredictionResult (see ALM)
confint(parm, level)	Confidence intervals for parameters
vcov()	Variance-covariance matrix
score(X, y, metric)	Score: "likelihood", "MSE", "MAE", "R2"

Examples

Basic Usage

# R
set.seed(42)
x1 <- rnorm(100)
x2 <- rnorm(100)
x3 <- rnorm(100)
y <- 1 + 2*x1 + 0.5*x2 + rnorm(100)
data <- data.frame(y=y, x1=x1, x2=x2, x3=x3)

result <- calm(data, ic="AICc", bruteforce=TRUE)
print(result)
summary(result)

# Python
import numpy as np
from greybox import CALM

np.random.seed(42)
x1 = np.random.normal(0, 1, 100)
x2 = np.random.normal(0, 1, 100)
x3 = np.random.normal(0, 1, 100)
y = 1 + 2*x1 + 0.5*x2 + np.random.normal(0, 1, 100)

data = {"y": y, "x1": x1, "x2": x2, "x3": x3}
result = CALM(data, ic="AICc", bruteforce=False)
print(result)

Accessing Importance and Weights

# R
result$importance
# (Intercept)          x1          x2          x3
#    1.000000    0.999800    0.723400    0.098200

result$combination

# Python
print(result.importance)
# [1.0000  0.9998  0.7234  0.0982]

print(result.weights)          # IC weights for each model
print(result.combination)      # Full combination matrix
print(result.coefficient_names)

Making Predictions

# R
new_data <- data.frame(x1=c(0, 1, 2), x2=c(0, 0, 0), x3=c(0, 0, 0))
pred <- predict(result, newdata=new_data, interval="prediction", level=0.95)
pred$mean
pred$lower
pred$upper

# Python
from greybox import formula

new_data = {"y": np.zeros(3), "x1": np.array([0, 1, 2]),
            "x2": np.zeros(3), "x3": np.zeros(3)}
_, X_new = formula("y ~ x1 + x2 + x3", new_data)
pred = result.predict(X_new, interval="prediction", level=0.95)
print(pred.to_dataframe())

Summary Output

# Python
summary = result.summary()
print(summary)
# The AICc combined model
# Response variable: y
# Distribution used in the estimation: Normal
# Coefficients:
#                 Estimate   Std. Error   Importance  Lower 2.5%  Upper 97.5%
#  (Intercept)      1.0234       0.1023        1.000      0.8211       1.2257 *
#           x1      1.9876       0.1056        1.000      1.7787       2.1965 *
#           x2      0.4123       0.1234        0.723      0.1684       0.6562 *
#           x3      0.0098       0.0987        0.098     -0.1843       0.2039
#
# Error standard deviation: 1.0234
# ...

References

• Burnham, K.P. and Anderson, D.R. (2002). Model Selection and Multimodel Inference. Springer.
• Svetunkov, I. (2023). Statistics for Business Analytics. https://openforecast.org/sba/