Skip to content

stepwise

Jump to bottom
Ivan Svetunkov edited this page Feb 19, 2026 · 4 revisions

stepwise — Forward Stepwise Variable Selection

Overview

The stepwise() function performs forward stepwise selection of regressors using partial correlations and an information criterion (IC). It selects variables that produce the linear regression with the lowest IC value. This is a simpler and faster alternative to step() from R's stats package.

Algorithm

  1. Fit intercept-only model — Compute the IC for the model with only an intercept.
  2. Compute correlations — Calculate the correlation (Pearson, Kendall, or Spearman) between the current residuals and all remaining candidate variables.
  3. Add the most correlated variable — Select the variable with the highest absolute correlation.
  4. Refit the model — Fit ALM with the intercept and all selected variables.
  5. Check IC improvement — If the IC improved (decreased), go to step 2. Otherwise, stop and return the previous model.

The algorithm is forward-only — variables are never removed once added.

Parameters

R: stepwise(data, ...)

Parameter Type Default Description
data data.frame — Response in first column, predictors in the rest
ic character "AICc" IC to use: "AIC", "AICc", "BIC", "BICc"
silent logical TRUE If FALSE, prints progress at each step
df integer NULL Extra degrees of freedom (e.g. when used on residuals)
formula formula NULL Restrict selection to variables in the formula
subset vector NULL Subset of observations to use
method character "pearson" Correlation method: "pearson", "kendall", "spearman"
distribution character "dnorm" Distribution for ALM (see ALM)
occurrence character "none" Occurrence model: "none", "plogis", "pnorm"
... Additional arguments passed to alm()

Python: stepwise(data, ...)

Parameter Type Default Description
data dict or DataFrame — Response in first column/key, predictors in the rest
ic str "AICc" IC to use: "AIC", "AICc", "BIC", "BICc"
silent bool True If False, prints progress at each step
df int None Extra degrees of freedom
formula str None Restrict selection to variables in the formula
subset array-like None Subset of observations to use
method str "pearson" Correlation method: "pearson", "kendall", "spearman"
distribution str "dnorm" Distribution for ALM
occurrence str "none" Occurrence model
**kwargs Additional arguments passed to ALM()

Return Value

Both R and Python return the final fitted ALM model. In Python, two additional attributes are attached:

Attribute Type Description
ic_values dict[str, float] IC value at each step, keyed by variable name (insertion order). The first key is always "Intercept".
time_elapsed float Seconds taken for the entire selection process

All standard ALM attributes and methods are available on the returned model (see ALM).

Examples

Basic Usage

# R
set.seed(42)
x1 <- rnorm(100)
x2 <- rnorm(100)
x3 <- rnorm(100)
y <- 1 + 2*x1 + 0.5*x2 + rnorm(100)
data <- data.frame(y=y, x1=x1, x2=x2, x3=x3)

model <- stepwise(data)
summary(model)
# Python
import numpy as np
from greybox import stepwise

np.random.seed(42)
x1 = np.random.normal(0, 1, 100)
x2 = np.random.normal(0, 1, 100)
x3 = np.random.normal(0, 1, 100)
y = 1 + 2*x1 + 0.5*x2 + np.random.normal(0, 1, 100)

data = {"y": y, "x1": x1, "x2": x2, "x3": x3}
model = stepwise(data)
print(model.summary())

Different Information Criteria

# R — use BIC (tends to select smaller models)
model_bic <- stepwise(data, ic="BIC")
# Python
model_bic = stepwise(data, ic="BIC")

Non-silent Mode (progress output)

# Python — see selection progress
model = stepwise(data, silent=False)
# Output:
# Formula: y ~ 1, IC: 298.4321
# Step 1. Formula: y ~ 1 + x1, IC: 210.1234
# Correlations:
#   x1: 0.876
#   x2: 0.234
#   x3: 0.012
#
# Step 2. Formula: y ~ 1 + x1 + x2, IC: 205.8765
# ...

Accessing IC Values

# Python — inspect the selection path
model = stepwise(data)
print(model.ic_values)
# {'Intercept': 298.43, 'x1': 210.12, 'x2': 205.88, 'x3': 206.01}
# Note: x3 was evaluated but didn't improve IC, so the final model uses x1 + x2

print(f"Selection took {model.time_elapsed:.2f} seconds")

With Non-Normal Distribution

# R — Poisson regression with stepwise selection
count_data <- data.frame(y=rpois(100, 5), x1=rnorm(100), x2=rnorm(100))
model <- stepwise(count_data, distribution="dpois")
# Python
count_data = {"y": np.random.poisson(5, 100),
              "x1": np.random.normal(0, 1, 100),
              "x2": np.random.normal(0, 1, 100)}
model = stepwise(count_data, distribution="dpois")

References

Clone this wiki locally