Skip to content

stepwise

Jump to bottom
Ivan Svetunkov edited this page Feb 19, 2026 · 4 revisions

stepwise — Forward Stepwise Variable Selection

Overview

The stepwise() function performs forward stepwise selection of regressors using partial correlations and an information criterion (IC). It selects variables that produce the linear regression with the lowest IC value. This is a simpler and faster alternative to step() from R's stats package.

Algorithm

  1. Fit intercept-only model — Compute the IC for the model with only an intercept.
  2. Compute correlations — Calculate the correlation (Pearson, Kendall, or Spearman) between the current residuals and all remaining candidate variables.
  3. Add the most correlated variable — Select the variable with the highest absolute correlation.
  4. Refit the model — Fit ALM with the intercept and all selected variables.
  5. Check IC improvement — If the IC improved (decreased), go to step 2. Otherwise, stop and return the previous model.

The algorithm is forward-only — variables are never removed once added.

Parameters

Parameter R Python Type Default Description
data data data data.frame / dict or DataFrame — Response in first column, predictors in the rest
ic ic ic character / str "AICc" IC to use: "AIC", "AICc", "BIC", "BICc"
silent silent silent logical / bool TRUE / True If FALSE/False, prints progress
df df df integer / int NULL / None Extra degrees of freedom
formula formula formula formula / str NULL / None Restrict selection to variables in formula
subset subset subset vector / array-like NULL / None Subset of observations to use
method method method character / str "pearson" Correlation method: "pearson", "kendall", "spearman"
distribution distribution distribution character / str "dnorm" Distribution for ALM
occurrence occurrence occurrence character / str "none" Occurrence model: "none", "plogis", "pnorm"
... ... **kwargs — — Additional arguments passed to alm() / ALM()

Return Value

Both R and Python return the final fitted ALM model. In Python, two additional attributes are attached:

Attribute Type Description
ic_values dict[str, float] IC value at each step, keyed by variable name (insertion order). The first key is always "Intercept".
time_elapsed float Seconds taken for the entire selection process

All standard ALM attributes and methods are available on the returned model (see ALM).

Examples

Basic Usage

# R
set.seed(42)
x1 <- rnorm(100)
x2 <- rnorm(100)
x3 <- rnorm(100)
y <- 1 + 2*x1 + 0.5*x2 + rnorm(100)
data <- data.frame(y=y, x1=x1, x2=x2, x3=x3)

model <- stepwise(data)
summary(model)
# Python
import numpy as np
from greybox import stepwise

np.random.seed(42)
x1 = np.random.normal(0, 1, 100)
x2 = np.random.normal(0, 1, 100)
x3 = np.random.normal(0, 1, 100)
y = 1 + 2*x1 + 0.5*x2 + np.random.normal(0, 1, 100)

data = {"y": y, "x1": x1, "x2": x2, "x3": x3}
model = stepwise(data)
print(model.summary())

Different Information Criteria

# R — use BIC (tends to select smaller models)
model_bic <- stepwise(data, ic="BIC")
# Python
model_bic = stepwise(data, ic="BIC")

Non-silent Mode (progress output)

# R — see selection progress
model <- stepwise(data, silent=FALSE)
# Output:
# Formula: y~1, IC: 298.4321
# Step 1. Formula: y~1+x1, IC: 210.1234
# Correlations:
#   x1: 0.876
#   x2: 0.234
#   x3: 0.012
#
# Step 2. Formula: y~1+x1+x2, IC: 205.8765
# ...
# Python — see selection progress
model = stepwise(data, silent=False)
# Output:
# Formula: y ~ 1, IC: 298.4321
# Step 1. Formula: y ~ 1 + x1, IC: 210.1234
# Correlations:
#   x1: 0.876
#   x2: 0.234
#   x3: 0.012
#
# Step 2. Formula: y ~ 1 + x1 + x2, IC: 205.8765
# ...

Accessing IC Values

# R — inspect the selection path
model <- stepwise(data)
print(model$IC)
# [1] 298.43 210.12 205.88 206.01
# Note: x3 was evaluated but didn't improve IC, so the final model uses x1 + x2
# Python — inspect the selection path
model = stepwise(data)
print(model.ic_values)
# {'Intercept': 298.43, 'x1': 210.12, 'x2': 205.88, 'x3': 206.01}
# Note: x3 was evaluated but didn't improve IC, so the final model uses x1 + x2

print(f"Selection took {model.time_elapsed:.2f} seconds")

With Non-Normal Distribution

# R — Poisson regression with stepwise selection
count_data <- data.frame(y=rpois(100, 5), x1=rnorm(100), x2=rnorm(100))
model <- stepwise(count_data, distribution="dpois")
# Python
count_data = {"y": np.random.poisson(5, 100),
              "x1": np.random.normal(0, 1, 100),
              "x2": np.random.normal(0, 1, 100)}
model = stepwise(count_data, distribution="dpois")

References

Clone this wiki locally