Skip to content

stepwise

Jump to bottom
Ivan Svetunkov edited this page Jun 24, 2026 · 4 revisions

stepwise — Forward Stepwise Variable Selection

Function signatures

R

stepwise(data, ic=c("AICc","AIC","BIC","BICc"), silent=TRUE, df=NULL,
         formula=NULL, subset=NULL,
         method=c("pearson","kendall","spearman"),
         distribution=c(                   "dnorm","dlaplace","ds","dgnorm","dlogis","dt","dalaplace",
                   "dlnorm","dllaplace","dls","dlgnorm","dbcnorm",
                   "dinvgauss","dgamma","dexp","dfnorm","drectnorm",
                   "dpois","dnbinom","dbinom","dgeom","dbeta","dlogitnorm",
                   "plogis","pnorm"),
         occurrence=c("none","plogis","pnorm"), ...)

Python

def stepwise(
    data: dict | DataFrame,
    ic: Literal["AICc", "AIC", "BIC", "BICc"] = "AICc",
    silent: bool = True,
    df: int | None = None,
    formula: str | None = None,
    subset: Any | None = None,
    method: Literal["pearson", "kendall", "spearman"] = "pearson",
    distribution: str = "dnorm",
    occurrence: Literal["none", "plogis", "pnorm"] = "none",
    **kwargs,
) -> ALM: ...

Overview

The stepwise() function performs forward stepwise selection of regressors using partial correlations and an information criterion (IC). It selects variables that produce the linear regression with the lowest IC value. This is a simpler and faster alternative to step() from R's stats package.

Algorithm

  1. Fit intercept-only model — Compute the IC for the model with only an intercept.
  2. Compute correlations — Calculate the correlation (Pearson, Kendall, or Spearman) between the current residuals and all remaining candidate variables.
  3. Add the most correlated variable — Select the variable with the highest absolute correlation.
  4. Refit the model — Fit ALM with the intercept and all selected variables.
  5. Check IC improvement — If the IC improved (decreased), go to step 2. Otherwise, stop and return the previous model.

The algorithm is the trace forward — variables are never removed once added.

Parameters

Parameter R Python Type Default Description
data data data data.frame / dict or DataFrame — Response in first column, predictors in the rest
ic ic ic character / str "AICc" IC to use: "AIC", "AICc", "BIC", "BICc"
silent silent silent logical / bool TRUE / True If FALSE/False, prints progress
df df df integer / int NULL / None Extra degrees of freedom
formula formula formula formula / str NULL / None Restrict selection to variables in formula
subset subset subset vector / array-like NULL / None Subset of observations to use
method method method character / str "pearson" Correlation method: "pearson", "kendall", "spearman"
distribution distribution distribution character / str "dnorm" Distribution for ALM
occurrence occurrence occurrence character / str "none" Occurrence model: "none", "plogis", "pnorm"
... ... **kwargs — — Additional arguments passed to alm() / ALM()

Return Value

Both R and Python return the final fitted ALM model. In Python, two additional attributes are attached:

Attribute Type Description
ic_values dict[str, float] IC value at each step, keyed by variable name (insertion order). The first key is always "Intercept".
time_elapsed float Seconds taken for the entire selection process

All standard ALM attributes and methods are available on the returned model (see ALM).

Examples

Basic Usage

# R
model <- stepwise(mtcars, subset=c(1:30))
summary(model)
# Python
import numpy as np
from greybox import stepwise, mtcars

model = stepwise(mtcars.head(30))
print(model.summary())

Forecasting

# R
predict(model, tail(mtcars,2), interval="prediction")
# Python
_, X = formula(model.formula, mtcars.tail(2))
model.predict(X, interval="prediction")

Different Information Criteria

# R — use BIC (tends to select smaller models)
model_bic <- stepwise(mtcars, ic="BIC")
# Python
model_bic = stepwise(mtcars, ic="BIC")

Non-silent Mode (progress output)

# R — see selection progress
model <- stepwise(mtcars, silent=FALSE)
# Output:
# Formula: `mpg`~ 1, IC: 209.1693
# 
# Step 1. Formula: `mpg`~ 1+`wt`, IC: 166.8866
# Correlations: 
#    cyl   disp     hp   drat     wt   qsec     vs     am   gear   carb 
# -0.852 -0.848 -0.776  0.681 -0.868  0.419  0.664  0.600  0.480 -0.551 
# ...
# Python — see selection progress
model = stepwise(mtcars, silent=False)
# Output:
# Formula: mpg ~ 1, IC: 209.1693
# 
# Step 1. Formula: mpg ~ 1 + wt, IC: 166.8866
# Correlations: 
#   cyl: 0.852
#   disp: 0.848
#   hp: 0.776
#   drat: 0.681
#   wt: 0.868
#   qsec: 0.419
#   vs: 0.664
#   am: 0.600
#   gear: 0.480
#   carb: 0.551
# ...

Accessing IC Values

# R — inspect the selection path
model <- stepwise(mtcars)
print(model$IC)
# Intercept        wt      qsec        am      carb 
#  209.1693  166.8866  158.2020  156.4271  157.9231
# Note: carb was evaluated but didn't improve IC, so the final model drops it
# Python — inspect the selection path
model = stepwise(data)
print(model.ic_values)
# {'Intercept': np.float64(209.16930921944117), ...

print(f"Selection took {model.time_elapsed:.2f} seconds")

With Non-Normal Distribution

# R — Poisson regression with stepwise selection
count_data <- data.frame(y=rpois(100, 5), x1=rnorm(100), x2=rnorm(100))
model <- stepwise(count_data, distribution="dpois")
# Python
count_data = {"y": np.random.poisson(5, 100),
              "x1": np.random.normal(0, 1, 100),
              "x2": np.random.normal(0, 1, 100)}
model = stepwise(count_data, distribution="dpois")

References

Clone this wiki locally