index.rst

Python implementation of elastic-net regularized generalized linear models

Pyglmnet is a Python library implementing generalized linear models (GLMs) with advanced regularization options. It provides a wide range of noise models (with paired canonical link functions) including gaussian, binomial, multinomial, poisson, and softplus. It supports a wide range of regularizers: ridge, lasso, elastic net, group lasso, and Tikhonov regularization.

[Repository] [Documentation]

A brief introduction to GLMs

Linear models are estimated as

y = β₀ + Xβ + ϵ

The parameters β₀, β are estimated using ordinary least squares, under the implicit assumption the y is normally distributed.

Generalized linear models allow us to generalize this approach to point-wise nonlinearities q(.) and a family of exponential distributions for y.

y = q(β₀ + Xβ) + ϵ

Regularized GLMs are estimated by minimizing a loss function specified by the penalized negative log-likelihood. The elastic net penalty interpolates between L2 and L1 norm. Thus, we solve the following optimization problem:

$$\min_{\beta_0, \beta} \frac{1}{N} \sum_{i = 1}^N \mathcal{L} (y_i, \beta_0 + \beta^T x_i) + \lambda [ \frac{1}{2}(1 - \alpha) \mathcal{P}_2 + \alpha \mathcal{P}_1 ]$$

where 𝒫₂ and 𝒫₁ are the generalized L2 (Tikhonov) and generalized L1 (Group Lasso) penalties, given by:

$$\begin{aligned} \mathcal{P}_2 & = & \|\Gamma \beta \|_2^2 \\\ \mathcal{P}_1 & = & \sum_g \|\beta_{j,g}\|_2 \end{aligned}$$

where Γ is the Tikhonov matrix: a square factorization of the inverse covariance matrix and β_j, g is the j th coefficient of group g.

Contents

install start tutorial cheatsheet auto_examples/index api developers requests resources whats_new

Questions / Errors / Bugs

If you have a question about the code or find errors or bugs, please report it here. For more specific question, feel free to email us directly.