Skip to content

Loss Functions

Jump to bottom
Ivan Svetunkov edited this page May 21, 2026 · 7 revisions

Loss Functions

This page documents the loss parameter used across smooth functions for model estimation.

Read more about them in Chapter 11 of Svetunkov (2023).

The loss Parameter

The loss parameter specifies the loss (or cost) function minimized during model estimation.

Basic Usage

# R: Maximum likelihood estimation (default)
model <- adam(y, model="AAA", lags=12, loss="likelihood")

# R: Mean Squared Error
model <- adam(y, model="AAA", lags=12, loss="MSE")
# Python: Maximum likelihood (default)
model = ADAM(model="AAA", lags=12, loss="likelihood")

# Python: Mean Squared Error
model = ADAM(model="AAA", lags=12, loss="MSE")

Available Loss Functions

Standard Loss Functions

Loss Full Name Description Formula
"likelihood" Maximum Likelihood Default. Uses the distribution specified in distribution parameter Depends on distribution
"MSE" Mean Squared Error Minimizes average squared errors mean((y - fitted)^2)
"MAE" Mean Absolute Error Minimizes average absolute errors, robust to outliers mean(abs(y - fitted))
"HAM" Half Absolute Moment Focuses on small errors, super robust to outliers mean(sqrt(abs(y - fitted)))

Regularization Loss Functions

Loss Description Use Case
"LASSO" L1 regularization Shrink parameters toward zero, variable selection
"RIDGE" L2 regularization Shrink parameters, prevent overfitting

Read paper of Pritularga et al. (2023) to understand what LASSO/RIDGE mean in case of dynamic models.

When using LASSO/RIDGE, the lambda parameter controls regularization strength and can be passed via ...:

model <- adam(y, model="AAA", lags=12, loss="LASSO", lambda=0.1)
model = ADAM(model="AAA", lags=12, loss="LASSO", lambda_param=0.1)

lambda parameter is restricted with (0, 1) region, defining whether to give more weight to MSE or to the parameter shrinkage (closer to 1 => higher shrinkage).

Multi-step Loss Functions

Multi-step loss functions optimize the model based on forecast errors at horizon h, not just one-step-ahead errors. The h parameter must be specified.

Loss Full Name Description
"MSEh" h-step MSE Only uses h-steps ahead forecast error
"TMSE" Trace MSE Sum of MSE for horizons 1 to h
"GTMSE" Geometric Trace MSE Geometric mean of MSE across horizons
"MSCE" Mean Squared Cumulative Error MSE of cumulative forecasts

Read more about multistep losses and their effect on models in Svetunkov et al. (2023a).

# R: Optimize for 12-step ahead forecasts
model <- adam(y, model="AAA", lags=12, loss="GTMSE", h=12)
# Python: Multi-step optimization
model = ADAM(model="AAA", lags=12, loss="GTMSE", h=12)

Absolute and Half Analogues

For completeness, absolute and half-moment versions of multi-step losses exist:

Squared Absolute Half
MSEh MAEh HAMh
TMSE TMAE THAM
GTMSE GTMAE GTHAM
MSCE MACE CHAM

Custom Loss Functions

You can provide your own loss function. It must accept three parameters:

  • actual: Vector of actual values
  • fitted: Vector of fitted values
  • B: Vector of all estimated parameters

R Example

# Custom loss: Mean Absolute Percentage Error
customLoss <- function(actual, fitted, B) {
    return(mean(abs((actual - fitted) / actual)))
}

model <- adam(y, model="AAA", lags=12, loss=customLoss)

Python Example

import numpy as np

# Custom loss: Mean Absolute Percentage Error
def custom_loss(actual, fitted, B):
    return np.mean(np.abs((actual - fitted) / actual))

model = ADAM(model="AAA", lags=12, loss=custom_loss)

Important Notes

  1. Model selection: Model selection and combination work properly only for loss="likelihood". Other loss functions may produce suboptimal model selection.

  2. Information criteria: When loss!="likelihood", the log-likelihood and information criteria (AIC, BIC) are calculated based on the connection between losses and distributions (if one exists). e.g. in case of loss="MAE", distribution is assumed to be Laplace. Only works for ADAM.

  3. Multi-step loss: Always specify h when using multi-step loss functions.

  4. LASSO/RIDGE normalization: Variables are not normalized prior to estimation, but parameters are divided by the mean values of explanatory variables.

  5. Parameter uncertainty: Fisher Information can only be calculated in case of loss="likelihood". In all the other case, when using summary(), vcov(), confint(), reapply() or reforecast(), use bootstrap=TRUE. (Bootstrap inference is R-only; the Python vcov()/confint()/summary() are Fisher-Information based and do not accept a bootstrap argument.)

When to Use Each Loss

Loss Best For
"likelihood" General use, model selection, prediction intervals
"MSE" Minimizing squared errors, produces mean forecasts
"MAE" Robust estimation, minimised by median
"HAM" Super robust estimation
"GTMSE" Optimizing multi-horizon forecasts
"LASSO" Alternative to multistep losses, better controlled
Custom Domain-specific loss functions

References

See Also

Clone this wiki locally