EnTransformer

EnTransformer is a deep generative forecasting framework that integrates engression-a stochastic learning paradigm for modeling conditional distributions-with the sequence modeling capabilities of Transformers. By injecting stochastic noise into the model representation and optimizing an energy-based scoring objective, it directly learns the conditional predictive distribution without imposing restrictive parametric assumptions. This enables EnTransformer to generate coherent multivariate forecast trajectories while preserving Transformers' ability to capture long-range temporal dependencies and cross-series interactions. Evaluated on widely used benchmarks (Electricity, Traffic, Solar, Taxi, KDD-Cup, and Wikipedia), EnTransformer achieves competitive or improved probabilistic forecasting performance, yielding sharper, better-calibrated predictive distributions.

Keywords: Multivariate time series, Transformer, Probabilistic forecasting, Generative Modeling.

Installation

# Create a virtual environment (optional)
python -m venv venv
source venv/bin/activate

# Install requirements
pip install -r requirements.txt

Running the Experiments

We provide a main entry point script run.py that utilizes our modularized codebase. It draws parameters cleanly from config.ini preventing hardcoded execution and runs through one designated dataset at a time. The script sets up the deterministic environments internally, preprocesses the time series, performs a model training run, evaluates metrics via the MultiVariate Evaluator (GluonTS), and reports back CRPS metrics exactly mimicking the prior notebooks.

Execute run.py by parsing --dataset argument. To enable model checkpointing, pass the --save_checkpoints flag:

python run.py --dataset solar_nips --save_checkpoints

Result: Saving Architecture

When the training and evaluation successfully finish, the generated metrics (CRPS, ND, NRMSE) are printed to the console and automatically appended to a results.csv file in the root directory. This allows you to easily track and aggregate multiple experiments across different datasets and seeds.

You can change existing hyper-parameters such as noise standard deviation, number of samples, sequence models in the config.ini for each specific dataset structure.

Supported Datasets:

• solar_nips
• wiki2000_nips
• electricity_nips
• taxi_30min
• traffic_nips
• kdd_cup_2018_without_missing

The data processing pipelines are located in src/data.py, evaluations in src/metrics.py, models in src/models.py, and seeding logic resides inside src/utils.py.

Citation

If you find this code or our paper useful, please consider citing:

@article{pathak2026entransformer,
  title={EnTransformer: A Deep Generative Transformer for Multivariate Probabilistic Forecasting},
  author={Pathak, Rajdeep and Goswami, Rahul and Panja, Madhurima and Ghosh, Palash and Chakraborty, Tanujit},
  journal={Under Review},
  year={2026}
}