MG-TSD: Multi-Granularity Time Series Diffusion Models with Guided Learning Process
Diffusion probabilistic models have the capacity to generate high-fidelity samples for generative time series forecasting; however, they also present issues of instability due to their stochastic nature. In order to tackle this challenge, we present a novel approach called Multi-Granularity Time Series Diffusion (MG-TSD), which capitalizes on the intrinsic granularity levels present in the data, utilizing them as predefined targets at various stages of the diffusion process. These targets are used to guide the learning trajectory of the diffusion models, ensuring a more stable and accurate forecast.
In the field of long-term forecasting, the Multi-Granularity Time Series Diffusion (MG-TSD) model has established a new state-of-the-art, showcasing a notable relative improvement that spans from 4.7% to 35.8% across six benchmarks.
We observe that the forward process of the diffusion model, which sequentially corrupts the data distribution to a standard normal distribution, intuitively aligns with the process of smoothing fine-grained data into a coarser-grained representation, both of which result in a gradual loss of finer distribution features. This provides the insights that intrinsic features within data granularities may also serve as a source of guidance.
Figure1: The process of smoothing data from
finest-grained to coarsest-grained naturally aligns
with the diffusion process
In this paper, we propose a novel Multi-Granularity Time Series Diffusion (MG-TSD) model that leverages multiple granularity levels within data to guide the learning process of diffusion models. The coarse-grained data at different granularity levels are utilized as targets to guide the learning of the denoising process. These targets serve as constraints for the intermediate latent states, ensuring a regularized sampling path that preserves the trends and patterns within the coarse-grained data. They introduce inductive bias which promotes the generation of coarser features during intermediate steps and facilitates the recovery of finer features in subsequent diffusion steps. Consequently, this design reduces variability and results in high-quality predictions.
Figure2: Overview of the Multi-Granularity Time Series Diffusion (MG-TSD) model
conda create -n mgtsd python=3.9.12
conda activate mgtsd
pip install -r requirements.txt
bash scripts/run_mgtsd.sh
The detailed descriptions about the arguments are as following:
| Parameter Name | Description |
|---|---|
num_gran |
The number of granularity levels, which should correspond to the length of mg_dict. |
mg_dict |
A list defining multiple granularity intervals, e.g., '1_4' for 1 hour and 4 hours, '1_4_8' for 1, 4, and 8 hours. |
share_ratio_list |
A list detailing the share ratios; '1_0.9' indicates a 90% share of diffusion steps with the finest granularity. |
weight_list |
A list of weights for each granularity level; '0.9_0.1' assigns 0.9 to the first level and 0.1 to the second. |
log_metrics |
Toggle for logging metrics to Weights & Biases (wandb) during training, which may affect training speed. |
wandb_space |
The designated space name for wandb logging. |
wandb_key |
Your personal wandb access key (please replace with your actual key). |
Alternatively, you can also directly run the following command to train and test with different settings.
Setting 1: Perform experiments with 3 granularities on the Solar dataset.
python src/run_mgtsd.py \
--result_path "./" \
--model_name "mgtsd" \
--epoch 30 \
--cuda_num 0 \
--dataset "solar" \
--diff_steps 100 \
--batch_size 128 \
--num_cells 128 \
--mg_dict "1_4_12" \
--num_gran 3 \
--share_ratio_list "1_0.8_0.6" \
--weight_list "0.8_0.1_0.1" \
--run_num 1 \
--log_metrics False Setting 2: Perform experiments with 4 granularities on the Solar dataset and log the metrics to Weights & Biases (wandb) during the process.
python src/run_mgtsd.py \
--result_path "./" \
--model_name "mgtsd" \
--epoch 30 \
--cuda_num 0 \
--dataset "solar" \
--diff_steps 100 \
--batch_size 128 \
--num_cells 128 \
--mg_dict "1_4_12_24" \
--num_gran 4 \
--share_ratio_list "1_0.8_0.6_0.6" \
--weight_list "0.7_0.1_0.1_0.1" \
--run_num 1 \
--log_metrics True \
--wandb_space mgtsd \
--wandb_key your-personal-wandb-access-keysee src/example.ipynb for more information.
Run the scripts/run_mgtsd.sh and the data will be automatically downloaded to ~/.mxnet/gluon-ts/.
Our comprehensive evaluation across six benchmarks and three performance metrics involved a comparison with nine baseline models. The results demonstrate that our model secures the state-of-the-art (SOTA) status, achieving a substantial improvement ranging from 4.7% to 35.8% on the
In Figure 3(a)-(d), the dashed blue curve in each plot represent
The four subplots from (a) to (d) illustrate a gradual smoothing transformation of the distribution of increasingly coarser targets. A key observation is that from the left to the right panel, the distribution of coarser targets gradually aligns with the distribution of intermediate samples at larger diffusion steps. More specifically, as granularity transitions from fine to coarse (4h→6h→12h→24h), the denoising steps at which the distribution most resembles the coarse-grained targets increase (approximately at steps 80→60→40→40). This comparison underscores the similarity between the diffusion process and the smoothing process from the finest-grained data to coarse-grained data, both of which involve a gradual loss of finer characteristics from the finest-grained data through a smooth transformation.
Figure 3: Selection of share ratio for MG-TSD models
To illustrate the guidance effect of coarse-grained data, we visualize the ground truth and the predicted mean for both 1-hour and 4-hour granularity time series across four dimensions in the Solar dataset in Figure 4. In the MG-TSD model, the coarse-grained samples display a more robust capacity to capture the trends, subsequently guiding the generation of more precise fine-grained data.
Figure 4: MG-TSD and TimeGrad prediction intervals and test set ground-truth for Solar data of
some illustrative dimensions of 370 dimensions from first rolling-window.
If you find this repo useful, please cite our paper.
@inproceedings{
fan2024mgtsd,
title={{MG}-{TSD}: Multi-Granularity Time Series Diffusion Models with Guided Learning Process},
author={Xinyao Fan and Yueying Wu and Chang Xu and Yuhao Huang and Weiqing Liu and Jiang Bian},
booktitle={The Twelfth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=CZiY6OLktd}
}