Seq2VAR: multivariate time series representation with relational neural networks and linear autoregressive model

This repository presents the code for our paper presented at the AALTD workshop of ECML/PKDD 2019 and appearing in the Springer Lecture Notes in Artificial Intelligence proceedings 11986.

Notebooks "Seq2VAR_permutation.ipynb" and "Seq2VAR_homogeneous_springs.ipynb" contain the code for paper reproduction. Notebook "RNN_overfitting.ipynb" contains results for part 4.1

All the code for the comparison with NRI (Kipf, 2018) directly comes from the Github page of the original paper.

Abstract

Finding understandable and meaningful feature representation of multivariate time series (MTS) is a difficult task, since information is entangled both in temporal and spatial dimensions. In particular, MTS can be seen as the observation of simultaneous causal interactions between dynamical variables. Standard way to model these interactions is the vector linear autoregression (VAR). The parameters of VAR models can be used as MTS feature representation. Yet, VAR cannot generalize on new samples, hence independent VAR models must be trained to represent different MTS. In this paper, we propose to use the inference capacity of neural networks to overpass this limit. We propose to associate a relational neural network to a VAR generative model to form an encoder-decoder of MTS. The model is denoted Seq2VAR for Sequence-to-VAR. We use recent advances in relational neural network to build our MTS encoder by explicitly modeling interactions between variables of MTS samples. We also propose to leverage reparametrization tricks for binomial sampling in neural networks in order to build a sparse version of Seq2VAR and find back the notion of Granger causality defined in sparse VAR models. We illustrate the interest of our approach through experiments on synthetic datasets.

Citing

@article{pineau2019seq2var,
         title={Seq2VAR: multivariate time series representation with relational neural networks and linear autoregressive model},
         author={Pineau, Edouard and Razakarivony, Sebastien and Bonald, Thomas},
         journal={Advanced Analytics and Learning on Temporal Data},
         year={2019},
         volume={11986},
         publisher={Springer}
}