Skip to content
Switch branches/tags

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

STCN: Stochastic Temporal Convolutional Networks

This repository contains the implementation of STCN: Stochastic Temporal Convolutional Networks paper.

The model is implemented in Python 3.5 by using Tensorflow library. Particularly the experiments were conducted with cuda 9, cudnn 7.2 and Tensorflow 1.10.1. Along with the implementation, we release our pretrained models achieving the state-of-the-art performance. You can reproduce the numbers and results by using the evaluation scripts explained below.


Python dependencies are listed in pip_packages file. You can install the packages by running pip install -r pip_packages command.


You should download the Blizzard, TIMIT and IAM-OnDB datasets. We compiled the preprocessing steps applied in the previous works. You can find the scripts for Blizzard and TIMIT dataset in experiments_speech directory. We will provide the script for IAM-OnDB handwriting dataset soon. The Deepwriting dataset is already provided.

The instructions are provided in and files. After you download the corresponding dataset, you just need to set the DATA_PATH variable in the code file and run the script.

After the preprocessing steps are applied, the data is transformed into a numpy representation required by our training pipeline. It is basically a dictionary with samples, targets and data statistics such as mean and variance. It can be inspected by loading via numpy.load.


We use configuration files to pass model and experiment parameters. You can find the configuration files in experiments_x/config_y folders. They are stored in JSON format. The provided configurations are the same with what we used. However, due to the randomness you may get better or worse results within a small margin.

You can train a model by passing a JSON configuration file. Most entries in the configuration are self-explanatory, and the rest is described in the code (see soruce/ model constructors). You should also have a look into source/ file which created command-line arguments and parses JSON configuration files. For example the following command runs STCN-dense model with GMM outputs on Deepwriting dataset.

--experiment_name stcn_dense_gmm 
--json_file ./config_deepwriting/stcn_dense_gmm.json 
--training_data [PATH-TO]/deepwriting_v2_training.npz 
--validation_data [PATH-TO]/deepwriting_v2_validation.npz 
--save_dir [PATH-TO]/runs 
--eval_dir [PATH-TO]/evaluation_runs 

We provide example commands for other datasets in the respective file. Please check it out since the normalization flags may differ. We apply early stopping on the validation ELBO performance. When training ends, the evaluation script is called if the run_evaluation_after_training flag is passed. You can evaluate a pre-trained model by using the following command:

--seed [if not passed, training seed is used]
--model_id [Unique timestamp ID in the experiment folder name. Something like 1549805923]
--save_dir [PATH-TO]/runs
--eval_dir [PATH-TO]/evaluation_runs
--test_data <PATH-TO>/blizzard_stcn_test.npz

Similarly, our models can be downloaded and evaluated by just passsing the unique model ID.

If you have questions on the code, feel free to create a Github issue or contact me.


If you use this code or dataset in your research, please cite us as follows:

  title={{STCN}: Stochastic Temporal Convolutional Networks},
  author={Emre Aksan and Otmar Hilliges},
  booktitle={International Conference on Learning Representations},


STCN: Stochastic Temporal Convolutional Networks





No releases published


No packages published