Skip to content

oshapio/Efficient-Cross-Validation-of-Echo-State-Networks

master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 

Efficient Implementations of Echo State Network Cross-Validation

Echo State Networks (ESNs) as a prime example of Reservoir Computing (RC) models are known for their fast and precise one-shot learning of time series. But they often need good hyper-parameter tuning for the best performance. For this good validation is key, but usually a single validation split is used.

This repository contains the code used for the experiments in [1].

1. Summary of best complexities of the different CV methods

2. Precision Experiments

Some details of implementation and computational savings depend on what type of task we are learning. Let us distinguish three types of temporal machine learning tasks:

  1. Generative tasks, where the computed output y(n) comes back as (part of) the input u(n + k). This is often pattern generation or multi-step timeseries prediction in a generative mode.

  2. Output tasks, where the computed output time series y(n) does not comeback as part of input. This is often detection or recognition in time series, or deducing a signal from other contemporary signals.

  3. Classification tasks, of separate (pre-cut) finite sequences, where a class y is assigned to each sequence u(n).

For the latter type of tasks we usually store only an averaged or a fixed number of states x(n) for every sequence in the state matrix X. It is similar to a non-temporal classification task.

For each of the learning task we apply our approach to verify its effectiveness. The code for each task is similar, yet divided into separate files to make application to each task clear. As such, each script can be easily alterted to support other datasets with minimal change.

2.1. Generative Mode

Usage

Exact experiments reported in the paper can be replicated by running:

./run_generative_experiments.sh

This will start a job with 4 threads and search for optimal hyper-parameters with 5 different initializations. After the search ends, all of the results will be saved in precision_experiments/generative_mode/results folder. Results can be viewed by running show_results.py.

Alternatively, individual experiments, for example, can be run by executing:

python precision_experiments/generative_mode/generative_exp.py --dataset gasoline --valid_method standard --final_model retrain

Full range of options can be seen by adding --help argument.

Results

2.2. Time Series Output

Usage

Same structure was followed as in Generative Mode experiment, see how individual experiments were executed in run_output_exps.sh.

Results

2.3. Time Series Classification

Usage

Same structure was followed as in Generative Mode experiment, see how individual experiments were executed in run_classification_exps.sh.

Results

3. Speed Experiment Open In Colab

A notebook (ESN_speed_experiment.ipynb) is available to replicate the speed experiment. Alternatively, experiment can be run locally by clicking the Open in Colab button.

References

[1] Lukoševičius, Mantas, and Arnas Uselis. "Efficient implementations of echo state network cross-validation". 2020. [arxiv]

Contact

Please contact Arnas Uselis (auselis gmx com) if any issues regarding the code arise.

About

Code for "Efficient Implementations of Echo State Network Cross-Validation"

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published