Overview

This repository provides R Source codes to reproduce numerical experiments in the following arXiv preprint:

@article{okuno2023HOVR,
    year      = {2023},
    publisher = {CoRR},
    volume    = {},
    number    = {},
    pages     = {},
    author    = {Akifumi Okuno},
    title     = {A stochastic optimization approach to train non-linear neural networks with a higher-order variation regularization},
    journal   = {arXiv preprint arXiv:2308.02293}
}

Main scripts

0_demonstration.R

You can train a single neural network with the proposed stochastic algorithm. You can replace the training data (x,y) and the optimization settings and the number of hidden units (stored in the ``constants'' variable) to explore our regularization!

1_illustration.R

This script provides illustration figures of the neural networks trained by several regularizations. Results are stored in the automatically generated ``A2_computed'' folder.

2_experiments.R

This script provides experimental results (predictive correlation) with several random seeds. Results are stored in the automatically generated ``A2_computed'' folder.

Verbose

A0_scripts/gen_data.R

1000 pairs of (x,y) following (i) linear model, (ii) quadratic model, and (iii) cubic functions, are generated. The generated instances are saved to the automatically generated ``A1_data'' folder.

A0_scripts/functions.R

This script provides functions describing neural networks and stochastic algorithms.

Contact info.

Akifumi Okuno (okuno@ism.ac.jp)