Accepted npj Artificial Intelligence paper: A Dynamic Fractional Generalized Deterministic Annealing for Rapid Convergence in Deep Learning Optimization
Installing dependencies:
Prerequisites:
- PyTorch
- torchvision
- numpy
- collections
Create a new Conda environment:
conda create -n DFGDA python=3.8
conda activate DFGDA
Install the libraries:
You can install the latest version of Pytorch/torchvision. We recommend installing this version to replicate our environment:
conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch
Preparing data:
First, download this repository and copy it to your desired directory.
Most datasets (MINST, CIFAR-10, USPS, and SVHN) will be automatically downloaded through the torchvision datasets library. For the MINST-M dataset, please download it from here and add the dataset root folder to the ./data/.
Running the models:
We provided the source codes to compare the proposed DFGDA with SGD based on all the datasets: MINST, CIFAR-10, USPS, SVHN, and MINST-M.
For the MINST dataset, run this command:
python DFGDA_MINST.py
For the CIFAR dataset, run this command:
DFGDA_CIFAR.py
For the USPS dataset, run this command:
DFGDA_USPS.py
For the SVHN dataset, run this command:
python DFGDA_SVHN.py
For the MINST-M dataset, run this command:
python DFGDA_MINSTM.py
After running each code, we will train with DFGDA first and SGD next. It will print both training and validation losses per epoch.