Autoencoders

한국어 버전의 설명은 여기를 참고하시기 바랍니다.

Introduction

An autoencoder is a model used for manifold learning that extracts meaningful latent variables to compress or reduce the dimensionality of the data through the process of compressing and reconstructing the original data. In this code, you can find implementations of three types of autoencoders. For the MNIST dataset, it also provides functionality to visualize the latent variables resulting from the trained model in 2D using t-SNE algorithm. For an explanation of autoencoders, refer to Autoencoder (오토인코더), and for information on t-SNE and UMAP, refer to t-SNE, UMAP.

Supported Models

Vanilla Autoencoder (AE)

• A vanilla autoencoder using nn.Linear is implemented.

Convolutional Autoencoder (CAE)

• A convolutional autoencoder using nn.Conv2d and nn.ConvTranspose2d is implemented. When you want to improve model performance, you can use CAE instead of AE.

Denoising Autoencoder (DAE)

• For the two models introduced above, you can train a denoising autoencoder by adding noise to the data. Denoising autoencoders can be used to extract more meaningful latent variables from the data.
  
  
  

Base Dataset

• Base dataset for tutorial is MNIST.
• Custom datasets can also be used by setting the path in the config/config.yaml. However, implementing a custom dataloader may require additional coding work in src/utils/data_utils.py.
  
  
  

Supported Devices

• CPU, GPU, multi-GPU (DDP), MPS (for Mac and torch>=1.12.0)
  
  
  

Quick Start

python3 src/run/train.py --config config/config.yaml --mode train

Project Tree

This repository is structured as follows.

├── configs                     <- Folder for storing config files
│   └── *.yaml
│
└── src      
    ├── models
    |   ├── autoencoder.py      <- Valilla autoencoder model file
    |   └── conv_autoencoder.py <- Convolutional autoencoder model file
    |
    ├── run                   
    |   ├── train.py            <- Training execution file
    |   ├── tsne_test.py        <- Trained model t-SNE visualization execuation file
    |   └── validation.py       <- Trained model evaulation execution file
    | 
    ├── tools                   
    |   ├── model_manager.py          
    |   └── training_logger.py  <- Training logger class file
    |
    ├── trainer                 
    |   ├── build.py            <- Codes for initializing dataset, dataloader, etc.
    |   └── trainer.py          <- Class for training, evaluating, and visualizing with t-SNE
    |
    └── uitls                   
        ├── __init__.py         <- File for initializing the logger, versioning, etc.
        ├── data_utils.py       <- File defining the custom dataset dataloader
        ├── filesys_utils.py       
        └── training_utils.py     

Tutorials & Documentations

Please follow the steps below to train the autoencoder.

1. Getting Started
2. Data Preparation
3. Training
4. ETC
   • Evaluation
   • t-SNE Visualization

Training Results

• Results of AE and DAE
  
  
  
  
• Latent space visualization via t-SNE