Truly Scale-Equivariant Deep Nets with Fourier Layers

[Project Page] [Paper (NeurIPS 2023)] [Quick Start Demo]

Md Ashiqur Rahman, Raymond A. Yeh

Department of Computer Science, Purdue University

Overview

This is the official implementation of "Truly Scale-Equivariant Deep Nets with Fourier Layers" accepted at NeurIPS 2023.

In computer vision, models must be able to adapt to changes in image resolution to effectively carry out tasks such as image segmentation; This is known as scale-equivariance. Recent works have made progress in developing scale-equivariant convolutional neural networks, e.g., through weight-sharing and kernel resizing. However, these networks are not truly scale-equivariant in practice. Specifically, they do not consider anti-aliasing as they formulate the down-scaling operation in the continuous domain. To address this shortcoming, we directly formulate down-scaling in the discrete domain with consideration of anti-aliasing. We then propose a novel architecture based on Fourier layers to achieve truly scale-equivariant deep nets, i.e., absolute zero equivariance-error. Following prior works, we test this model on MNIST-scale and STL-10 datasets. Our proposed model achieves competitive classification performance while maintaining zero equivariance-error.

Setup Dependencies

For a full list of requirements, please refer to Scale_Equivarinat_Fourier_Layer/requirements.txt. To install the dependencies, please execute:

pip install -r requirements.txt

Demo

A notebook containing a demonstration of the scale equivariant layer and their uses in Deep neural networks is available in the notebook demo_and_quickstart.ipynb. The notebook can also be executed on Google Colab by following the link

Running Tests

To run the equivarinace test of each layer, run the following command

cd si_neural_opt_code
nose2

Running the Models

Steps:

• Download the project
• Update the model flag to select desired model
• Update the project_name flag to match the Neptune project
• Update the data_path to the dataset loaction.

The execute the following commands

python3 train_script GPU_ID

train_script: train_1d.py, train_mnist.py, train_stl.py

GPU_ID: int, device id of the GPU to train on