Skip to content
master
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 

README.md

Non-Correlating Multiplicative Noise

This repository contains the code for the paper Removing the Feature Correlation Effect of Multiplicative Noise. Non-Correlating Multiplicative Noise (NCMN) exploits batch normalization to remove the feature correlation effect of multiplicative noise (e.g. dropout).

The code is based on a PyTorch implementation of wide residual networks.

Usage

For practical uses, NCMN-$0$ is simple, fast, and can be applied to any batch-normalized neural networks, while NCMN-2 yields better generalization performance on ResNets.

NCMN-0

## CIFAR-10
main.py --ncmn 0 0.35 --weightDecay 5e-6 --depth 22 --width 7.5 --dataroot ../cifar10 --save ./logs/resnet_model --ngpu 1 --gpu_id 0
## CIFAR-100
main.py --ncmn 0 0.25 --weightDecay 2e-5 --depth 22 --width 7.5 --dataset CIFAR100 --dataroot ../cifar100 --save ./logs/resnet_model --ngpu 1 --gpu_id 0

NCMN-1

## CIFAR-10
main.py --ncmn 1 0.35 --weightDecay 5e-6 --depth 22 --width 7.5 --dataroot ../cifar10 --save ./logs/resnet_model --ngpu 1 --gpu_id 0
## CIFAR-100
main.py --ncmn 1 0.25 --weightDecay 2e-4 --depth 22 --width 7.5 --dataset CIFAR100 --dataroot ../cifar100 --save ./logs/resnet_model --ngpu 1 --gpu_id 0

NCMN-2

## CIFAR-10
main.py --ncmn 2 0.4 --lr 0.03 --weightDecay 2e-5 --depth 22 --width 7.5 --dataroot ../cifar10 --save ./logs/resnet_model --ngpu 1 --gpu_id 0
main.py --ncmn 2 0.45 --lr 0.03 --weightDecay 2e-5 --depth 28 --width 10 --dataroot ../cifar10 --save ./logs/resnet_model --ngpu 1 --gpu_id 0
## CIFAR-100
main.py --ncmn 2 0.3 --weightDecay 2e-4 --depth 22 --width 7.5 --dataset CIFAR100 --dataroot ../cifar100 --save ./logs/resnet_model --ngpu 1 --gpu_id 0
main.py --ncmn 2 0.35 --weightDecay 2e-4 --depth 28 --width 10 --dataset CIFAR100 --dataroot ../cifar100 --save ./logs/resnet_model --ngpu 1 --gpu_id 0

About

A PyTorch implementation of non-correlating multiplicative noise (NCMN).

Resources

License

Releases

No releases published

Packages

No packages published

Languages