Skip to content

liuzechun/ReActNet

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 

ReActNet

This is the pytorch implementation of our paper "ReActNet: Towards Precise Binary NeuralNetwork with Generalized Activation Functions", published in ECCV 2020.

In this paper, we propose to generalize the traditional Sign and PReLU functions to RSign and RPReLU, which enable explicit learning of the distribution reshape and shift at near-zero extra cost. By adding simple learnable bias, ReActNet achieves 69.4% top-1 accuracy on Imagenet dataset with both weights and activations being binary, a near ResNet-level accuracy.

Citation

If you find our code useful for your research, please consider citing:

@inproceedings{liu2020reactnet,
  title={ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions},
  author={Liu, Zechun and Shen, Zhiqiang and Savvides, Marios and Cheng, Kwang-Ting},
  booktitle={European Conference on Computer Vision (ECCV)},
  year={2020}
}

Run

1. Requirements:

  • python3, pytorch 1.4.0, torchvision 0.5.0

2. Data:

  • Download ImageNet dataset

3. Steps to run:

(1) Step1: binarizing activations

  • Change directory to ./resnet/1_step1/ or ./mobilenet/1_step1/
  • run bash run.sh

(2) Step2: binarizing weights + activations

  • Change directory to ./resnet/2_step2/ or ./mobilenet/2_step2/
  • run bash run.sh

Models

Methods Top1-Acc FLOPs Trained Model
XNOR-Net 51.2% 1.67 x 10^8 -
Bi-Real Net 56.4% 1.63 x 10^8 -
Real-to-Binary 65.4% 1.83 x 10^8 -
ReActNet (Bi-Real based) 65.9% 1.63 x 10^8 Model-ReAct-ResNet
ReActNet-A 69.5% 0.87 x 10^8 Model-ReAct-MobileNet

Contact

Zechun Liu, HKUST (zliubq at connect.ust.hk)

Zhiqiang Shen, CMU (zhiqians at andrew.cmu.edu)

About

ReActNet: Towards Precise Binary NeuralNetwork with Generalized Activation Functions. In ECCV 2020.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published