Skip to content
Positional Normalization (PONO) and Moment Shortcut (MS)
Branch: master
Clone or download
Latest commit 5cc0715 Jul 10, 2019
Type Name Latest commit message Commit time
Failed to load latest commit information.
figs Add files Jul 10, 2019
.gitignore Add files Jul 10, 2019
LICENSE Add files Jul 10, 2019 Update Jul 11, 2019 Add files Jul 10, 2019 Add files Jul 10, 2019

Positional Normalization


*: Equal Contribution


The repo contains TensorFlow and PyTorch Implementation of Positional Normalization (PONO) and Moment Shortcut (MS), described in the paper Positional Normalization.

This image illustrates how PONO differs from some popular nomalization methods: In the figure, each subplot shows a feature map tensor, with B as the batch axis, C as the channel axis, and (H, W) as the spatial axis. The entries colored in green or blue (ours) are normalized by the same mean and standard deviation. Unlike previous methods, our method processes each position independently.

It's suggested to put PONOs right after convolution operations in the encoder (former layers) and MSs right before convolution operations in the decoder (latter layers). One may use a ConvNet taking μ and σ extracted from PONO as inputs to generate β and γ for MS, which we refer as Dynamic Moment Shortcut (DMS). Here we show the illustration.

More information will be updated.

If you find this repo useful, please cite:

  title={Positional Normalization},
  author={Li, Boyi and Wu, Felix and Weinberger, Kilian Q. and Belongie, Serge},
  journal={arXiv preprint arXiv:1907.04312},
You can’t perform that action at this time.