Spin-Weighted Spherical CNNs

Abstract

Learning equivariant representations is a promising way to reduce sample and model complexity and improve the generalization performance of deep neural networks. The spherical CNNs are successful examples, producing SO(3)-equivariant representations of spherical inputs. There are two main types of spherical CNNs. The first type lifts the inputs to functions on the rotation group SO(3) and applies convolutions on the group, which are computationally expensive since SO(3) has one extra dimension. The second type applies convolutions directly on the sphere, which are limited to zonal (isotropic) filters, and thus have limited expressivity. In this paper, we present a new type of spherical CNN that allows anisotropic filters in an efficient way, without ever leaving the spherical domain. The key idea is to consider spin-weighted spherical functions, which were introduced in physics in the study of gravitational waves. These are complex-valued functions on the sphere whose phases change upon rotation. We define a convolution between spin-weighted functions and build a CNN based on it. Experiments show that our method outperforms the isotropic spherical CNNs while still being much more efficient than using SO(3) convolutions. The spin-weighted functions can also be interpreted as spherical vector fields, allowing applications to tasks where the inputs or outputs are vector fields.

Spherical image from vector field

Spherical vector field from image

Datasets

The data used for the spherical MNIST experiments is available here. It was created using this file from jonkhler/s2cnn.

Demo

A JAX implementation that reproduces the results of this paper can be found at this Google Research repo.

Reference

Carlos Esteves, Ameesh Makadia, Kostas Daniilidis. “Spin-Weighted Spherical CNNs”, arXiv’20. http://arxiv.org/abs/2006.10731v1

@article{esteves20_spin_weigh_spher_cnns,
  author =       {Esteves, Carlos and Makadia, Ameesh and Daniilidis,
                  Kostas},
  title =        {Spin-Weighted Spherical Cnns},
  journal =      {CoRR},
  year =         2020,
  url =          {http://arxiv.org/abs/2006.10731v1},
  archivePrefix ={arXiv},
  eprint =       {2006.10731},
  primaryClass = {cs.CV},
}

Authors

Carlos Esteves [1], Ameesh Makadia [2], Kostas Daniilidis [1]

[1] GRASP Laboratory, University of Pennsylvania

[2] Google Research