Sparse Convolutions on Continuous Domains

This repository provides instructions on running experiments for the paper Sparse Convolutions on Continuous Domains forPoint Cloud and Event Stream Networks (ACCV 2020).

• ACCV 2020 Paper
• Spotlight Video (1min)
• Oral Presentation (9min)

@InProceedings{Jack_2020_ACCV,
    author    = {Jack, Dominic and Maire, Frederic and Denman, Simon and Eriksson, Anders},
    title     = {Sparse Convolutions on Continuous Domains for Point Cloud and Event Stream Networks},
    booktitle = {Proceedings of the Asian Conference on Computer Vision (ACCV)},
    month     = {November},
    year      = {2020}
}

The paper investigates two applications based on the same theoretical convolution operator. These are implemented in separate repositories. See the respective READMEs for instructions on installation / experimentation details.

• Point Cloud Networks: pcn
• Event Cloud Networks: ecn

requirements.txt contains exact versions of packages to aid installation. For development, you may wish to clone the repositories and as below.

# shared pip dependencies
pip install tensorflow  # or tf-nightly
pip install absl-py gin-config setproctitle scipy uuid psutil pyyaml numba

# shared dependencies
git clone https://github.com/jackd/kblocks.git
git clone https://github.com/jackd/meta-model.git
git clone https://github.com/jackd/tfrng.git
git clone https://github.com/jackd/wtftf.git
pip install --no-deps -e kblocks
pip install -e meta-model
pip install -e tfrng
pip install -e wtftf

# for point cloud models
git clone https://github.com/jackd/pcn.git
git clone https://github.com/jackd/shape-tfds.git
git clone https://github.com/jackd/numba-neighbors.git
pip install --no-deps -e pcn
pip install -e shape-tfds
pip install -e numba-neighbors

# for event streams
git clone https://github.com/jackd/ecn.git
git clone https://github.com/jackd/events-tfds.git
git clone https://github.com/jackd/numba-stream.git
pip install --no-deps -e ecn
pip install -e events-tfds
pip install -e numba-stream

Project Dependencies

To assist code reuse, this project has been broken into many sub-repositories.

• shared dependencies:
  • kblocks: dependency-injectable keras blocks
  • meta-model: framework for building multiple networks simultaneously for models with architecture-dependent data pipelining
  • tfrng: a uniform interface for tensorflow random number generation implementations and custom dataset map implementation
  • wtftf: keras layer wrappers that will be removed once a stable tensorflow 2.4 is released
• Point convolution specific:
  • pcn: point cloud convolution implementations and training configs
  • numba-neighbors: optimized nearest-neighbors implementations with numba
  • shape-tfds: tensorflow-datasets implementations of various shape datasets
• Event convolution specific:
  • ecn: event convolution implementations and training configs
  • numba-stream: optimized implementations of event stream preprocessing operations with numba
  • events-tfds: tensorflow-datasets implementations of various shape datasets