Skip to content
This is an implementation of Shearlet Transform (ST) for light field reconstruction using TensorFlow.
Python
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
Fig new complete dataflow graph Apr 8, 2019
data/ssepi/dishes_r5 add model.py and data.py Jun 20, 2019
model
LICENSE Initial commit Apr 6, 2019
README.md add poster to readme Jun 21, 2019
data.py add model.py and data.py Jun 20, 2019
model.py add model.py and data.py Jun 20, 2019
validate.py add model.py and data.py Jun 20, 2019

README.md

Shearlet-Transform-TensorFlow

This is an implementation of Shearlet Transform (ST) [1, 2] for light field reconstruction using TensorFlow. If you find this code useful in your research, please consider citing [1, 2] and

@inproceedings{tfst2019,
Author = {Gao, Yuan and Koch, Reinhard and Bregovic, Robert and Gotchev, Atanas},
Title = {Light Field Reconstruction Using Shearlet Transform in TensorFlow},
Booktitle = {ICME Workshops},
Year = {2019}
}

This code was tested on an Ubuntu 18.04 system using Tensorflow 1.13.1. Here is the associated poster presented at ICME 2019 Demo/Expo.

Introduction

ST is designed for reconstructing a Densely-Sampled Light Field (DSLF) from a Sparsely-Sampled Light Field (SSLF). It typically consists of pre-shearing, shearlet system construction, sparsity regularization and post-shearing. This TensorFlow implementation of ST focuses on sparsity regularization, which is composed of analysis transform, hard thresholding, synthesis transform and double overrelaxation. A dataflow graph of these four components are illustrated as below:

alt text

A demo video of iterative sparse regularization with 30 iterations is shown as below:

IMAGE ALT TEXT HERE

Getting started

Python requirements

conda install tensorflow-gpu
conda install -c conda-forge opencv

Prepare datasets

Prepare the pre-sheared sparsely-sampled Epipolar-Plane Images (EPIs) and masks. Put them into folders like

./data/ssepi/dishes_r5

and name them like

0001_rgb.png, 0002_rgb.png, ...
0001_mask.png, 0002_mask.png, ...

For example, "0458_rgb.png" and "0458_mask.png" are presented as follows:

alt text

alt text

Sparsity Regularization

python validate.py --validate_path=./data/ssepi/ --save_path=./data/rec_dsepi --batch_size=4 --tensorboard_path=./tensorboard --shearlet_system_path=./model

The reconstructed EPI corresponding to "0458_rgb.png" is presented as follows:

alt text

Note that the shearlet system for the pre-sheared sparsely-sampled EPIs should be prepared in advance. It is placed in the folder "./model" by default. How to generate a specially-tailored shearlet system can be found in this repository.

Visualization

The visualization of the pipline of ST is performed using TensorBoard:

tensorboard --logdir=./tensorboard

Then visit

http://localhost:6006

The dataflow graph is like

alt text

Intermediate results are like

alt text

Reference

[1] S. Vagharshakyan, R. Bregovic, and A. Gotchev, “Light field reconstruction using shearlet transform,” IEEE TPAMI, vol. 40, no. 1, pp. 133–147, 2018.

[2] S. Vagharshakyan, R. Bregovic, and A. Gotchev, “Accelerated shearlet-domain light field reconstruction,” IEEE J-STSP, vol. 11, no. 7, pp. 1082–1091, 2017.

You can’t perform that action at this time.