In this paper we present a method for intrinsic image decomposition of a scene into its reflectance and shading components. Our method works on focal stacks which are sets of differently focused images of the scene captured at varying focal distances. We use a robust focus measure and generalized random walk algorithm to compute dense probability maps across the stack. These maps are then used to define sparse local and global pixel neighbourhoods adhering to the structure of the underlying 3D scene. We use these neighbourhood correspondences with standard chromaticity assumptions as constraints in an optimization system to obtain the two intrinsic components. We present a novel method for estimating intrinsic images for any wild scene without any restrictions on the complexity, illumination or scale of the image. Furthermore we affirm the hypothesis that focus measures can be used as a proxy for scene depth for computer vision applications. We present our results on both indoor and outdoor scenes using manually captured stacks of random objects under natural and artificial lighting conditions. We also test our system on a larger dataset of synthetically generated focal stacks from NYUv2 and MPI Sintel datasets and show comparable performance against current state-of-the-art depth based IID methods.
Paper : Intrinsic image decomposition using focal stacks , ICVGIP 2016 Paper link : http://dl.acm.org/citation.cfm?id=3010046
This software is influenced by and borrows parts from the code by the Jeon et al's implementation of 'Intrinsic image decomposition using structure texture separation and surface normals ECCV 2014' (https://link.springer.com/chapter/10.1007/978-3-319-10584-0_15 and https://github.com/JunhoJeon/intrinsic_texture )
The software has been developed using Ubuntu 14.04, opencv2, ANN1.2.2, Matlab R2014a
LIBS folder has :
- aif_MRF : All-in-focus image generation from fstack images (Usage : ./labelFocalStack #NumberOfStackImages ). Recommended over all-in-focus generated by 'getFocusPmaps.m'
- rp-master : Randomized Prim's algorithm for 'objectness' proposal generation (https://github.com/smanenfr/rp)
- segment : Felzenszwalb's superpixeling code
- GRWfusion : Generalized Random Walk image fusion implementation
- features : Geometric Context features and denseSiftFeatures computation code
- RGF : RollingGuidanceFilter for smoothing
USAGE :
- Change tmp diectory path in ./libs/features/GeometricContext/src/im2superpixels_1.m
- Compile mex files in ./mex folder (Requires opencv linking and ANN linrary 'http://www.cs.umd.edu/~mount/ANN')
- see fstackIID.m for usage (try changing the tol=0.5, 0.1, 0.05 or gamma3=0.05, 0.1, 0.5 etc) if the method is not converging. This might happen sometimes due to instability of graph weights)
The code is quite unoptimized and hence slow. It can be improved specifically by computing Laplacian, feature variances using mex.
@inproceedings{Saini:2016:IID:3009977.3010046,
author = {Saini, Saurabh and Sakurikar, Parikshit and Narayanan, P J},
title = {Intrinsic Image Decomposition Using Focal Stacks},
booktitle = {Proceedings of the Tenth Indian Conference on Computer Vision, Graphics and Image Processing},
series = {ICVGIP '16},
year = {2016},
pages = {88:1--88:8},
articleno = {88},
numpages = {8},
doi = {10.1145/3009977.3010046},
}