Differentiable Imaging: a new tool for computational optical imaging

Advanced Physics Research, 2023

Ni Chen¹, Liangcai Cao², T.-C. Poon³, Byoungho Lee⁴, Edmund Y. Lam⁵

Computational imaging has made significant advancements in recent years, but faces limitations due to the restrictions imposed by traditional computational techniques. Differentiable programming offers a promising solution by combining the strengths of classical optimization and deep learning. By integrating physics into the modeling process, differentiable imaging - which employs differentiable programming in computational imaging - has the potential to overcome challenges posed by sparse, incomplete, and noisy data. This could be a key factor in advancing the field of computational imaging and its various applications.

Modeling the physics accurately is of great importance in differentiable imaging. Since many computational imaging techniques rely on ray tracing and diffraction , we have developed both a ray-tracing differentiable framework and a diffraction-based differentiable framework. These frameworks have applications in self-calibration, end-to-end lens design, metrology , holography (∂H), Phase Retrieval, Fourier Ptychographic Microscopy and more. The differentiable ray-tracing framework has proven to be highly efficient and enables easy incorporation of neural networks, thereby promoting lens design.

Please refer to the paper to find how this differentiable imaging enables more efficient and effective imaging, and our insights on its potential impact in the computational imaging field.