End-to-end Optics Design for Computational Cameras
Imaging systems have long been designed in separated steps: the experience-driven optical design followed by sophisticated image processing. Such a general-propose approach achieves success in the past but left the question open for specific tasks and the best compromise between optics and post-processing, as well as minimizing costs. Driven from this, a series of works are proposed to bring the imaging system design into end-to-end fashion step by step, from joint optics design, PSF optimization, phase map optimization to a general end-to-end complex lens camera.
Overview
Abstract
Imaging systems have long been designed in separated steps: the experience-driven optical design followed by sophisticated image processing. Such a general-propose approach achieves success in the past but left the question open for specific tasks and the best compromise between optics and post-processing, as well as minimizing costs. Driven from this, a series of works are proposed to bring the imaging system design into end-to-end fashion step by step, from joint optics design, PSF optimization, phase map optimization to a general end-to-end complex lens camera.
To demonstrate the joint optics application with image recovery, we applied it to flat lens imaging with a large field of view. In applying a super-resolution SPAD camera, the PSF encoded by DOE is optimized together with the post-processing, which brings the optics design into the end-to-end stage. Expanding to color imaging, optimizing PSF to achieve DOE fails to find the best compromise between different wavelengths. Snapshot HDR imaging is achieved by optimizing a phase map directly. All works are demonstrated with prototypes and experiments in the real world.
To further compete for the blueprint of end-to-end camera design and break the limits of a simple wave optics model and a single lens surface. Finally, we propose a general end-to-end complex lens design framework enabled by a differentiable ray tracing image formation model. All works are demonstrated with prototypes and experiments in the real world. Our frameworks offer competitive alternatives for the design of modern imaging systems and several challenging imaging applications.
Brief Biography
Qilin Sun received the BSc degree in Optical Information Science and Technology from Huazhong University of Science&Technology(HUST). Later he joined the Visual Computing Center, King Abdullah University of Science and Technology as an MS/Ph.D. student in Electrical and Computer Engineering in 2015. Currently, he is pursuing his Ph.D. degree with a focus on computational imaging, in particular, camera design through end-to-end optimization of optics, illumination, sensors, and post-processing.
