Light field photography is a technology borrowed from the field of computer graphics and applied to imaging. The original goal of this technology which was developed at the end of the 1990s was to perform a photo-realistic image synthesis of real objects using newly selected imaging parameters but without underlying complex mathematical models. To achieve this objective, image-based interpolation processes were developed which could generate new images from any viewpoint, based on a discrete number of object views from different perspectives. These technologies, along with image-based illumination, have revolutionized film technology. These days you can see the results in virtual special effects which can barely be distinguished from real images captured on film.
We now understand that the image-based graphic processes actually constitute subaperture photographic processes. Using this combination, algorithms and ideas were transfered from areas such as signal processing in radio astronomy, e.g. aperture coding and synthesis, so that today we have effective photographic and synthesis processes.
In practice, this enables light field photography to correct the traditional physical parameters of an image e.g. aperture size, focal plane, depth of field and, to a certain degree, the capture position after the photo has been taken. The last of the aforementioned possibilities can be used e.g. for purely digital image stabilization without movable parts in video recordings.
And what does this have to do with microscopy? We should add here that light field processes have already been tested in microscopy. The results are promising, especially for capturing dynamic samples because it is possible to refocus in post-editing– and thus perform the critical selection of the correct focal plane. In other words: there is ‘more’ data available. Unfortunately, ‘more’ data currently also means the unacceptably high loss of spatial resolution. The ability to reconstruct 3D information provides additional benefits. In the future, we can expect a clear increase in performance because 3D reconstruction and algorithmic super-resolution go hand in hand.
Over the past years, computational imaging has become a research area which enjoys strong growth and enormous application potential. The driving factors will be the growth of pixel numbers beyond the directly acceptable degree of spatial resolution and the simultaneous growth of computer capacities. The latter will make it possible to solve the extremely time-consuming inverse problems resulting from computational imaging in a reasonable amount of time. Both factors are based on the continued integrative growth of microelectronics which will ultimately be possible through continued innovation in the field of lithography. In computational imaging, we might say that projection optics advance imaging optics by taking a detour through electronics, algorithmics and mathematics.
Lars Omlor and Ivo Ihrke
Atcheson, B., Ihrke, I., Heidrich, W., Tevs, A., Bradley, D., Magnor, M., &amp; Seidel, H. P. (2008, December). Time-resolved 3d capture of non-stationary gas flows. In ACM transactions on graphics (TOG) (Vol. 27, No. 5, p. 132). ACM.
M. L. Faulkner and J. M. Rodenburg. Movable aperture lensless transmission microscopy: A novel phase retrieval algorithm. Phys. Rev. Lett., 93:023903, 2004.
Levoy, M., &amp; Hanrahan, P. (1996, August). Light field rendering. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques (pp. 31-42). ACM.
Levoy, M., Ng, R., Adams, A., Footer, M., &amp; Horowitz, M. (2006). Light field microscopy. ACM Transactions on Graphics (TOG), 25(3), 924-934.
Mignard-Debise, L., &amp; Ihrke, I. (2015, October). Light-field Microscopy with a Consumer Light-field Camera. In 3D Vision (3DV), 2015 International Conference on (pp. 335-343). IEEE.
Tian, J. Wang, and L. Waller, “3D differential phase-contrast microscopy with computational illumination using an LED array,” Optics Letters, vol. 39, no. 5, pp. 1326–1329, March 2014.